The OpenBSD team’s main argument against binary drivers is that they harm system security. OpenBSD has a reputation for having first-rate security because every part of the system is under a constant process or code review.
If you run a piece of unaudited (or malicious) code in userspace on OpenBSD, some mechanisms are in place to attempt to mitigate vulnerabilities. Sometimes these features aren’t enough, and then it’s a matter of damage control. Fortunately, most services run with limited privileges. If a web application is compromised on an OpenBSD box, the attacker can’t touch anything that the web server couldn’t touch—usually limited to a very small set of tightly controlled data.
The situation is somewhat different if the compromised code is in kernelspace. By necessity, the kernel runs in the highest privilege mode, and therefore can do anything—access any device, inspect any bit of memory. Once an attacker has compromised the kernel, the only safe thing to do is erase the disk and reinstall.
This aspect recently became visible with the discovery of a security hole in a binary driver for a WiFi card that allows a machine to be compromised by any attacker within WiFi range (about 100 meters, in good conditions).