Preserving Our Peer Processes
If consumers are to have measures of software quality to inform their acquisition choices, that information must come in some recognizable form. Rubber stamps, however, are exactly the wrong kind of packaging. If consumers are to continue to benefit from the strong independent peer-review process that open source exemplifies, the equivalent of a rubber stamp needs to be independently peer-reviewed as well. End-to-end transparency is the only possible solution, with no exceptions.
Like the stock market, the health system, the economy, and the environment, the information we need as software consumers is a set of technical indicators, not a set of rubber stamps. It's easy to produce indicators of software size, complexity, or quality. David A. Wheeler's website, for example, lists a number of independent statistics that reflect the quality of open source products according to a number of different definitions for "quality." There is no single, sanctified definition.
We need academics to argue what the leading indicators for software defects should be, and we need them to resist the special-interest arguments of those that would distort or avoid the statistics. We need independent bodies providing regimes, audits, and tests that can be used to produce indicators of anyone's software. We need the results transparently reported, and we need a graduated system that creates absolutely no barriers to entry, so that innovation is not stifled. After all, many people are happy to use software that's "Yet to be adequately completed: quality still low." That's the basis of many open source and commercial beta programs. Labels to that effect are surely overdue for many software products.
Such quality indicators are all rather awkward if you're a vendor that doesn't want to show anyone your source code, littered as it may be with the defects of intent that you put into it on purpose. They are a must for consumers, though. Alas, we know more about the contents of a jam jar than we do about the software we use.