Software [In]security: BSIMM Begin
Common Activities for Getting Started with Software Security
The Building Security In Maturity Model (BSIMM) was released in March 2009 with much fanfare. Since March, the BSIMM has evolved and expanded in several ways. Most importantly, the BSIMM study has added data for fourteen companies to the original nine, bringing the study total to twenty-three (with three further efforts underway as this article goes to press). These data indicate the model as originally devised is robust enough to retain its utility well into the future.
The original nine include Adobe, The Depository Trust & Clearing Corporation (DTCC), EMC, Google, Microsoft, QUALCOMM, Wells Fargo and two un-named financial institutions. The new data include a number of companies that are also household names in verticals branching from ISVs and financial services into insurance and pharmaceuticals. Later this year, these data will be released under the Creative Commons as BSIMM II.
BSIMM Europe, which will be systematically covered in a future column, is a study of nine large-scale European software security initiatives. Comparing the European market for software security tools and services to the US market has traditionally involved some guesswork (see, for example, Software Security Comes of Age). Data as gathered and reported in BSIMM Europe will shed plenty of light on the complexities of the real situation.
BSIMM progress in the form of BSIMM II and BSIMM Europe is particularly good news for the observation-based model, which is based directly on hard data observed from the field. The more data we gather, the more we can say with confidence about the state of software security in the world. We're looking forward to the time (coming soon) when our data set reaches a size where statistically significant trends can be measured and reported.
At this point in the study we are interested in increasing the number of observations covering software security initiatives that are just getting started. To do that, we introduce BSIMM Begin, a Web-based study focused on 40 of the 110 activities covered in the full BSIMM.
Even if your organization is just getting started with a software security initiative, we hope that you will participate in the BSIMM Begin study yourself. Not only will you help make the study more thorough, you'll also come away with some idea of how your basic software security activities stack up against those practiced by others. Take the survey yourself at http://bsi-mm.com/begin. In fact, do what you can to get your friends and colleagues in other companies to take it too. The more data we gather the better off we'll all be.
Note that BSIMM Begin does not take the place of a full BSIMM assessment in any way. The full study focuses on activities that can be used to measure and compare fairly mature, large-scale software security initiatives. By contrast, BSIMM Begin focuses on new initiatives that are just getting off the ground. To make this clear, consider that the full BSIMM expects an organization to have a formalized a software security group (SSG) charged with carrying out or directing BSIMM activities. BSIMM Begin does not assume the existence of an SSG. In fact, we're interested in finding out who is carrying out various introductory software security activities without an SSG. The other major difference between the full BSIMM and BSIMM Begin is that BSIMM Begin will gather data in a self-reporting mode. BSIMM Begin data will be segregated in a separate set of results and analyzed accordingly.
Activities "Everybody Does" Updated
In the article originally titled Nine Things Everybody Does: Software security activities from the BSIMM, we laid out nine activities observed in the original BSIMM data that a large majority of the companies studied carried out. Here is a chart showing those common activities in terms of objectives. The "Observed" column shows the number of companies in the original BSIMM study where the activity was observed (of nine) as well as the number of companies in BSIMM II (of twenty-three). Note that one of the two level 2 activities in this list [T2.2] was observed the fewest times of all of these commonly-observed activities (and in fact now falls below our threshold for counting as a common activity). This is worth mentioning because BSIMM Begin will focus its attention primarily on level one activities — those activities that are the easiest to instantiate. Also worth noting is the fact that every company observed in the BSIMM study to date practices good network security. First things first!
|build support throughout organization||8 of 9
15 of 23
|[SM1.2] create evangelism role/internal marketing|
|meet regulatory needs or customer demand with a unified approach||
9 of 9
19 of 23
|[CP1.3] create policy|
|promote culture of security throughout the organization||
9 of 9
18 of 23
|[T1.1] provide awareness training|
|see yourself in the problem||
8 of 9
11 of 23
|[T2.2] create/use material specific to company history|
|create proactive security guidance around security features||
9 of 9
22 of 23
|[SFD1.1] build/publish security features (authentication, audit/log, crypto, ...)|
|build internal capability on security architecture||
8 of 9
15 of 23
|[AA1.3] have SSG lead review efforts|
|use encapsulated attacker perspective||
9 of 9
15 of 23
|[ST2.1] integrate black box security tools into the QA process|
|demonstrate that your organization's code needs help, too||
9 of 9
22 of 23
|[PT1.1] use external pen testers to find problems|
|provide a solid host/network foundation for software||
9 of 9
23 of 23
|[SE1.2] ensure host/network security basics in place|
In addition to the original "nine things everybody does," the data suggest a number of other basic activities that are commonly observed. Those activities are shown in the table below. The criterion for inclusion on this list is an "observed" number over fifteen (of twenty-three).
|establish SSDL gates (but do not enforce)||19 of 23||[SM1.4] identify gate locations, gather necessary artifacts|
|understand compliance drivers (FFIEC, GLBA, OCC, PCI, SOX, SAS 70, HIPAA, etc.)||18 of 23||[CP1.1] know all regulatory pressures and unify approach|
|promote privacy||19 of 23||[CP1.2] identify PII obligations|
|meet demand for security features||16 of 23||[SR1.1] create security standards (T: sec features/design)|
|get started with AA||15 of 23||[AA1.1] perform security feature review|
|review high-risk applications opportunistically||15 of 23||[CR1.2] have SSG perform ad hoc review|
|drive efficiency/consistency with automation||16 of 23||[CR2.1] use automated tools along with manual review|
|execute adversarial tests beyond functional||15 of 23||[ST1.1] ensure QA supports edge/boundary value condition testing|
|use ops data to change dev behavior||16 of 23||[CMVM1.2] identify software bugs found in ops monitoring and feed back to dev|
Of the twelve practices that make up the Software Security Framework, only the Attack Models practice has no commonly observed activities. This is an interesting phenomenon. We believe that the attacker's perspective and activities around the attacker's perspective may require more attention than we are observing in practice. Further, it appears that mature software security initiatives have a much better handle on this practice than new initiatives. As we analyze the BSIMM II data more thoroughly, we'll be able to explain this observation more clearly.
Level One Activities
In addition to probing the three level two activities in the seventeen most common activities outlined above, BSIMM Begin is structured to probe all 37 level one activities. This will provide us some understanding of how common the most basic software security activities are.
Using a Web-based survey, BSIMM Basic allows organizations to self assess. This is a significant shift from the standard data-gathering regimen followed during a full BSIMM analysis. To be perfectly blunt, we are not yet sure how the self-reporting paradigm will work. We designed the survey so that "making stuff up" is a non-trivial proposition, but it remains to be seen what kind of data can be gathered in this exercise.
As always, we intend to publish our data once they have been properly vetted and analyzed. We hope that the BSIMM Begin effort will broaden our understanding of small-, medium-, and large-sized businesses just getting started with software security.
Take the BSIMM Begin survey yourself at http://bsi-mm.com/begin.