Home > Articles > Software Development & Management

An Interview with Watts Humphrey, Part 27: Formal Methods, the Failure of PSP, and Tragedy of Beautiful Software

  • Print
  • + Share This
In this transcript of an oral history, Grady Booch interviews SEI Fellow Watts Humphrey. In Part 27, Humphrey discusses his work with formal methods, why he had to move away from the PSP process, and why beautiful work in software does not get noticed.

This interview was provided courtesy of the Computer History Museum.

See the entire interview.

Like this article? We recommend

Formal Methods

Booch: A question for you with regards to the defect-free programming, because my recollection is that this was around the same time that Dr. Tony Hoare was talking about the notions of formalisms in development. Edsger W. Dijkstra of course as well around that time. And their thought was, you know, really formal specifications of the programs, but you were looking at an alternate view with regards to process. How did that fit in? What knowledge did you have of what they were up to?

Humphrey: I got fascinated with the formalisms and it was very attractive. I took a tutorial with Tony at ICSE, I had a group working in Germany while I was in my IBM quality process job. I used to go and visit with the people, particularly in Germany, because people there, they were doing marvelous work. A man named Horst Remus at IBM -- retired about the time I did -- introduced me to a bunch of these people, and they were academics they were working with in Germany, and they had these formal design methods. I also ran into some in the Netherlands that were doing that. So I was looking at that, the Zed language, if you’re familiar with that.

Booch: Very much, yes.

Humphrey: So I got interested in that, and I actually tried some of these methods on my own work, and I found it was quite helpful -- the discipline was helpful -- but I found the problem was I wasn’t that familiar with the formalisms, and that I tended to make more errors when I did it that way than I did otherwise. Part of the reason was that I wasn’t that good with the formalism stuff, so familiarity with the language was a real problem for me. But the other was that I then had to translate it into code, and I made a lot of errors there too. So my error rates actually went up rather significantly when I tried to do that, and I think part of it was my own problem. I think the formalism approaches will work at the individual level, and I think that’s useful. I don’t know if you’re familiar with the people with the SPARK programs at Praxis in England. Have you heard about those?

Booch: I have. There’s John. One of the early Ada guys was involved in that one.

Humphrey: Oh yeah. What they did is they took an Ada subset, and they subsetted Ada to make basically what they concluded was a precise language that didn’t have all of the misunderstandings and confusions in a lot of the languages. I actually went and took their course, their SPARK course, in England.

Booch: It was John Barnes. I think that was the guy still involved with it.

Humphrey: Well, I just got an e-mail from one of the guys the other day. They’ve now made their whole language and toolset available open source, which is marvelous. And they’re having good experience. One of their top engineers-- Rod Chapman I think is his name -- I went over there and he taught me and Noopur Davis, one of our people, and we had some other people in the class. He actually taught us the SPARK method, and we wrote some programs and went through and did it. It was an eye-opener. To identify all sources of buffer overflow is a hell of a challenge, and the mathematical methods are extremely helpful in doing that. So I found that I’m very impressed with those methods; I think they’re very powerful. And Rod Chapman actually came over and got trained as a PSP instructor, a TSP coach. So he’s qualified in the PSP and TSP. And Rod and I have discussed this at some length, and we both believe that the two methods are quite complementary.

They both address important but slightly different facets of the problem. And we both believe that the two methods really ought to get merged. One of the problems, of course, with SPARK is that it’s an Ada subset language, so that’s a problem. Later on, when I talk about the future, I want to talk about languages. So I agree, the formalism stuff is great. But what was amazing to me was we can get such an enormous level of improvement without it. And that’s what I’ve found is amazing, is we go through training engineers with the PSP, just gathering data on their defects, just looking at it, analyzing defects.

The reason it’s so extraordinary is that when you make a mistake-- and I tried to eliminate my mistakes, and I literally couldn’t do it. I found that there was no way that I could write programs without making errors. And the reason was that I was human, contrary to popular opinion, right? But there was no way I could literally do it. Because I was human, I tended to make various kinds of errors. But the thing that was interesting about it was I tended to make a particular category of errors. So I’d make them repetitively, and what surprised me was that if I gathered data on the kinds of mistakes I had made and used that data to put together a checklist to go back and review my code to find those errors, I got damn good at it. I got to where I could find most of the errors before I even compiled my program. I didn’t get to where I could find all of them. I did occasionally. But I find really good programmers do get that good.

The Yield Measure

What we’ve got in the PSP is the measure called “yield,” and you have the yield data. What percentage of the defects do you find in compile, and what percent do you find in the testing, what percent do you find in inspection or a code review, and that sort of thing. What I’ve found as I’ve looked at the data, on code review yields, with a lot of work, you can get up to 70 to 80 percent. I was typically getting close to 80 percent yields on my code reviews, which is pretty good. I find most people don’t even bother doing a code review, so most of the defects just slip through. You find also compiler yields are typically in the 50 percent range or less. I find a surprising number where the defects that you would classify as syntax defects, but they’re not really true syntax defects. It’s like a misspelled word that turns out to be a valid other word. So you make syntax defects that turn out to produce valid syntax. So it’s not really a syntax defect, but it’s that kind of a problem. It’s a typo; it’s a whatever. But it ends up producing valid code. It will compile.

So one of the things we try to convince people in the PSP course is that compilers or builders, whatever you got -- I know many people are getting away from compilers -- but whatever that tool is, its job is to try to take whatever you produce and make running code out of it. So it’s not really trying to find errors; it’s trying to produce code. So to count on any automated code-building tool like that to find errors for you is a mistake. Now, that isn’t true of things like the SPARK tools, which are designed to find errors, that really analyze it, and they’re looking at it. But that certainly is true of most of the code-building tools.

So that is really kind of a key question. You really have to focus on the quality of what the individual does. What I’ve found as part of that, that the various kinds of quality reviews, I do personal reviews. I review designs; I review the code. So I produce a program, for instance, and suppose I spend an hour writing the code. I force myself to spend at least 30 minutes going over the code, going over it carefully to try to find every error. And as I said, I can find 70 to 80 percent of them, or I could when I was writing programs a lot. So people can get pretty good at it. But what’s interesting is the defects I find personally when I do reviews, I’ll find them quickly, but I’ll find quickly a lot of the defects that would take you forever to find in tests.

Conversely, defects that are bloody hell to find individually pop up right away in tests. Like the blue screen -- all that stuff. It doesn’t work at all. Well, some of those things you completely miss -- the systemic-level stuff shows up right away in test, but you can’t see it. So there are a whole lot of issues like that that I find are kind of fascinating. And the thing that’s amazing to me is if you take and improve the quality of the code going into test by 10 times, you’d think you’d improve the quality coming out of test by 10 times. What’s amazing is you don’t. You improve it by close to 100 times. Sort of amazing.

So it takes a relatively modest investment, but we’re getting extraordinary improvements in quality by really addressing the individuals and getting them conscious of the quality of their personal work, and giving them data on it so they can look at it. So that’s what we did with the PSP, and that’s what I’ve found. So I taught the PSP course at CMU, and I think I mentioned that four of the people who took the course got so excited about it that they decided to change their lifetime careers, and I’ve got people working with me now that were in that course.

So it’s a transforming experience to go through this, and we have thousands of engineers who have now done it. And with very few exceptions, they come out of this saying, “Wow. That’s the way I want to work.” I’d say about, oh, 60-70 percent come about with a sort of “wow” feeling. Another 30-40 percent come out with an, “Well, okay, but I’m not sure I can do it on the job” feeling. And then there’s a small number of hardheads that really don’t buy it. But it’s kind of interesting. But in any event, so we got the PSP going, and we started to teach classes and bring people in to take courses.

I worked with Embry-Riddle University, and they gave me my very first experience with using the TSP. I put together the process and I talked to the people at Embry-Riddle who -- let me back up before I do that, because I ran into Embry-Riddle people at an SEI symposium. SEI would run a symposium every year. We’d bring in DoD people and industry people, et cetera, and we’d present our plans and what we were doing.

I was in one of these meetings; I made a presentation and I was sitting watching somebody else, and a guy came up and tapped me on the shoulder and said he wanted to talk to me. So I stepped out to see what he wanted. His name was Soheil Kajenuri. He was a professor at Embry-Riddle University, and he said, “We’re building a master’s curriculum at Embry-Riddle, and we’d like to build it around the CMM.” So he said, “I’d like you to talk to us.” I said, “Well, I’ll come out and talk to you. I can’t do it now, but I can come out.” We picked the time.

So he was there with their department chairman, Iraj Hirmanpour, and so I met with them. So I went through what they wanted to do, and they wanted to build a curriculum about the CMM. I said, “Well, basically, doing that’s fine, but you need to have the PSP in there.” So I described all of that, and they got all excited about it and said, “Yeah, that’s a good idea.” So they made the PSP the first required course in the graduate program. And Soheil taught a PSP course shortly thereafter, out of my textbook. I think the book was published by then, I believe, I don’t think it was the manuscript. Although it may still have been a manuscript. This was quite early.

So it was like 1995, ’94, something like that. Some of the first people writing programs in ’93-- that’s when Howie Dow did his. I taught in early ’94. That’s right. And then my discipline with PSP book, which I taught from in January through June of ’94, I was getting the class data, and then using that data to re-do the book. So I rewrote that book that spring and summer and got the manuscript to my publisher that fall, and it was published actually in December of ’94 with a ’95 copyright. By the way, a side point, I worked with Addison-Wesley, and Peter Gordon is my editor on the Managing the Software Process book, the very beginning. He’s been my editor now on nine books. Early on, I used the PSP to write my books. And he told me, he said he’s never had an author who makes every schedule. And I have. I mean, I’ll go through it, I’ll make a plan, and I’ll meet it. Things don’t come out to be completely predictable. I end up completely rewriting books and that sort of thing. My Winning with Software book this just didn’t work on, so I ended up throwing it away and rewriting it three times. So I was predictable on each version. But in any event, so I had the book, Soheil taught from it, and as I say, it was working great until the people at Embry-Riddle started trying to teach local industry people. They got some companies that were interested and they were teaching there, and they had several courses at like Boeing, Motorola, and other places. And I kept going to check, and I discovered that even though they had the course, and the students came out all excited about it, and they’d start using it on the job, it died out very quickly.

So PSP use did not persist. There are various reasons, but fundamentally, the reason was that the engineers said -- I think I mentioned this before -- that they were working alone. I mean, they were trying to do disciplined work. The analogy I use is trying to practice to be a concert pianist in a deaf world. No one will pay attention. Well, Beethoven was deaf when he wrote his Ninth Symphony, but there aren’t many Beethovens in the world. It’s extraordinary that people really, even though they’re convinced, they have a great deal of difficulty working in the environment where their managers are pushing them to do other stuff, everybody’s got to get into test, quality is not a priority, their peers are racing into test, everybody’s getting credit for getting into test early but they are not recognized for doing quality work.

As I said, I talked about this engineer at Siemens, in Siemens research. He did beautiful work, but no one knew. He was doing it all by himself. He was his own audience. No one else was paying any attention. So beautiful work in software does not get noticed. It’s a tragedy. There was one guy I ran into in IBM – Kingston -- exactly the same thing. I remember it was a long story. Remember, one of the things we’d put in place at IBM before I retired there from the quality job, we discovered that over 20 percent of IBM’s APAR (Applied Programming Assistance Request) repairs -- these were customer-reported defect repairs -- over 20 percent of them actually injected defects.

So we would put out fixes that were themselves defective, and the fixing process was in the maintenance organization; it wasn’t done by the developers. So this became a big issue because when we really looked at the data we discovered it was enormously expensive -- like 40% of our programming service costs. I mean, fixes get slapped in. People don’t install them slowly. So when you have fix problems, you’ve got crises. And we had lots of them. So we started something called APAR certification, where before the maintenance people could ship a fix, they had to verify it with the developer. So we put that in place, and we had this procedure everybody was supposed to follow.

I remember, I was up in Kingston, and I was doing a random check, just talking to experienced engineers who were doing development. I called in one engineer they sent in to get a random check to just see. And I used to like to do this, go talk to working engineers, see what they’re doing, what is going on. So I asked him about APAR certification, what he thought about it and how it worked. And he said, “What’s that?” I was really irate. I said, “What do you mean? We’ve got this enormous effort getting APAR certification across the company. Here’s why we’re doing it,” and that sort of thing. And I said, “How come you don’t know about it?” He said, “Well, I’ve got 30 modules in the core of the MVS system that I’m responsible for, and I’ve never had a defect in any of them.” And I said, “You win. You don’t need APAR certification.” Well, no one knew that. We’ve got this guy doing extraordinary work and no one knew it. But that’s what our world is like. You have these people doing extraordinary work and no one recognizes it. It’s terribly sad.

So in any event, I discovered the PSP -- the personal software process -- much as it worked, and much as it enabled people to do truly gorgeous work, very few people really have the personal discipline and the drive and the commitment, and they certainly don’t have the environment to permit them to do that work. So that’s why I concluded that we really had to move from the PSP to the team process. I had to put together a team environment where you could really do beautiful work. And that’s what led us to the TSP. Okay?

Booch: Got it. Very good. Let me level-set here. What year would this have been you’re starting to see the transition to the team software process?

Humphrey: It was 1995 I published the PSP book -- the end of ’94 actually, January ’95. We spent ’95 trying to put all this stuff together, running into all these experiences, and it was in early ’96 it became pretty obvious we had to put together a team process. So I started to construct a TSP, team process.

  • + Share This
  • 🔖 Save To Your Account

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020