Becoming a Software Developer Part 5: Creating Acceptance Tests from Use Cases
By Pete McBreen
Date: May 10, 2002
Article is provided courtesy of Addison Wesley.
Introduction
At the end of my last article, "Becoming a Software Developer, Part 4: Understanding Use Cases and Requirements," we finished off with a partially elaborated use case. To fully elaborate the use case, we need to investigate the extension conditions and the extension handling steps.
NOTE
For those of you who did the optional reading, Alistair Cockburn's book Writing Effective Use Cases (Addison-Wesley, 2000, ISBN 0-201-70225-8), our sample use case is only documented to two levels of precision, the extension conditions will add a third level of precision, and the extension handling steps will add a fourth level.
Right now, we have a readable description of how an actor (the Club Secretary) uses the system to achieve one of the actor's user goals (in this case, notifying members about special events). We also have some guarantees that define correct operation of the system.
Club Secretary : Notify members about special events. Send out information about forthcoming events and races to members.
Minimal Guarantee:
Event details are recorded in the application.
Members are never sent a duplicate notice about the same event.
Success Guarantee:
Selected members are sent email containing an event notice.
The notice is recorded as being sent to the selected members.
Main Success Scenario:
Club Secretary: Enter event details.
Club Secretary: Choose members to send notice to.
Membership System: Display number of members to be notified and await confirmation from the Club Secretary.
Membership System: Send email notification to every selected member.
Membership System: Record that notice was sent to selected members.
Involve Testing and Quality Assurance Before Thinking About Design
Although you might want to start thinking about how to design the application as soon as you have a hint of a requirement, it's better to curb this initial enthusiasm for design and instead think about how to test the resulting application. Thinking about quality assurance first is a natural extension of the ideas of test-driven development (see Part 2 of this series) and results in fewer surprises later on.
By thinking early about how to test the application, you can get a good handle on all of the weird and wonderful things that can go wrong at each step of the use case. Ideally you should involve your quality assurance and testing people in eliciting requirements, because they'll be great at suggesting "what if" scenarios for users to think about. Testers are a great addition to the requirements elicitation activities because they're detail-oriented and want to know what's supposed to happen in really specific circumstances like these:
What happens if the network connection to the email server fails after we send an email but before we get back the confirmation? Should the application consider that email as sent, or not?
If sending is interrupted in any way, what is the application supposed to do? Automatically retry later, try a different email server, let the user resend the notification, or what?
Identifying the Ways in Which a Use Case Can Fail
Every step in the main success scenario is a "subfunction goal" for an actor or the application. Every single subfunction goal can fail in one way or another. If the failure is unrecoverable, the use case still must deliver on the minimum guarantees, and it must not deliver on the success guarantees. On the other hand, if the failure is recoverable, the overall use case must still deliver on all of the guarantees.
Recording how the application recovers from these failures is the job of the extension handling steps, but don't get ahead of yourself. Before wondering how to recover from any of the failures, you first need to get the help of the users and testers in identifying all of the possible things that could go wrong. For every failure condition identified, you need to confirm that it's something the application can detect, and that the users think it's important that the failure is handled. Everything that passes these two checks should be recorded in the extensions section of the use case, immediately below the main success scenario.
Extensions:
1a Computer crashes while entering event details.
1b Duplicate event.
2a No members selected.
3a Club Secretary selects Cancel.
3b Timeout on waiting for user response.
3c Club Secretary chooses new subset of membership.
4a Mail server does not respond.
4b Mail messages rejected by mail server.
5a Member not found.
* A user closes the application.
The extensions are labeled with the step number and a letter to distinguish multiple failure conditions on the same step.
Sometimes a failure can occur on any step. In that case, you can either denote the step number with an asterisk (*) or report the range of steps over which it could occur (1–5 in this example).
Even though the list of extensions is incomplete (invalid email addresses, mailbox full, and so on), you should now be getting a sense of the complexity that can hide within a use case. As a developer, you have to design the application to detect and handle each one of these failure conditions. You also have to make sure that if extra failure conditions are identified later, these new extensions can be incorporated into the application relatively painlessly.
Although you now have enough information to start thinking about the design of the application to support this particular use case, defer making any design decisions until you have the final piece of the use case puzzle—the extension handling steps.
Documenting the Extension Handling Steps
How the application is supposed to respond to the various failure conditions is something that you, as a developer, need the users to tell you. Yes, the users may need some guidance as to what's possible, but usually the goal is to make the application easy to use and productive for the users. Sometimes we can get away with doing whatever is easiest to code, but that's the path to really abysmal, difficult-to-use software.
When faced with the following extensions, the application could simply stop, but that would probably not be very friendly to the users:
4c Mail server reports that email address is invalid.
4d Selected member does not have an email address.
Involving users in designing the extension handling steps means that they're in charge of the requirements for the application. Once the users have designed the requirements, you can then proceed to design an application that satisfies their requirements.
Documenting the extension handling steps is relatively simple. Inside the extensions section of the use case, write down the steps necessary to recover from the failure.
Extensions:
...
4d Selected member does not have an email address.
.1 Membership System: Display list of members without an email address.
.1a Club Secretary: Cancel print option.
.2 Membership System: Print event details with email-challenged members' names and phone numbers.
.2a Printer not available.
.2b Member does not have a contact phone number.
.2b1 Membership System: Print ".........." where phone number should be.
...
Now we start to see the full complexity of the use case, since every subfunction goal inside the extension handling steps could also fail. It also explains why you want testers involved in eliciting the requirements, since they're so good at finding things that can go wrong.
On the other hand, for many applications it's perfectly valid to just ignore some errors and leave them for the users to handle manually. The failure condition 4d2b Member does not have a contact phone number probably falls into this category—as the step of printing ".........." suggests.
Once you have specified the extension handling steps for every extension, you have a fully elaborated use case. You also have spent a considerable amount of time and effort working with your users and testers to understand just this one small part of the application. With the knowledge you've gained, you're probably ready to start working on the design, although you might want to wait until you have a few more use cases fully elaborated. But don't make the mistake of waiting until you have all of your use cases fully elaborated; that's the slow waterfall process—you want to capture requirements incrementally so that you can deliver incrementally.
Creating the Acceptance Test Cases
In parallel with designing the application for a particular use case, it's very useful and effective to also design the acceptance test cases for the use case. If you're lucky, this is something that your testers and users will do for you; even if they do, however, I normally spend some time thinking about how to test the use case.
Conceptually, an acceptance test case is very simple. It defines correct operation of the system by stating:
The initial state of the application
The various inputs from the users
The final state of the application
The various outputs from the application
Although some people prefer to number their test cases, personally I prefer to name every test case with a combination of the use case goal and the extension conditions that are being tested.
Notify About Event: Member without email or phone
Initial State:
Membership system contains a member with the name "Pete" and blank email address and phone number.
Inputs:
Club Secretary specified event "Club AGM," date 1-April-2002, location "TBA."
Secretary then chose to print event notice for "Pete."
Final State:
"Club AGM" event is stored.
Member "Pete" is flagged as being sent "Club AGM" notice.
Outputs:
Application showed that 1 member would be notified, and prompted Club Secretary to print event details for "Pete."
"Club AGM" event notice is printed with phone contact list containing the name "Pete" and the phone number "......."
NOTE
Dates in acceptance tests get stale really quickly, so I often choose a date in the past or an easy-to-remember date—April Fools' Day works great for me. If the date absolutely has to be a future date, I use a relative date, such as "now + 30 days".
As you can see, fully specifying one acceptance test case for this use case requires a lot of detail. The full suite of acceptance tests for this one use case could easily require 20 or 30 acceptance test cases. Luckily, once you have all of the detail in the use case, creating these test cases is a fairly mechanical process, but it still has to be done. After all, it would be fairly embarrassing to ask the users to decide how to deal with all of the weird and wonderful error conditions and then have the application fail the first time the user tries to do anything out of the ordinary.
The other reason for documenting these acceptance test cases in such detail is that they are now available for use when you start doing design and test-driven development of the application (see Part 2 of this series for more on test-driven development). The next article in this series will take this use case and associated acceptance test cases and look at the "simple matter" of programming an application (with various detours to discuss design and the craft of programming).