Home > Articles

  • Print
  • + Share This
This chapter is from the book

Step 4: Try It Out

I next needed to ask a number of users if they’d look at my app. I emailed my student list, asking for anyone who rode commuter rail and wouldn’t mind taking a look. Obviously, this selection has a certain amount of skew. My students tend to be younger than the average rider, although since I teach in the continuing education division, they’re not undergrads. They tend to be better educated than the majority of commuter rail users, although here in the Boston area we have more college grads than most places. All of them did explicitly identify themselves as commuter rail users, either current or recent, so they do know what they themselves need.

If I were going to invest major money in this mobile app, I’d need real users. I’d probably hand out cards at the commuter rail station offering $20 to any rider who’d do a half-hour Skype call to look at the app. I’d get as many riders as I wanted. Again, at this stage, I need to move quickly, so I’m thinking fewer test users rather than more. I decided to go with Steve Krug’s suggestion of three. If all three of them like something, it’s probably pretty good, and if all three of them hate something, it’s probably pretty bad.

I arranged Skype calls with three of my former students who are regular commuter rail riders. They know me and aren’t afraid to call ‘em as they see ‘em. Once we got our Skype session established, we chatted a little bit about their current activities to break the ice and get the conversation flowing, and then I started with open-ended prompts such as “Tell me about your commute.” Then I read them the blurb from Chapter 4, how “We’re not testing you. You cannot possibly make a mistake here. We’re checking how well our software fits what you’re trying to do.”

I used Story 3 above, the one about Claire waking up in the morning. I figured it was the most critical one. I read it to the test subject, substituting his name for hers: “Imagine, John, that you’ve woken up at five in the morning to get ready for your day. You’re sipping your first cup of coffee and you look out the window and damn, it’s snowed six inches. You figure you had better check and see how the trains are running. You pull out your phone, tap the icon [now I share the screen in Balsamiq], and here’s what you see. What do you do now?”

One of the users started talking out loud as she thought, which is helpful. I had to prompt the other two to get them started: “John, it would be really helpful if you could maybe speak out loud as you are thinking. You’re obviously figuring something out; could you maybe tell me about it?” and so on.

I won’t repeat the conversations verbatim, but here is a summary of what I found. The first thing every user noticed was the set of radio buttons showing Today/Tomorrow/Other. I thought those buttons were important, but the users didn’t know what to make of them. I said, “OK, no problem, ignore them for now and continue. What are you thinking?” If these buttons had been a real problem, if they were too distracting to continue this test, I could have easily removed them from the Balsamiq mockup, but these users were happy enough continuing on.

They started looking at the departure times in the left list box. All of them said that they would find the one that they wanted to take and tap on it, expecting to see information about that train. I had expected the departure time to be all the information that the user needed, but these users didn’t see it that way. They wanted to see the arrival time, and they wanted some sort of indicator to know if it really was on time.

Two of the three noticed the countdown timer and alerts box and understood what it was. It’s a little harder to notice here in the static mockup, because in a real app you would see the time counting down and that would give a clue to its purpose. The third subject didn’t notice it at first but understood it immediately when I asked him to look at it.

This was all great feedback. I thought initially that the users would not care about the arrival time, because they already knew how long the train ride takes. That wasn’t the case. They all wanted to see it when making their choice of trains. They didn’t want to have to add it mentally; they wanted that information in tandem with the departure times, perhaps to go to the same place in their brains. Also, they said that they didn’t care about seeing the evening return trains at this point. They’d look at return trains in the afternoon when they were thinking of heading home. The return trains weren’t horribly distracting, but they weren’t the slightest help right then either, and their real estate could be used for something else that users cared about.

They all preferred the vertical arrangement of trains in Figure 8.9a to the horizontal line in Figure 8.9b. That’s how they’re shown in the timetable, that’s how they’re shown on the monitor in the station, and the testers found it confusing to view them otherwise. No one liked the horizontal line. OK, no problem; I learned something else.

They all said that the layout was too cramped, making it too hard to pick out the thing that they needed. In my zeal to get rid of all the navigation, I had crammed too much into one place and made the user do work of a different sort. Clearly, I hadn’t found the optimax yet.

Was I angry that a bunch of idiots couldn’t see the brilliance of the design? Did I scream to the heavens demanding to know how I had gotten such a brain-damaged set of test users? On the contrary. They told me things I didn’t know. They made this app better. They made me smarter. And now you too, I hope.

So back I went to Balsamiq. I split this information into two different screens, with the easiest possible navigation between them. The hamburger control is popular in mobile apps, but this app has such a small amount of navigation that the extra tap it would require (tap the hamburger to see the choices, then tap the choice you want) would have been cumbersome. Instead, I used a tab control for its visible navigation. The amount of screen real estate it consumes is small compared to the benefit of instant and obvious access to this small number of choices. I put the schedule on one tab and the ticket or pass on another. You can see it in Figure 8.10.

Figure 8.10

Figure 8.10 Second iteration of the MBTA mobile app, incorporating feedback from the first attempt.

The test users liked this one much better. They all said it was way easier to find the times that they wanted. They didn’t have to tap on anything to see the arrival times. “That table, that’s how they’re shown on the monitor in the station, so it’s really familiar,” said one, and the others agreed when I asked them. “We never use the train number at all. We always think of them as departure time, like the 8:57 train. So you might as well get rid of the column.” The other users, when I asked them, said that no, they never used it either and agreed we might as well get rid of it.

They loved the tab control for navigation. I thought it might be more of a PC-based idiom, not so much for mobile phones, but they all grabbed onto it right away. Maybe it’s the fact that the test users are older and more used to PCs, but then so are the actual commuter rail riders. The only disagreement was over where the tabs should go. At the top, as I originally showed? One tester wanted them at the bottom so he could select the tab with the thumb of his hand that held the phone, convenient when the train got crowded. I decided to keep them on top for now. It’s not a huge deal one way or the other. That choice can easily be made in further refinements, and if we really need it both ways, we could make it configurable.

Again I encountered the desire for separation of morning inbound train times and afternoon outbound train times. Every test user said, “Better, but I still have to tell one table from another. It’s easier than the first one you showed us, but why not have each in a separate tab?”

Finally, I asked each tester about the alert and countdown timer box. One said that the countdown timer was extremely important, as it told her when she had to run to make her next train. She wanted it on top for that reason. It would be nice to have the track number on it as well, if we were able to get that. That’s the sort of information you will get only from conversations with your users. Telemetry can’t tell you that.

Again I went back and made the changes suggested by the test users. They had said they were happy with tab navigation. They can see all the choices at once, and it takes only one tap to select any of them. They didn’t care about having inbound and outbound on the same screen, so I decided to place each on a separate tab.

I moved the countdown timer to the top. Only one test user had specifically requested that, but the others all said they liked it when they saw this design.

I used the extra space on the tab to show more trains, and to make the font larger for easier reading. Think about the demographics. Half of the users are age 45 and over. The developer community skews much younger than this. It is common for them to dismiss the need for larger type as a special need; it’s not the main thing, so we’ll get to it when we can—definitely not version 1, probably not version 2 either, maybe version 3, or then again maybe not. But age-related presbyopia (farsightedness) begins around age 40. At least half of our user base would appreciate something easier to read. You can’t call half the user population an obscure special case. They need relaxed-fit text as they need relaxed-fit jeans. They probably aren’t happy about needing it, but they’d sure appreciate having it. The third tab made reading easier for everyone. The users’ requirements are now working together toward the optimal solution. The result is shown in Figure 8.11.

Figure 8.11

Figure 8.11 Third iteration of the MBTA mobile app, incorporating feedback from previous attempts.

I’m still stuck on the case of Charlie sitting at home Sunday night, needing to take the train in on Monday and wondering what the schedule will be. That was the point of the radio buttons in the original mockup, which all the test users hated. But we don’t want Charlie to do any additional work by having to click the Schedule for Other Days link. So I brought in a pattern I see in airports. When the monitor lists flights by time, at the end of the day when there aren’t many left, they show a little banner after the last of tonight’s flights, and then they start showing tomorrow’s flights. So we can easily see that we’re at the end of today and what’s happening first thing tomorrow. I made a small change and came up with the iteration shown in Figure 8.12.

Figure 8.12

Figure 8.12 Fourth iteration of the MBTA mobile app, automatically showing tomorrow’s trains at the end of the day.

The app would be smart enough to bring up the correct tab based on the user’s location. If Charlie opened it while downtown, it would figure he’s most likely headed home, so it would bring up the outbound tab. Story 2 discusses this. But if Charlie opens the app in the suburbs in the evening, it would automatically show the inbound tab, with the last of today’s inbound trains (if any) and the first of tomorrow’s inbound trains. All the users loved this when I showed it to them.

Some other ideas surfaced here as well. One tester said, “What about a space for ads?” Much as I’d personally hate to see them, the rail agency could charge a lot of money for delivering this clientele to advertisers, especially if it were location based and time based. Imagine that you’re walking toward the station and your phone beeps. “I see you have an extra ten minutes. Here’s a coupon for a free donut with purchase of a coffee at Dunkin’ Donuts.” Or whatever.

There was some discussion about phone-level alerts or text messages when the schedule got seriously messed up, not just notifications within the app. Interestingly, all three test users expressed a desire for some information as to how crowded a train was. After further discussion, we judged these to be impractical, as you don’t know for sure how crowded a train will be until it fills up around departure time. Also, regular commuters usually know which trains are the most crowded. And there’s not much they can do about a crowded train anyway.

The point of this discussion is not to learn how to make your first design perfect. Your first design will never be perfect. It probably won’t even be all that great; mine wasn’t. It is meant to stimulate discussion, to get users thinking and talking to you. You need to iterate, and therefore you need to iterate as quickly and cheaply as possible. It shouldn’t take more than a week to go from sketches to testing and iterations. Lather, rinse, repeat.

There will be more detailed design, and better graphic design, as we continue development. But the point is that rapid sketching and quick feedback give us huge, huge advantages very quickly and very cheaply. We were able to switch designs before we’d spent much money, or much time, or gotten emotionally invested in any design. As I put on the shoes of my users, I learned all sorts of things that I never knew that I never knew. Now I know them, and so do you.

  • + Share This
  • 🔖 Save To Your Account

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information

To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.


Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.


If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information

Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.


This site is not directed to children under the age of 13.


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information

If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information

Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents

California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure

Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact

Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice

We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020