- Share Test Scripts Between Teams
- Distribute the Ability To Execute Smoke Tests
- Perform Runtime Analysis Together
- Use Log Files To Isolate Problems
- Use Defect-Tracking Systems Effectively
- Speak Face to Face
- Next Steps
- References
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Distribute the Ability To Execute Smoke Tests
Every time a developer, integrator, or build-master (depending on your choice of terminology) creates a build, there's potential for something to go wrong: Something is left out, a file doesn't end up where it was supposed to be, the wrong version is compiled, something goes wrong when the code is moved into the target environment, and so on. On a team that does daily builds or even multiple builds a day, these issues crop up from time to time. The problem with a bad build is that you won't necessarily know it's bad until you get in there and do some testing. The solution is to create a series of tests that exercise the entire system from end to end. These tests, taken as a whole, are commonly called smoke tests. I believe the term comes from a rudimentary form of testing applied to electronic equipment, in which power is applied and the tester checks for sparks, smoke, or other dramatic signs of fundamental failure. A smoke test doesn't have to be exhaustive, but it should be capable of exposing major problems. If the smoke test fails, you can assume that the build is not stable enough to be tested more thoroughly.
In a web environment, smoke tests can even check the status of services. Web applications often use third-party services; checking all of them quickly can be difficult. If you build into your smoke tests a series of checks for those services, you can kill two birds with one stone. I often include a "service" smoke test with the "regular" smoke testing; the service smoke test can be executed on its own for a quick web-service status update. On some projects, I've even automated such service checks to run hourly and send email when services go down.
If you don't have a smoke test, create one. If it's not automated, automate it. Automated smoke tests are particularly powerful:
They're used often, possibly many times a day.
They provide meaningful information (the system is at an acceptable state for testing and all services are up—or not).
They provide feedback quickly, typically in a matter of minutes.
They're easy to execute and distribute.
The sooner the smoke test is executed after the build, the faster the feedback can get from testing to development. The easiest way to ensure that your smoke test is executed is to include it in the build process. Depending on the available tools, smoke tests can be added to a build process via a batch file, if your build process is automated. If you have a manual build process, add a step at the end so that the person who performs the build also executes the smoke test to verify the build results.
Make the smoke test available to both testers and developers by using a central interface such as a project web site or a test-management tool. If everyone has the ability to execute the smoke test and the results are simple to interpret, testers won't be pressured to provide this service for everyone else on the team. Getting team members other than testers to run the smoke test may take a little patience and prompting the first couple of times, but after that most people will prefer not relying on someone else for such a simple task.
NOTE
Of course, this general availability means that the results must be easy to interpret.
By this step, you've begun giving developers a window into the mind of a tester.
Distributing the ability to execute smoke tests also increases communication between developers and testers. Not only does this practice get everyone using the same tools; it can get developers and testers collaborating in script development and maintenance. Seeing what the test team developed, developers may offer help in integrating selected unit tests that check essential back-end functionality, and help in creating interfaces to check the status of web services.