- Principles of Executable Design
- Executable Design in Practice
- Summary
This chapter is from the book
This chapter is from the book
This chapter is from the book
Executable Design in Practice
Executable Design involves the following practices:
- Test automation
- Continuous unit test execution
- Merciless refactoring
- Need-driven design
- Test-Driven Development (or Design?)
- Modeling sessions
- Transparent code analysis
The rest of this chapter will provide detailed information about all of these practices in terms of Executable Design.
Test Automation
This practice may seem implied by Executable Design, but the approach used for test automation is important to sustainable delivery. Also, teams and organizations sometimes think that automating tests is a job for the test group and not for programmers. The approach to test automation in Executable Design is based on the approach to testing in XP. Taking a whole team approach to testing is essential to having a successful and sustainable test automation strategy. This does not mean that all team members are the best test case developers and therefore are generalists. It does mean that all team members are able to understand the test strategy, execute the tests, and contribute to their development when needed, which is quite often. The following principle for automated test accessibility sums up the Executable Design suggested approach:
- Everyone on the team should be able to execute any and all automated and manual test cases.
This is an extremely important statement because it expresses the importance of feedback over isolation in teams. If any team member can run the tests, the team member can ensure the integrity of changes closer to the time of implementation. This lessens the amount of time between introducing a defect and when it gets fixed. Defects will exist in the software for a shorter duration on average, thus reducing the defect deficit inherent in traditional test-after approaches.
The focus on how automated tests are used is also important. Automated tests at all levels of execution, such as unit, acceptance, system, and performance, should provide feedback on whether the software meets the needs of users. The focus is not on whether there is coverage, although this may be an outcome of automation, but to ensure that functionality is behaving as expected. This focus is similar to that of Behaviour-Driven Development (BDD), where tests validate that each application change adds value through an expected behavior. An approach to automating tests for Executable Design could be the use of BDD.
In addition to the automating test development approach, understanding how the test infrastructure scales to larger projects is essential for many projects. Structure and feedback cycles for each higher layer of test infrastructure can make or break the effective use of automated tests for frequent feedback. Over time, the number of tests will increase dramatically. This can cause teams to slow down delivery of valuable features if the tests are not continually maintained.
The most frequent reason for this slowdown is that unit tests are intermingled with slower integration test executions. Unit tests should run fast and should not depend on special configurations, installations, or slow-running dependencies. When unit tests are executed alongside integration tests, they run much slower and cause their feedback to be available less frequently. This usually starts with team members no longer running the unit tests in their own environment before integrating a change into source control.
A way to segregate unit tests from integration tests is to create an automated test structure. In 2003, while working on an IBM WebSphere J2EE application with a DB2 on OS/390 database, our team came up with the following naming convention to structure our automated tests:
- *UnitTest.java: These tests executed fast and did not have dependencies on a relational database, JNDI (Java Naming and Directory Interface), EJB, IBM WebSphere container configuration, or any other external connectivity or configurations. In order to support this ideal unit test definition, we needed to isolate business logic from "glue" code that enabled its execution inside the J2EE container.
- *PersistanceTest.java: These tests depended on a running and configured relational database instance to test integration of EJB entity beans and the database. Because our new architecture would be replacing stored procedure calls with an in-memory data cache, we would need these integration test cases for functional, load, performance, and stress testing.
- *ContainerTest.java: These tests were dependent on integrating business logic into a configured IBM WebSphere J2EE container. The tests ran inside the container using a framework called JUnitEE (extension of JUnit for J2EE applications) and would test the container mappings for application controller access to EJB session beans and JNDI.
In our development environments we could run all of the tests whose names ended with "UnitTest.java". Team members would execute these tests each time they saved their code in the IDE. These tests had to run fast or we would be distracted from our work. We kept the full unit test execution time within three to five seconds. The persistence and container tests were executed in a team member's environment before larger code changes—meaning more than a couple of hours of work—were checked in.
The full suite of automated programmer tests was executed on our continuous integration server each time code was checked into our source control management system. These took anywhere from 5 to 12 minutes to run. The build server was configured with a WebSphere Application Server instance and DB2 relational database. After the build and automated unit tests ran successfully, the application was automatically deployed into the container, and the database was dropped and re-created from scratch. Then the automated persistence and container tests were executed. The results of the full build and test execution were reported to the team.
Continuous Unit Test Execution
Automated programmer tests aren't as effective if they are not executed on a regular basis. If there is an extra step or more just to execute programmer tests in your development environment, you will be less likely to run them. Continuous programmer test execution is focused on running fast unit tests for the entire module with each change made in a team member's development environment without adding a step to the development process. Automating unit test execution each time a file is modified in a background process will help team members identify issues quickly before more software debt is created. This goes beyond the execution of a single unit test that tests behavior of the code under development. Team members are continually regressing the entire module at the unit level.
Many platforms can be configured to support continuous unit test execution:
- In Eclipse IDE, a "launcher," similar to a script that can be executed, can be created that runs all of the unit tests for the module. A "launcher" configuration can be saved and added to source control for sharing with the entire team. Another construct in Eclipse IDE called "builders" can then be configured to execute the "launcher" each time a file is saved.
- If you are into programming with Ruby, a gem is available called ZenTest with a component named autotest for continuous unit test execution. It also continuously executes unit tests when a file is changed. Autotest is smart about which tests to execute based on changes that were made since the last save.
- Python also has a continuous unit test execution tool named tdaemon that provides similar functionality to ZenTest for Ruby.
As you can see, continuous testing works in multiple programming languages. Automating unit test execution with each change lessens the need for adding a manual step to a team member's development process. It is now just part of the environment. Teams should look for ways to make essential elements of their software development process easy and automatic so it does not become or appear to be a burden for team members.
Merciless Refactoring
Refactoring is an essential practice for teams developing solid software and continually evolving the design to meet new customer needs. The web site managed by Martin Fowler, who wrote the original book conveniently called Refactoring, says:
- Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.1
It is important to understand that refactoring is not just restructuring code. Refactoring involves taking small, disciplined steps, many of which are documented in books and online resources on refactoring, to alter the internal structure of the software. (If you haven't done so already, please read the books and online resources on refactoring to learn how it is applied more effectively. This book will not describe specific refactorings in detail.)
When taking an iterative and incremental approach such as Scrum, it is imperative that the software continue to be changeable. Merciless refactoring is an approach that teams should adopt whether they are working on new or legacy software.
- merciless—adj.: having or showing no [mercy—show of kindness toward the distressed]
To refactor mercilessly means that the team will
- Relieve distressed code through kindness and disciplined restructuring.
Some teams wonder if they will be allowed to apply merciless refactoring in their project. It is important to understand that teams are not asked to develop software that does not allow for new changes to be easily added. Stakeholders do tend to want features quickly, but that is their role. Teams should understand that their role is to create quality software that does not accrue abnormal costs with each change. Robert C. Martin wrote in his book Clean Code: A Handbook of Agile Software Craftsmanship about a simple rule that the Boy Scouts of America have:
- Leave the campground cleaner than you found it.2
Teams that I work with use a variant of this simple rule in their own working agreements:
- Always leave the code in better shape than when you started.
Teams demonstrating this mind-set will continually improve the software's design. This leads to acceleration in feature delivery because the code will be easier to work with and express its intent more concisely. On a project that is well tended in terms of its design and structure, the act of refactoring can be elegant and liberating. It allows teams to continually inspect and adapt their understanding of the code to meet the customer's current needs.
On a legacy application or component, the act of refactoring can seem overwhelming. Although refactoring involves making small, incremental improvements that will lead to improvement in the software's design, figuring out where to start and stop in a legacy system is often unclear. How much refactoring is sufficient in this piece of code? The following questions should help you decide whether to start refactoring when you see an opportunity for it:
- Does this change directly affect the feature I am working on?
- Would the change add clarity for the feature implementation?
- Will the change provide automated tests where there currently are none?
- Does the refactoring look like a large endeavor involving significant portions of the application components?
If the answer to the first three questions is yes, I lean toward refactoring the code. The only caveat to this answer is when the answer to the fourth question, Does the refactoring look like a large endeavor?, is yes. Then I use experience as a guide to help me produce a relative size estimate of the effort involved in this refactoring compared to the initial estimate of size for the feature implementation. If the size of the refactoring is significantly larger than the original estimate given to the Product Owner, I will bring the refactoring up to the team for discussion. Bringing up a large refactoring to the rest of the team will result in one of the following general outcomes:
- The team thinks it is good idea to start the large refactoring because its estimated size does not adversely affect delivery of what the team committed to during this iteration.
- The team decides that the refactoring is large enough that it should be brought up to the Product Owner. The Product Owner could add it to the Product Backlog or decide to drop scope for the current iteration to accommodate the refactoring.
- Another team member has information that will make this refactoring smaller or not necessary. Sometimes other team members have worked in this area of code or on a similar situation in the past and have knowledge of other ways to implement the changes needed.
After starting a refactoring, how do we know when to stop? When working on legacy code, it is difficult to know when we have refactored enough. Here are some questions to ask yourself to figure out when you have refactored enough:
- Is the code I am refactoring a crucial part of the feature I was working on?
- Will refactoring the code result in crucial improvements?
Adopting a merciless refactoring mind-set will lead to small, incremental software design improvements. Refactoring should be identified in the course of implementing a feature. Once the need for a refactoring is identified, decide if it is valuable enough to do at this point in time, considering its potential cost in effort. If it meets the criteria for starting a refactoring, use disciplined refactoring steps to make incremental improvements to the design without affecting the software's external behavior.
Need-Driven Design
A common approach to designing application integrations is to first identify what the provider will present through its interface. If the application integration provider already exists, consumers tend to focus on how they can use all that the provider presents. This happens when integrating services, libraries, storage, appliances, containers, and more.
In contrast, Need-Driven Design approaches integration based on emergence and need. The approach can be summarized in the following statement:
- Ask not what the integration provider gives us; ask what the consumer needs.
Need-Driven Design, in its basic form, is based on the Adapter design pattern3 as shown in Figure 4.1. There are two perspectives for integration in Need-Driven Design:
- Consumer: Ask what the consumer needs from the interface contract.
- Provider: A provider's interface emerges from needs expressed by more than one consumer.
)
Figure 4.1 Example implementation of the Need-Driven Design approach for exploiting an external component using the Adapter design pattern
From the consumer perspective, the idea is to depend only on what the software actually needs and no more. Instead of coupling the application to a web service directly, create an interface in between that defines only what the software needs, then implement the interface to integrate with the web service. This way, dependency on the web service is limited to the implementation of the interface and can be modified or replaced if the need arises in a single place. In the case of integrating a library, creating too much dependence on the library could make the application less changeable for new user needs. Wrapping the specific aspects of a library that the application uses could be an approach worth pursuing.
From a provider perspective, generalizing an interface should be done only after there is more than one consumer of the provider's capabilities. This contrasts with how many organizations approach application integration. They might have a governance program that identifies services before construction of software begins. The Need-Driven Design approach is to wait for more than one consumer to need access to a particular capability. Once the need to create a provider interface is identified, it is promoted to a reusable asset.
The Need-Driven Design approach is best applied in conjunction with automated unit tests. The automated unit tests describe what each side of the adapter, the consumer interface and provider interface, will be responsible for. The application using the adapter should handle general usage of the client interface and should not care about specialized concerns of the provider interface. This can be explained with the following real-world example where Need-Driven Design was applied.
Instead of designing software toward what an external dependency can provide, decide what the application needs. This need-driven approach focuses on adding only what is necessary rather than creating dependence on the external component. Need-Driven Design has the following steps:
-
Assess the need: Add external dependencies to your project only when the value outweighs the integration costs.
-
Define the interface: Create an interface that will provide your application with the capabilities that it needs.
-
Develop executable criteria: Write automated unit tests for expected scenarios your application should handle through the defined interface; mock up and/or simulate the various scenarios.
-
Develop the interface implementation: Create automated unit tests and the interface implementation to integrate external components, and make sure to handle conditions that the application does not need to be aware of.
By driving integration strategies through the steps defined in Need-Driven Design, we can decrease integration costs, reduce coupling to external dependencies, and implement business-driven intentions in our applications.
Test-Driven Development (or Design?)
Test-Driven Development (TDD) is a disciplined practice in which a team member writes a failing test, writes the code that makes the test pass, and then refactors the code to an acceptable design. Effective use of TDD has been shown to reduce defects and increase confidence in the quality of code. This increase in confidence enables teams to make necessary changes faster, thus accelerating feature implementation throughput.
It is unfortunate that TDD has not been adopted by the software development industry more broadly. The TDD technique has been widely misunderstood by teams and management. Many programmers hear the name and are instantly turned off because it contains the word test. Teams that start using TDD sometimes misinterpret the basics or have difficulty making the mind-set shift inherent in its use. Using tests to drive software design in an executable fashion is not easy to grasp. It takes tremendous discipline to make the change in approach to design through micro-sized tests.
The following statement summarizes how I describe TDD to teams that are having difficulty adopting the approach in their development process:
- TDD is about creating a supportable structure for imminent change.
Applications and their components change to meet new business needs. These changes are effected by modifying the implementation, improving the design, replacing aspects of the design, or adding more functionality to it. Taking a TDD approach enables teams to create the structure to support the changes that occur as an application changes. Teams using a TDD approach should maintain this structure of unit tests, keeping the unit tests supportable as the application grows in size and complexity. Focusing on TDD in this manner helps teams understand how they can apply it to start practicing a design-through-tests approach.
Automated unit tests are added through a test-driven approach and tell us if aspects of each component within an application are behaving as expected. These tests should be repeatable and specific so they can be executed with the same expected results each time. Although the tests are important, teams should not lose focus on how these tests drive the software design incrementally.
A basic way to think about TDD is through a popular phrase in the TDD community:
- Red, Green, Refactor.
This simple phrase describes the basic steps of TDD. First, write a failing test that describes the scenario that should work at a micro level of the application component. Then write just enough code to make it pass, and no more. Finally, refactor the implementation code and tests to an acceptable design so they can be maintained over time. It is important to emphasize once again to write only enough code to make the current failing test pass so no untested code is written. Untested code is less safe to change when it's time to make necessary refactorings. Figure 4.2 shows the basic steps involved in the TDD approach.
)
Figure 4.2 The basic steps of Test-Driven Development are to write a failing test, write only the code that makes the test pass, and refactor to an acceptable design.
These three basic steps are not always sufficient to do TDD effectively. Uncle Bob Martin wrote "The Three Laws" of TDD as follows:
- Test-Driven Development is defined by three simple laws.
- You must write a failing unit test before you write production code.
- You must stop writing that unit test as soon as it fails; and not compiling is failing.
- You must stop writing production code as soon as the currently failing test passes.4
He goes on to say that software developers should do TDD as a matter of professionalism. If, as software developers, we do TDD effectively, we will get better at our craft. Uncle Bob Martin provides an initial list of things we could improve by doing TDD:
- If you follow the three laws that seem so silly, you will:
- Reduce your debug time dramatically.
- Significantly increase the flexibility of your system, allowing you to keep it clean.
- Create a suite of documents that fully describe the low level behavior of the system.
- Create a system design that has extremely low coupling.5
Modeling Sessions
When team members get together and discuss software design elements, they sometimes use visual modeling approaches. This usually happens at a whiteboard for collocated team members. While the team names model elements and their interactions, the conversation revolves around how the model enables desired functionality. The points discussed can be thought of as scenarios that the solution should support. These scenarios are validated against the model throughout the design conversation and can be easily translated into one or more test cases.
As the modeling session continues, it becomes more difficult to verify the number of scenarios, or test cases, that have already been discussed. When an interesting scenario emerges in conversation and causes the model to change, the group must verify the model against all the scenarios again. This is a volatile and error-prone approach to modeling because it involves manual verification and memorization. Even so, modeling is a valuable step since it helps team members arrive at a common understanding of a solution for the desired functionality. Minimizing the volatile and error-prone aspects of this technique improves the activity and provides more predictable results. Using TDD to capture the test cases in these scenarios will eventually make them repeatable, specific, and executable. It also helps to ensure that the test cases providing structure to the solution's design are not lost. Without the test cases it is difficult to verify the implementation and demonstrate correct and complete functionality.
By no means would I prescribe that teams eliminate quick modeling sessions. Modeling sessions can provide a holistic view of a feature or module. Modeling only becomes an issue when it lasts too long and delays implementation. The act of designing should not only be theoretical in nature. It is good to time-box modeling sessions. I have found that 30 minutes is sufficient for conducting a modeling session. If a team finds this amount of time insufficient, they should take a slice of a potential solution(s) and attempt to implement it before discussing the rest of the design. The act of implementing a portion of the design provides a solid foundation for further exploration and modeling.
Modeling Constraints with Unit Tests
To reduce duplication and rigidity of the unit test structure's relationship to implementation code, teams should change the way they define a "unit." Instead of class and method defined as the only types of "unit," use the following question to drive the scenario and test cases:
- What should the software do next for the intended user?
The approach for writing unit tests I follow is that of Behaviour-Driven Development (BDD).6 Thinking in terms of the following BDD template about how to model constraints in unit tests helps me stay closer to creating only the code that supports the desired functionality:
Given <some initial context> When <an event occurs> Then <ensure some outcomes>.
By filling in this template I can generate a list of tests that should be implemented to supply the structure that ensures the desired functionality. The following coding session provides an example of applying this approach. The fictitious application is a micro-blogging tool named "Jitter." The functionality I am working on is this:
- So that it is easier to keep up with their child's messages, parents want shorthand in the messages to be automatically expanded.
The acceptance criteria for this functionality are:
- LOL, AFAIK, and TTYL are expanded for a parent.
- It should be able to expand lower- and uppercase versions of the shorthand.
The existing code is written in Java and already includes a JitterSession class that users obtain when they authenticate into Jitter. Parents can see their child's messages in their session. The following unit test expects to expand "LOL" to "laughing out loud":
public class WhenParentsWantToExpandMessagesWithShorthandTest {
@Test
public void shouldExpandLOLToLaughingOutLoud() {
JitterSession session = mock(JitterSession.class);
when(session.getNextMessage()).thenReturn("Expand LOL");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand laughing out loud"));
}
}
Before we continue with the unit test code example, let's look more closely at how it is written. Notice the name of the programmer test class: WhenParentsWantToExpandMessagesWithShorthandTest.
For some programmers, this long name might seem foreign. It has been my experience that it is easier to understand what a programmer test has been created for when the name is descriptive. An initial reaction that programmers have to long names for classes and methods is the fear they will have to type them into their editor. There are two reasons why this is not an issue:
- Because this is a unit test, other classes should not be using this class.
- Modern integrated development environments have code expansion built in.
Also notice that the name of the test method is shouldExpandLOLToLaughingOutLoud. This naming convention supports how we drive design through our unit tests by answering the question "What should the software do next for the intended user?" By starting the method name with the word should, we are focusing on what the software should do for the user identified in the unit test class name. This is not the only way to write unit tests. People have a wide variety of preferences about how to write their tests, so please find the way that fits your team's intended design strategy best.
The MessageExpander class does not exist, so I create a skeleton of this class to make the code compile. Once the assertion at the end of the unit test is failing, I make the test pass with the following implementation code inside the MessageExpander class:
public String getNextMessage() {
String msg = session.getNextMessage();
return msg.replaceAll("LOL", "laughing out loud");
}
This is the most basic message expansion I could do for only one instance of shorthand text. I notice that there are different variations of the message that I want to handle. What if LOL is written in lowercase? What if it is written as "Lol"? Should it be expanded? Also, what if some variation of LOL is inside a word? The shorthand probably should not be expanded in that case except if the characters surrounding it are symbols, not letters. I write all of this down in the unit test class as comments so I don't forget about it:
// shouldExpandLOLIfLowerCase // shouldNotExpandLOLIfMixedCase // shouldNotExpandLOLIfInsideWord // shouldExpandIfSurroundingCharactersAreNotLetters
I then start working through this list of test cases to enhance the message expansion capabilities in Jitter:
@Test
public void shouldExpandLOLIfLowerCase() {
when(session.getNextMessage()).thenReturn("Expand lol please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand laughing out loud please"));
}
At this point, I find the need for a minor design change. The java.lang.String class does not have a method to match case insensitivity. The unit test forces me to find an alternative, and I decide to use the java.util.regex.Pattern class:
public String getNextMessage() {
String msg = session.getNextMessage();
Pattern p = Pattern.compile("LOL", Pattern.CASE_INSENSITIVE);
Return p.matcher(msg).replaceAll("laughing out loud");
}
Now I make it so that mixed-case versions of "LOL" are not expanded:
@Test
public void shouldNotExpandLOLIfMixedCase() {
String msg = "Do not expand Lol please";
when(session.getNextMessage()).thenReturn(msg);
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(), equalTo(msg));
}
This forces me to use the Pattern.CASE_INSENSITIVE flag in the pattern compilation. To ensure that only the code necessary to make the test pass is created, I match only "LOL" or "lol" for replacement:
public String getNextMessage() {
String msg = session.getNextMessage();
Pattern p = Pattern.compile("LOL|lol");
return p.matcher(msg).replaceAll("laughing out loud");
}
Next, I make sure that if "LOL" is inside a word it is not expanded:
@Test
public void shouldNotExpandLOLIfInsideWord() {
String msg = "Do not expand PLOL or LOLP or PLOLP please";
when(session.getNextMessage()).thenReturn(msg);
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(), equalTo(msg));
}
The pattern matching is now modified to use spaces around each variation of valid "LOL" shorthand:
return Pattern.compile("\\sLOL\\s|\\slol\\s").matcher(msg)
.replaceAll("laughing out loud");
Finally, it is important that if the characters around LOL are not letters, such as a space, it still expands:
@Test
public void shouldExpandIfSurroundingCharactersAreNotLetters() {
when(session.getNextMessage()).thenReturn("Expand .lol!
please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand .laughing out loud! please"));
}
The final implementation of the pattern-matching code looks like this:
return Pattern.compile("\\bLOL\\b|\\blol\\b").matcher(msg)
.replaceAll("laughing out loud");
I will not continue with more of the implementation that would expand other shorthand instances. However, I do want to discuss how the focus on "What should the software do next?" drove the design of this functionality. Driving the code using TDD guides us to implement only what is needed. It also helps us approach 100% code coverage for all lines of code. For programmers who have experience writing object-oriented code, the modules will likely have high cohesion, focused on specific responsibilities, and maintain low coupling to other code. The failing unit test represents something that the software does not do yet. We focus on modifying the software with the simplest implementation we can think of that will make the unit test pass. Then we focus on enhancing the software's design with the refactoring step. It has been my experience that refactoring takes most of the effort when applying TDD effectively. This does not mean refactoring is used with each TDD cycle. It means that overall, programmers spend more time refactoring to enhance the design.
Software Design beyond TDD
Most software design approaches are concerned with documenting design artifacts. Agile teams look for ways to reduce documentation to only what is necessary. Because of this statement, many teams and organizations mistakenly think Agile means no documentation. This is an inappropriate interpretation of Agile software development and is not corroborated by thought leaders and books from the Agile community.
To better enable cost-effective and high levels of support for applications deployed in production, teams ought to be aware of artifacts that assist ongoing maintenance. Software development goes beyond just writing code. It also includes demonstrating the integrity of component integration, alignment to business objectives, and communication of the software's structure for continued maintenance. Some of these aspects can be validated through integration tests. As pointed out in the section on test automation earlier in this chapter, integration tests are not executed as frequently as fast unit tests, such as in each team member's environment. Instead, they are executed in an integration environment when changes are integrated into a common stream of work in source control.
Teams that must consider some or all of the aspects listed above should have processes and tools that support effective maintenance. On top of automated integration testing, they might also benefit from
- Frequent and enhanced compliance auditing
- A team member with specific knowledge or appropriate training and practice
- Push-button deployment and rollback capability to all associated environments
- Production-like staging and test environments for more realistic integration testing
As a team, think about which aspects of software design you should be concerned with and then figure out how you will manage them in the software development process.
Transparent Code Analysis
Code coverage tools measure whether all discernible paths through the code have been tested. It is impossible, except in the most basic instances of code, to validate that all paths through the code have been tested with every potential input. On the other hand, it is possible to ascertain whether each line of code in a module has been tested. This involves measuring test coverage by some basic metrics:
- Statement coverage checks how many lines of code have been executed.
- Decision coverage checks if each path through a control structure is executed (i.e., "if/else if/else" structures).
- Condition coverage checks if each Boolean expression is evaluated to both true and false.
- Path coverage checks if combinations of logical code constructs are covered, including sufficient loop evaluation (i.e., executing a loop 0, 1, and more than 1 time).
- Relational operator coverage checks if inputs in relational operator evaluation are sufficiently verified (i.e., executing a < 2 with a = 1, a = 2, a = 3).
Executing code coverage tools inside each development and continuous integration environment can be helpful feedback to identify lapses in test-driven discipline. As a programmer gains more experience with TDD, it becomes practical to approach 100% coverage in most instances. Tools can provide feedback about code coverage, as shown in Figure 4.3.
)
Figure 4.3 The Eclipse IDE plug-in, EclEmma, showing a view of the project's code coverage inside the Eclipse IDE
It is a common belief that striving for 100% code coverage is too expensive and does not provide enough value to offset its costs. In my experience, approaching 100% code coverage increases confidence and accelerates delivery of working software. There are some factors that inhibit teams from approaching 100% code coverage:
- Working with an existing code base that has significantly less than 100% code coverage: If this is the case, track increases in code coverage rather than whether it approaches 100%.
- A brittle component or application where finding a place to put a valid unit test is difficult, or nearly impossible: In this case, using a black-box testing tool that executes the code in its packaged or installable form could be a better option until the code is less tangled. The packaged or installed application could be instrumented sometimes so that code coverage is evaluated during the black-box test execution.
- When code is generated: Teams should look for ways to isolate generated code from code implemented by team members and evaluate code coverage only for the latter. In some circumstances, our team has been able to generate the unit and integration tests for generated code when we had access to the code generation templates.
- When code integrates with third-party components: Although the third-party component probably will not have 100% code coverage, integration tests can be developed that verify how the code is expected to integrate. Evaluate code coverage on only code that your team created. See the Need-Driven Design section earlier in this chapter.
When working with an existing code base, approachiíng 100% is probably not attainable. In this case, make sure that the current code coverage does not deteriorate. Teams should look for ways to slowly increase code coverage of the existing software over time.
Tools to determine code coverage are not the only code analysis tools available. There are many tools in the static code analysis arena as well. Static code analysis tools can provide feedback through a dashboard, such as in Figure 4.4, on aspects of the software such as
- Team's preferred coding rules
- Lines of code
- Cyclomatic complexity
- Duplicated code
- Lack of cohesion
- Maintainability
- Dependencies
- Design
- Architecture
- Technical debt
)
Figure 4.4 The Sonar dashboard showing metrics for lines of code, technical debt ratio, code coverage, duplicated lines of code, and build time7
I do not suggest that the metrics generated for all of these aspects of the software are exact. They do provide feedback about our software internals and can help guide our development efforts. Also, it is sometimes easier to drill down into the static code analysis dashboard for an area of code that will be involved in changes for the next feature. Looking at higher-level metrics for the code can provide useful information to guide implementation and opportunities for code improvement.