Becoming a Software Developer Part 2: Test Driven Development with Ruby
By Pete McBreen
Date: Apr 12, 2002
Article is provided courtesy of Addison Wesley.
Introduction
Once you've taken your first steps as a "Ruby nuby", you're ready to start thinking about how to write software that works. Software development is simultaneously both very simple and extraordinarily complex. It's simple because every line of code we write is easy to read and understand, but it's complex because the interactions between all the lines of code give the software its interesting behavior.
As you'll have discovered if you played around with the toy Odometer example from my last article, it's very easy to make a mistake when coding. That example was tiny; it only had three classes and less than 100 lines of code. Imagine the grief you could have working on a small application, say 5,000 lines of code, and then scaling up that fun factor as you realize that commercial applications often contain 50,000 to 500,000 lines of code (though some are much, much bigger). When applications get to be that size, verifying that the code works becomes a major challenge.
Obviously, someone will have to test the overall application once you've managed to get parts of it working, but getting large enough parts of it working for someone to test it is difficult. Yes, you can use the traditional design and code reviews to have other pairs of eyes look for mistakes, and you can try stepping through the code in a debugger to see what's really happening, but there's a much better way: test-driven development.
Test-Driven Development
Before you write any method in any class, figure out what the method is supposed to accomplish and then write a test to verify that the method does what it's supposed to do. Take for example the CircularCounter class from my last article. To test that class, we need the following Ruby code to be saved into a file called tc_CircularCounter.rb, which we can then run through the Test::Unit testing framework. (The set_up and tear_down methods are called by Test::Unit for every test case to make sure that each unit test starts off with known values.)
require 'test/unit' require 'CircularCounter' class TC_CircularCounter < Test::Unit::TestCase def set_up # sets up the initial values for each test @counter = CircularCounter.new(8,6) end def test_increment @counter.increment assert_equal(7, @counter.value, "Failed to increment") end def tear_down # does any necessary cleanup for each test @counter = nil end end
Running this code will demonstrate that we haven't yet implemented the methods:
>ruby tc_CircularCounter.rb tc_CircularCounter.rb:2:in ´require': No such file to load -- CircularCounter (LoadError) from tc_CircularCounter.rb:2 Loaded suite tc_CircularCounter Started... Finished in 0.01 seconds. 0 runs, 0 assertions, 0 failures, 0 errors
No real surprises there. It can't load the CircularCounter file because it hasn't been written yet. No problem; just grab the definition from last time and save it into CircularCounter.rb.
class CircularCounter def initialize (limit, initialValue) @limit, @value = limit, initialValue end def value @value #could use 'return @value' to be more explicit end def reset @value = 0 end def increment # returns true if overflow occurred @value = @value + 1 if (value >= @limit) reset true else false end end end
Rerunning the unit test shows that the CircularCounter class is now found and the assertion we made that the counter would be incremented turns out to be true:
>ruby tc_CircularCounter.rb Loaded suite tc_CircularCounter Started... .. Finished in 0.01 seconds. 1 runs, 1 assertions, 0 failures, 0 errors
To make sure that the counter rolls over the limit correctly, we should add another assertion to the test_increment method:
def test_increment
@counter.increment
assert_equal(7, @counter.value, "Failed to increment")
7.times {@counter.increment}
assert_equal(6, @counter.value, "Failed to roll over")
end
Running the unit test will then show that both assertions are true:
1 runs, 2 assertions, 0 failures, 0 errors
Since this is looking rather boring, let's introduce a deliberate (though easy to make) mistake. Say, for example, we had written this:
if (value > @limit)
rather than the correct entry:
if (value >= @limit)
The unit tests would have complained with the following message:
>ruby tc_CircularCounter.rb Loaded suite tc_CircularCounter Started... .. Failure occurred in test_increment(TC_CircularCounter) [tc_CircularCounter.rb:12]: Failed to roll over. Expected <6> but was <5> Finished in 0.05 seconds. 1 runs, 2 assertions, 1 failures, 0 errors
Test::Unit reports the failing assertion, telling us the line number of the failing assert, printing out our error message Failed to roll over, and it prints both the value we expected to see and the actual value. Given all of that information, the mistake is usually very easy to seeif it isn't immediately obvious, either get someone else to have a look or add more assertions to the test code. In our case, we can exploit the fact that on rollover the increment method is supposed to return true. (We also have to reduce the number of times we call increment in the loop because we're also calling it twice outside the loop.)
def test_increment
@counter.increment
assert_equal(7, @counter.value, "Failed to increment")
assert(@counter.increment, "Should have rolled over")
assert(!@counter.increment, "Should not have rolled over")
5.times {@counter.increment}
assert_equal(6, @counter.value, "Failed to roll over")
end
With this extra assertion, Test::Unit will now report that the CircularCounter failed to roll over, giving an even stronger hint about the (deliberate) mistake:
Failure occurred in test_increment(TC_CircularCounter) [tc_CircularCounter.rb:11]: Should have rolled over
Incremental Test-Driven Development
When doing test-driven development, you're supposed to work in extremely small increments. For each method you write, start off with a unit test containing just one or two simple assertions, implement the method to make those assertions true, and then add more assertions to the unit test.
Ideally you want to be bouncing between adding assertions, revising the code, and running the tests every minute or so. Take very small steps and run the tests after every small change to make sure that you're heading in the right direction. After all, as Nathaniel Talbott, the author of Test::Unit, states, the assertions are the heart of any unit testing framework:
Think of an assertion as a statement of expected outcome, i.e. "I assert that x should be equal to y." If, when the assertion is executed, it turns out to be correct, nothing happens, and life is good. If, on the other hand, your assertion turns out to be false, an error is propagated with pertinent information so that you can go back and make your assertion succeed, and, once again, life is good.
To see this in practice, we can add another unit test to the TC_CircularCounter class, a test for the decrement method, with a single initial assertion:
def test_decrement @counter.decrement assert_equal(5, @counter.value, "Failed to decrement") end
Initially, obviously, Test::Unit will complain because the CircularCounter doesn't even have a decrement method:
>ruby tc_CircularCounter.rb Loaded suite tc_CircularCounter Started... .. Error occurred in test_decrement(TC_CircularCounter): NameError: undefined method ´decrement' for #<CircularCounter:0x2c2c290> tc_CircularCounter.rb:17:in ´test_decrement' tc_CircularCounter.rb:22 . Finished in 0.07 seconds. 2 runs, 4 assertions, 0 failures, 1 errors
But, as Talbott says, the fact that decrement is an undefined method is pertinent information. So we can go and add an empty decrement method to CircularCounter:
def decrement end
Now we get different information reported, but it's just as useful:
Failure occurred in test_decrement(TC_CircularCounter) [tc_CircularCounter.rb:18]: Should have decremented correctly. Expected <5> but was <6> . Finished in 0.04 seconds. 2 runs, 5 assertions, 1 failures, 0 errors
So we can go ahead and implement the method:
def decrement @value = @value - 1 end
Once again, Test::Unit will report that life is good. The time has come to make another assertion, stating that when the CircularCounter reaches zero the next decrement sets it back to the maximum value:
def test_decrement
@counter.decrement
assert_equal(5, @counter.value, "Failed to decrement")
6.times {@counter.decrement}
assert_equal(7, @counter.value, "Failed to roll over")
end
And once again our assertion turns out to be false. We have another failing test, so we can change the implementation of the decrement method until life is good again. (This is left as an exercise for the reader.)
Failure occurred in test_decrement(TC_CircularCounter) [tc_CircularCounter.rb:20]: Failed to roll over. Expected <7> but was <-1>
The Payoff for Test-Driven Development
An obvious objection to test-driven development is that writing all of these tests takes time that could be used to write more production code. Yes, it does, no argument there. It takes a lot of time to think up useful test cases and to implement all of the assertions in the unit tests. But that's not the whole story. Writing the code has never been the limiting factor on projects. (If it was, programmers would be sent to "touch typing" courses.) The benefits of test-driven development show up in how the rest of the software development activities are improved. The time taken to produce quality applications is reduced by the following factors:
Much less time spent debugging mistakes in the code. Life is just too short to spend time stepping through the code line by line in a debugger to see what's going wrong in the code. Although I have in the dim and distant past spent lots of hours stepping through assembler code in a debugger, since adopting test-driven development I've only ever used a debugger for teaching purposes. In fact, I don't think I've ever used the Ruby debugger. (But if you really want to try, just start Ruby with the command ruby -r debug tc_CircularCounter.rb and then enter help at the prompt to see the commands that are available.)
Less time trying to locate mistakes. There is no need to search all over the source; you know where to start lookingin the method you're currently working on. Yes, you might find that one of your "tested" methods is actually incorrect, but most of the time the mistake is going to be in the method you're working on.
Fewer mistakes getting through to functional and acceptance testing. By building up the unit test as you go along, test-driven development helps catch more mistakes while the methods are being written. This is great news because it means there are fewer mistakes for the testers to catch, which in turn means that there are fewer mistakes for you to find and fix when everyone is asking "Why haven't you finished? You said you were nearly done a week ago!"
Documentation on how to actually use a class. This advantage is slightly counter-intuitive, but I've lost count of the number of times I've looked at the test cases to see how to use a method. The set_up method is a great place to find out how to create the objects, and then the test_xxx methods are a great way to see how to use the various methods. The great thing about using tests as documentation is that you know the example code works because you can run the tests successfully.
More confidence that changes in one place haven't broken something else. Programmers are a suspicious lot. Many have justifiable fears that making a change to one aspect of a design will break code in other parts of the application, so changing working code is always viewed with trepidation. Test-driven development reduces these fears because after every small change you can rerun the entire suite of unit tests to make sure that you haven't broken something else. It's also really useful for when upgrades come alongupgrading is less stressful when the unit tests give you confidence that the application still works.
A reasonable quality level in applications without imposing a heavyweight process on developers. Although remembering to always write the unit tests first can be difficult, it's a lot easier to stick with test-driven development than it is to use any heavyweight process. Design and core reviews are very effective at improving the quality of an application, but it's very easy to slack off on the reviews. Many teams find that although they're supposed to do reviews, in practice it only happens for some small fraction of the code. In contrast, with test-driven development you're less likely to slack off because the benefits are so immediate.
Learning to Write Good Unit Tests
Like a good many things, we can learn to get good at writing unit tests by reading the unit tests that other people have written. A great place to start is Nathaniel Talbott's Test::Unit web site, which contains the online documentation for Test::Unit. You can also download the complete Test::Unit package from there so you can see the unit tests that drove the development of Test::Unit.
NOTE
Developers with a background in Java might also want to check out the JUnit site, which is the site for the Java version of the Unit Testing framework. It contains many interesting articles and experience reports of Extreme Programming style test-driven development in Java.
When learning to write good unit tests, the thing to remember is that unit testing is intended to validate that a class and its methods do what the developer intended them to do. Unit tests are not intended to replace functional or acceptance testing; rather, they ensure that the code we release to functional testing is better than it would be without our unit tests. With that in mind, at first what matters is that you start to write the unit tests first. Once you've started writing the tests, you'll quickly learn what extra tests you need to write.
You know that you need to write extra assertions in your unit tests when mistakes slip through your unit test suite to cause grief later. Whenever that happens, rather than starting the debugger, or running the code interactively in an irb session [ref "Ruby for the Nuby" article], write more assertions into your test cases. One or more of these assertions will give you enough "pertinent information" to enable you to localize the mistake.
Unit tests can be applied to any class, even those that use lots of other objects. The two Ruby scripts below are the start of the test suite that was used to create the Odometer example from the last article. The first script replaces the OdometerTest.rb from last time, which showed how to test the Odometer using irb, but this format is so much easier as it avoids the need to manually check the printed output. The second script is just an easier way to run all of the test cases with a single command line.
# tc_Odometer.rb
require 'test/unit'
require "CircularCounter.rb"
require "OdometerWheel.rb"
require 'Odometer.rb'
class TC_Odometer < Test::Unit::TestCase
def set_up
@odo = Odometer.new(6,10)
@bin = Odometer.new(8,2)
end
def test_increment
12345.times { @odo.increment }
assert_equal("012345", @odo.reading, "Wrong reading")
32.times {@bin.increment}
assert_equal("00100000", @bin.reading, "Wrong reading")
end
def tear_down
@odo = nil
end
end
# ts_OdometerProject.rb require "tc_Odometer.rb" require "tc_CircularCounter.rb"
So start practicing test-driven development by adding more assertions to this test suite and then adding whatever functionality you desire to the Odometer. When you feel comfortable with that, start practicing with your own projects.
NOTE
Although there are several books on Ruby, my personal favorite is still Programming Ruby: The Pragmatic Programmer's Guide (Addison-Wesley, 2000, 0-201-71089-7) by the The Pragmatic Programmers, Andy Hunt and Dave Thomas. The first place to visit though is the main Ruby site, because that's where you can find the places to download Ruby, as well as a wealth of articles and tips about using Ruby.
All of this sample code was tested with a fresh download of Ruby version 1.6.6.