- Needed: Rapid Feedback on Environmental Performance
- Simple Solution to a Complex Problem
- Implementing the Solution in a Simple Automation Script
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Implementing the Solution in a Simple Automation Script
Keep in mind that while my example shows implementation in a single test script, integrating this type of information-gathering in an automation framework is much more practical and powerful. The following Ruby code is for a WATIR script that opens Internet Explorer to http://www.bookpool.com, performs a search on Ruby, and verifies that seven books are returned:
#includes
require '../watir.rb' # the controller
#variables
testSite = 'http://www.BookPool.com'
#open the IE browser to http://www.BookPool.com
$ie = IE.new
$ie.goto(testSite)
puts 'Step 2: enter "Ruby: in the search text field under Simple Search'
$ie.textField(:name, "qs").set("Ruby") # qs is the name of the search field
puts 'Step 3: submit the search form.'
$ie.form(:index, "1").submit #submitting first form found in the page.
puts 'Expected Result: '
puts ' - 7 results should be shown.'
a = $ie.pageContainsText("All 7 results for Ruby:")
if !a
puts "Test Failed! Could not find test string: 'All 7 results for Ruby:'"
else
puts "Test Passed. Found the test string: 'All 7 results for Ruby:.'"
end
puts 'Close the browser'
$ie.close()
Note that if you download the sample code, there will be more comments.
The puts command in Ruby simply outputs the following string to the console window. Everything else should be fairly self-explanatory even if you don't know Ruby. We create a new browser ($ie), navigate to http://www.bookpool.com, enter our search criteria into the search field with the name qs, click Search, and then check the results using an assertion.
If we take the same script and simply wrap each of the page transitions with a timer, we get the following:
#includes
require '../watir.rb' # the controller
#variables
testSite = 'http://www.BookPool.com'
executionEnvironment = 'Test' #This could be read from a configuration file at runtime
beginTime = 0
endTime = 0
#open spreadsheet
timeSpreadsheet = File.new( Time.now.strftime("%d-%b-%y") + ".csv", "a") #Note this creates a new file every day...
#open the IE browser to http://www.BookPool.com
$ie = IE.new
beginTime = Time.now
$ie.goto(testSite)
endTime = Time.now
#Log the time for the Home Page to load
timeSpreadsheet.puts executionEnvironment + ",Home Page," + (endTime - beginTime).to_s
puts 'Step 2: enter "Ruby: in the search text field under Simple Search'
$ie.textField(:name, "qs").set("Ruby") # qs is the name of the search field
puts 'Step 3: submit the search form.'
beginTime = Time.now
$ie.form(:index, "1").submit #submitting first form found in the page.
endTime = Time.now
#Log the time for the search
timeSpreadsheet.puts executionEnvironment + ",Time to execute search," + (endTime - beginTime).to_s
#All 7 results for Ruby:
puts 'Actual Result: Check that the "All 7 results for Ruby:" message actually appears'
a = $ie.pageContainsText("All 7 results for Ruby:")
if !a
scriptLog.puts "Test Failed! Could not find test string: 'All 7 results for Ruby:'"
else
scriptLog.puts "Test Passed. Found the test string: 'All 7 results for Ruby:'"
end
puts 'Close the browser'
$ie.close()
#close the file
timeSpreadsheet.close
Note that if you download the sample code, there will be more comments.
By simply wrapping the page transitions with start and end times, we can calculate the load times for each page and create a makeshift timer. Implementation for this tool is almost identical in Rational, as I imagine it would be with any tool. Notice at the start of the script that we set the environment in which the script is running. This information could be read into the script at runtime, providing the greatest flexibility (at the very least, you could use a global variable in most tools). It might also be possible to use a gettime() function, parse it into a variable, and use simple subtraction to calculate the actual load time.
Executing this script produces a spreadsheet similar to the one in Figure 5.
)
When you're ready to format the data, you can load the macros included at the end of the file, or simply copy the data into a spreadsheet that has the macros.
All said and done, gathering this performance data allowed us to fix problems, better leveraging our automation framework, and gave us a gold star with project management. I've rarely seen so little work pay off so richly. If you happen to be in an environment where implementing this type of performance testing makes sense, I encourage you to try it. Implementation was trivial; four hours for one person, in my case, but it will depend on how your scripts are structured. The benefits are substantial in both practical ways (fix problems) and political ways (wow management). While this isn't a complete performance test, it's a first step that ensures that major performance problems are detected early, leaving the performance testing team to focus on load, stability, and concurrency.
For more information, check out the following:
- Sample code and spreadsheets for this article
- WATIR project home page
- RubyForge project for the tool
- Web Testing with Ruby (WTR) Wiki, where it all started (I think)