The next 10,000 hours

[Please note - if you are familiar with BDD, Cucumber, or Selenium, parts of this may be a tad basic - but I thought it was worth writing a from-scratch guide for those to whom this is unfamiliar territory...]

What is Behaviour Driven Development?

BDD is a new-ish term used as a contrast to Test Driven Development – it was coined by Dan North in 2006, as described in his article at http://dannorth.net/introducing-bdd – there’s also a good introduction in wikipedia.

In a nutshell, BDD is all about describing the behaviour of an application, in a plain-text language that can be understood by end users, testers, and developers – and then hopefully automating acceptance testing, so you can prove that at any time, the application meets the BDD description.

How about Cucumber? and Selenium?

Cucumber, which grew out of the Ruby Rspec framework, is a tool to enable the writing of acceptance criteria in a controlled plain-text format, and then running those criteria via some ruby back-end code.

Selenium is a set of tools for automating tests in web browsers.  There are actually two main flavours of Selenium, Selenium IDE that runs in a browser-based gui, and Selenium RC, a client/server version with a java server and clients in a number of languages. We’ll use Selenium RC, with the Ruby front end (as it makes integrating with Cucumber very easy).  You can also use alternative tools such as Watir or Celerity to do similar things in different ways; each has it’s merits and limitations.

Ok, give me an example

I’ve used the Twitter home page as an example, as it has a bunch of asynchronous ajax behaviour, which is very hard to test without automating a web browser.

The scenarios for a related set of Twitter features are defined in a feature file, such as trending_topics.feature :

Feature: Twitter trending topics
In order to tap into the zeitgeist
as a web surfer
I should be able to see what's being discussed on twitter

Scenario: view popular topics
When I visit the Twitter home page
Then I should be able to see popular topics right now

Scenario: search for most popular topics
Given I am on the Twitter home page
And I see the most popular topic
When I search for the most popular topic
Then I should see results containing the most popular topic
And I should see a message indicating more results exist within 40 seconds

The first few lines are a preamble – they define the background of the feature.

The actual scenarios are runnable definitions of how the feature actually behaves.

Making the features runnable

There are several layers of code to make these features actually runnable.

First, we have step definitions – these are code that matches each “Given”, “When” or “Then” step with a regular expression or a plain text string, that identifies the matching ruby logic, and possibly extracts user parameters.  For example, the step “And I should see a message indicating more results exist within 40 seconds” is matched by ruby code like:

Then /^I should see a message indicating more results exist within (\d+) seconds$/ do |timeout|

- the regular expression matches the step, and pulls out the timeout parameter into a variable.

The full code of the steps file is as follows:

twitter_steps.rb

Given "I am on the Twitter home page" do
  @home_page.visit
end

Given "I see the most popular topic" do
  @most_popular = @home_page.most_popular_topic
  puts "Most popular topic: '#{@most_popular}'"
end

When "I visit the Twitter home page" do
  Given "I am on the Twitter home page"
end

When "I search for the most popular topic" do
  @home_page.search(@most_popular)
end

Then "I should be able to see popular topics right now" do
  @home_page.should have_popular_topics
end

Then "I should see results containing the most popular topic" do
  @results_page.tweets.each do |result|
     result[:tweet].downcase.should include @most_popular.downcase
  end
end

Then /^I should see a message indicating more results exist within (\d+) seconds$/ do |timeout|
  @results_page.wait_for_more_results(timeout.to_i)
end

As you can see, the step implementations are very simple – all the heavy lifting is done by page model objects – @home_page and @results page wrap all the logic related to two different web pages.  The twitter_steps.rb file just handles matching the steps, and tracking some state, such as the @most_popular variable.

twitter_home_page.rb defines the home page model:

class TwitterHomePage
  PAGE_URL = "http://www.twitter.com"
  def initialize(world)
    @world = world
    @browser = $selenium_helper.browser
  end

  def visit
    @browser.open PAGE_URL
    @browser.wait_for_page_to_load
    @browser.title.should == "Twitter"
  end

  def has_popular_topics?
    @browser.element? POP_TOPICS_LOCATOR
  end

  def most_popular_topic
    @browser.text FIRST_POP_TOPIC_LOCATOR
  end

  def search(topic)
    @browser.type SEARCH_BOX_LOCATOR, topic
    @browser.click SEARCH_BUTTON_LOCATOR
    @browser.wait_for_element RESULTS_HEADING_LOCATOR
  end

  private

  POP_TOPICS_LOCATOR = %Q{//div[@id = "trends"]//div[@class = "current"]}
  FIRST_POP_TOPIC_LOCATOR = "#{POP_TOPICS_LOCATOR}/ul/li[1]/a"
  SEARCH_BOX_LOCATOR = %Q{//input[@id="home_search_q"]}
  SEARCH_BUTTON_LOCATOR = %Q{//a[@id="home_search_submit"]}
  RESULTS_HEADING_LOCATOR = %Q{//div[@id="content"]/h2[@id="timeline_heading"]}
end

twitter_results_page.rb is for handling search results:

class TwitterResultsPage
  def initialize(world)
    @world = world
    @browser = $selenium_helper.browser
  end

  def tweets
    # ajax results, give them a chance to load
    @browser.wait_for_element(TIMELINE_LOCATOR)
    result_count = @browser.get_xpath_count(TIMELINE_LOCATOR).to_i

    (1..result_count).collect do |count|
      {
        :author => @browser.get_text(tweet_author_locator(count)),
        :tweet => @browser.get_text(tweet_text_locator(count))
      }
    end
  end

  def wait_for_more_results(timeout_secs)
    @browser.wait_for_element(MORE_RESULTS_LOCATOR,{:timeout_in_secs => timeout_secs})
  end

  private

  def tweet_author_locator(count)
    %Q{#{tweet_locator(count)}//a[contains(@class, "screen-name")]}
  end

  def tweet_text_locator(count)
    %Q{#{tweet_locator(count)}//span[contains(@class, "msgtxt")]}
  end

  def tweet_locator(count)
    %Q{#{TIMELINE_LOCATOR}[#{count}]}
  end

  TIMELINE_LOCATOR = %Q{//ol[@id="timeline"]/li}
  MORE_RESULTS_LOCATOR = %Q{//div[@id="new_results_notification"]/a[@id="results_update" and not(contains(@style, "display: none"))]}
end

The page models use a lot of xpath locators to find items in web pages; they work well on most modern browsers but might be inconsistent on IE6 – if you want to test on IE6 you might have to look into other ways to locate elements, such as finding them by ID.

There are some utility classes that set up the environment, user configuration, and the Selenium interface:

env.rb is the main entry point for Cucumber – it is loaded first, before any other ruby files, and it sets up globals and startup/shutdown code:

BASEDIR = File.join(File.dirname(__FILE__),"..") unless defined? BASEDIR
PRJDIR = File.join(File.dirname(__FILE__),"..","..","..") unless defined? PRJDIR
require 'spec/expectations' # rspec extras
require File.join(BASEDIR,'support/selenium_helper.rb')
require File.join(BASEDIR,'support/user_config.rb')

# globals - keep these to a minimum!
$user_config = UserConfig.new
$selenium_helper = SeleniumHelper.new($user_config)  # better a global than a singleton - still need something global as it's used in monkey-patching bits below

at_exit do
  $stderr.puts "global exit block - closing browser"
  $selenium_helper.shutdown
end

module MyWorld
  # add methods here you want accessible from all cucumber steps
end

World(MyWorld)

Before do
  @home_page = TwitterHomePage.new(self)
  @results_page = TwitterResultsPage.new(self)
end

The ‘MyWorld’ module is there as a starting point for your own extensions – generally I have often-needed functions in this module; see the cucumber documentation for more.

The ‘Before’ block is called before every scenario – here it just sets up the page objects, but other per-scenario stuff can also be added here.

Other than that, the main things included are a user config class, that loads user configuration from a file named for the user’s host name (so on “my_pc” it will load a config file called “config/my_pc.config”); see http://gist.github.com/224106 for code and a typical example.

… and the real work of loading Selenium is in the selenium_helper.rb file:

require 'selenium/client'
require 'selenium/rspec/spec_helper'
require File.join(BASEDIR,'support/user_config.rb')

if defined? JRUBY_VERSION
  # jruby has it's own process-handling code, as 'fork' is unreliable in java
  require 'java'
end

class SeleniumHelper
  attr_accessor :max_timeout, :browser
  def initialize(user_config)
    @user_config = user_config
    @max_timeout = 45  # maximum allowed timeout
    @selenium_port = user_config['selenium.port']
    @selenium_browser = @user_config['selenium.browser.name']
    @selenium_process = nil
    @browser = nil
    start_selenium
    start_browser
  end

  def shutdown
    stop_selenium
  end

  private

  def start_selenium
    selenium_jar_path = File.expand_path(File.join(PRJDIR,"lib","selenium","selenium-server.jar"))
    raise "Can't find #{selenium_jar_path}" unless File.exists?(selenium_jar_path)
    cmd = "java -jar #{selenium_jar_path} -timeout #{@max_timeout} -port #{@selenium_port}"

    if defined? JRUBY_VERSION # java magic to run a process
      @selenium_process = java.lang.ProcessBuilder.new(cmd.split(" ")).redirectErrorStream(true).start
      # spawn a thread to redirect background process to log file
      $stderr.puts "selenium process started in background"
      output_stream_to_log(@selenium_process.getInputStream, "selenium.log")
    else # not java - use fork
      @selenium_process = Process.fork do
          # Note: you need to redirect stdout this way
          # - if you try using "cmd > selenium.log" ruby spawns a subprocess, which you can't kill
         $stdout.reopen(File.new("selenium.log", "w"))
         exec cmd
      end
      $stderr.puts "selenium process started with pid #{@selenium_process}"
    end
    sleep 2  # give it a chance to start
    $stderr.puts "Selenium output sent to selenium.log"
  end

  def start_browser
    begin
      base_url = "http://localhost"
      @browser = Selenium::Client::Driver.new("localhost", @selenium_port, @selenium_browser, base_url, @max_timeout)
      @browser.start_new_browser_session
    rescue Exception
      stop_selenium
      raise
    end
  end

  def stop_selenium
    $stderr.puts "killing background selenium task"
    # could open http://localhost:selenium_port/selenium-server/driver/?cmd=shutDown - but this is more reliable!
    if defined? JRUBY_VERSION
      @selenium_process.destroy
      $stderr.puts "and waiting..."
      @selenium_process.waitFor
    else
      Process.kill("HUP", @selenium_process)
      $stderr.puts "and waiting..."
      Process.wait
    end
    $stderr.puts "dead."
  end

  def output_stream_to_log(inputStream, logfilename)
    Thread.new do
      File.open(logfilename,"w") do |f|
        output = java.io.BufferedReader.new(java.io.InputStreamReader.new(inputStream))
        while (line = output.readLine) != nil
          f.puts line
        end
        output.close
      end
    end

  end
end

Note you can just load selenium.jar from a command line, and save yourself much of the effort here – but this will load and unload it for you, which can keep things simpler. There’s some complexity involved in getting it to work both in JRuby (which has to use Java’s ProcessBuilder class) and in vanilla Ruby.

Building and running this example

To get this example up and running, you need:

• Ruby, or JRuby – native Ruby is faster, but if you are in the Java world, JRuby can be handy as it’s 100% java
• Ruby-gems, the ruby packaging system (included in JRuby)
• The ruby gems:
  • rspec (tested with version 1.2.6) – not essential, but adds many nice matchers and features to cucumber
  • cucumber (tested with version 0.3.9)
  • Selenium (tested with version 1.1.14)
  • selenium-client (tested with version 1.2.15)
• Java 1.5 or later (needed for the selenium server)
• The selenium-rc server (version 1.0.1 or later)
  • you really only need the selenium-server jar file, everything else is included in the ruby gems above
  • download selenium-rc from http://seleniumhq.org/download/ , unzip selenium-remote-control-???.zip, and extract the file ’selenium-server.jar’

The file structure is as follows:

• Root folder – optionally contains a cucumber.yml file (see cucumber docs for more)
  • test/features – base directory for all feature files
    • /trending_topics.feature
    • /step_definitions
      • twitter_steps.rb
    • /support
      • env.rb
      • selenium_helper.rb
      • user_config.rb
      • /pages
        • twitter_home_page.rb
        • twitter_results_page.rb
  • lib/selenium/selenium-server.jar – the selenium server itself

The complete set of sample files (except the selenium jar file!) is available at http://sietsma.com/korny/cuke_sample.zip.

Running it!

To run all the scenarios in the feature file, assuming all the software is installed and in the path as required, run:

$ cucumber test/features/trending_topics.feature

The output should be similar to:

selenium process started with pid 25298
Selenium output sent to selenium.log
Feature: Twitter trending topics
  In order to tap into the zeitgeist
  as a web surfer
  I should be able to see what's being discussed on twitter

  Scenario: view popular topics                           # test/features/trending_topics.feature:6
    When I visit the Twitter home page                    # test/features/step_definitions/twitter_steps.rb:10
    Then I should be able to see popular topics right now # test/features/step_definitions/twitter_steps.rb:18

  Scenario: search for most popular topics                                     # test/features/trending_topics.feature:10
    Given I am on the Twitter home page                                        # test/features/step_definitions/twitter_steps.rb:1
Most popular topic: '#unseenprequels'
    And I see the most popular topic                                           # test/features/step_definitions/twitter_steps.rb:5
    When I search for the most popular topic                                   # test/features/step_definitions/twitter_steps.rb:14
    Then I should see results containing the most popular topic                # test/features/step_definitions/twitter_steps.rb:22
    And I should see a message indicating more results exist within 40 seconds # test/features/step_definitions/twitter_steps.rb:26

2 scenarios (2 passed)
7 steps (7 passed)
0m33.328s
global exit block - closing browser
killing background selenium task
and waiting...
dead.

Note sections in italics are actually green on a terminal, to indicate success. You can also format output as html, for prettier display, or for embedding in a Continuous Integration report.
—

There is far more that could be said on this topic – this is just the tip of the iceberg. Hopefully this is a useful starting point however!

As mentioned earlier, you can download all the code in this article from http://sietsma.com/korny/cuke_sample.zip.

This entry was posted on Tuesday, November 3rd, 2009 at 8:22 pm and is filed under Uncategorized and tagged bdd, cucumber, selenium, tdd, testing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.