1. Multiply each item in a list by 2

(map #(* 2 % ) (range 1 10))

2. Sum a list of numbers

(reduce + (range 1 1000))

3. Verify if exists in a string

(def words ["clojure", "stm", "compojure", "leiningen", "clojure.core"])
(def tweet "This is an example tweet talking about clojure and leiningen.")

; 3a
(reduce #(or %1 (.contains tweet %2)) false words)

; 3b
(some #(.contains tweet %) words)

This is still ugly in that it uses String.contains whereas it really should split the string into word tokens – I’ll leave that as an exercise for the reader.

4. Read in a file

(ns ten
  (:require
    [clojure.java.io :as io]))

; 4a
(slurp "data.txt")

; 4b
(take 3 (line-seq (io/reader "data.txt")))

; 4c
(with-open [rdr (io/reader "http://oldweb.cecm.sfu.ca/projects/ISC/dataB/isc/C/pi10000.txt")] (take 3 (line-seq rdr)))

; 4d
(with-open [rdr (io/reader "http://oldweb.cecm.sfu.ca/projects/ISC/dataB/isc/C/pi10000.txt")] (doall (take 3 (line-seq rdr))))

However for a bigger file, it’s good to use clojure’s innate laziness – 4b reads *only* the first 3 lines of the file, if you don’t need the rest, it doesn’t read it.

Unfortunately, we never close the file handle in 4b, which is a bad thing to do. 4c uses ‘with-open’ to read a large file, and then close it once we’re done – much safer, but sadly this throws an IOException – because of clojure’s laziness. The ‘(take 3 (line-seq rdr))’ function is evaluated lazily, when the “println” function is called – by which time, the file handle is closed!

So 4d calls ‘doall’ which forces the sequence to be evaluated immediately. This is getting a bit long for a one-liner – but it’s very flexible; it only reads as many lines as are needed, which for large files or other streams, is extremely useful.

5. Happy birthday!

; 5a
(map #(str "Happy Birthday " (if (= % 3) "dear NAME" "to You")) (range 1 4))

; 5b
(for [x (range 1 5)] (println (str "Happy Birthday " (if (= x 3) "dear NAME" "to You"))))

; 5c
(doseq [x (range 1 5)] (println (str "Happy Birthday " (if (= x 3) "dear NAME" "to You"))))

5b uses ‘for’, which is much clearer for most folks. The only trouble in this example is that ‘for’ actually returns a list of values, so calling 5b results in the output:

5b: (Happy Birthday to You
Happy Birthday to You
Happy Birthday dear NAME
Happy Birthday to You
nil nil nil nil)

5c uses ‘doseq’ which has a syntax just like ‘for’ but just calls the side effects of the loop (i.e. the “println” functions) and ignores the result. So the output is:

Happy Birthday to You
Happy Birthday to You
Happy Birthday dear NAME
Happy Birthday to You
5c: nil

6. Filter list of numbers

(group-by #(if (> % 60) :passed :failed) [49, 58, 76, 82, 88, 90])

6: {:failed [49 58], :passed [76 82 88 90]}

Note that clojure has a ‘partition-by’ function similar to scala’s ‘partition’, but with a subtly different behaviour – it will start a new list every time a function changes, so (partition-by #(> % 60) [49, 58, 76, 82, 88, 90, 33]) gets messed up by the final number, resulting in “((49 58) (76 82 88 90) (33))”. Group-by matches the original requirements better.

7. Fetch and parse an xml feed

(ns ten
  (:require
    [clojure.xml]))

(clojure.xml/parse "http://search.twitter.com/search.atom?&q=clojure")

8. Find minimum in a list

; 8a
(reduce min [14, 35, -7, 46, 98])

; 8b
(min 14 35 -7 46 98)

; 8c
(apply min [14, 35, -7, 46, 98])

8b is better – the “min” function can take multiple arguments, and finds the minimum of all of them.

If you have an array already, you can use ‘apply’ to pass the array as parameters to ‘min’, as in 8c.

9. Parallel processing

(ns ten
  (:require
    [clojure.java.io :as io]))

(pmap #(count %) (line-seq (io/reader "http://oldweb.cecm.sfu.ca/projects/ISC/dataB/isc/C/pi10000.txt")))

10. Sieve of Eratosthenes

Clojure is very big on laziness… and I’m feeling to lazy to re-implement the Sieve in a one-liner. I’m sure it could be done! In the meantime, I’ll point you to cgrand’s great article about the Sieve of Eratosthenes in clojure.

Phew – it was far more work blogging about these than it was solving the problem.

If you want more clojure, I recently wrote a simple Twitter web app in clojure (and coffeescript) – I might blog about that in more detail at some stage, but in the meantime there’s a lot of info in the readme.

Update – removed redundant printf statements. sorry!

One of my main problems is that static typing sometimes makes it hard to test without jumping through a lot of hoops. Consider this code:

  def ingestAllFromDatabase() {
    solrIngester.clear
    personRepository.foreach(solrIngester.addDocument(_))
    solrIngester.commit()
  }

It’s nice concise code, relatively readable – this method sets up an ingestor, traverses a database, and for every record in the database it ingests it. Should be relatively straightforward to unit test, right?

The trouble is, to properly test it, I’d like the personRepository and the solrIngester to both be mocked out. They aren’t part of the code under test – they’re separate classes, and I want to test how the function under test interacts with them. Specifically, I’d like to test that the inline function “solrIngester.addDocument(_)” is passed to the function “personRepository.foreach()”.

(an aside – yes, this could be structured better. But if you wait to structure your code perfectly before you write any unit tests, it’s awfully hard to TDD anything…)

Well, I found a solution: (Note, we’re using the Specs test library, with Mockito matchers – other tools can probably do something similar, somehow, I hope…)

  def stubForEach[T](list: List[T]): (Any) => Unit = {
    providedFunction => {
      val function = providedFunction.asInstanceOf[(T) => Unit]
      list.foreach {
        value =>
          function.apply(value)
      }
    }
  }

  "Ingestion Manager should" {
    "ingest all people from the database" in {
      val solrIngester = mock[SolrIngester]
      val personRepository = mock[PersonRepository]
      val ingestionManager = new IngestionManager(ldifLoader, solrIngester, personRepository)
      val person1 = mock[Person]
      val person2 = mock[Person]

      personRepository.foreach(any) answers { stubForEach(List(person1,person2)) }

      ingestionManager.ingestAllFromDatabase

      there was one(solrIngester).clear
      there was one(solrIngester).addDocument(person1)
      there was one(solrIngester).addDocument(person2)
      there was one(solrIngester).commit
    }
  }

It doesn’t actually use a matcher for the passed function – I couldn’t work out a way to manage that. Instead, it builds a stub function object that behaves like foreach(), but uses a hardcoded list of person objects – and then gets the mock personRepository object to use the stub function object instead of the real foreach function

This illustrates the good and bad things about Scala. You couldn’t do this in Java. You can do it in Scala. But it took a while to figure it out – and even longer to explain it to the rest of the team. And the test code is longer and (worse) harder to understand than the code under test

After using ruby and rspec for a long time, this just feels like waste. I’ve heard static typing fans talking about how it gives you for free, a whole lot of testing you’d have to do manually in a dynamic language. But the rest of your testing can be very painful indeed.

Update – the ruby/rspec equivalent of the above:

describe IngestionManager do
  it "should ingest all the people from the database" do
    ingester = mock('ingester')
    repository = mock('repository')
    manager = IngestionManager.new(ingester, repository)
    person1 = mock('person1')
    person2 = mock('person2')

    ingester.should_receive :clear
    repository.should_receive(:each).and_yield(person1).and_yield(person2)
    ingester.should_receive(:addDocument).with(person1)
    ingester.should_receive(:addDocument).with(person2)
    ingester.should_receive :commit

    manager.ingestAllFromDatabase
  end
end

I’ll try a new tack – post whatever comes up, briefly, and then if I find the time, post a more detailed follow-up.  Maybe that will get the juices flowing again.

A brief summary of the last year:

• I ran an awesome R&D team – and we did lots of awesome stuff(tm) – the source of most of the geekery below, though a fair bit was done in my own time as well
• I did a lot of Ruby / Sinatra / Javascript UI building – it’s a great way to rapidly develop web apps, though the javascript is growing and the ruby is shrinking these days.  Must find time to do some Node.js
• I played a chunk with html5 and css3 – wonderful changes in the web UI world, that are making pretty sites achievable, without the horror of old-school html
• I used a bunch of NoSQL databases, mostly MongoDB – it’s a little dodgy in parts, but it’s an awesomely simple and powerful beast.  And it makes simple app development a dream.
• I did some big-data mangling on Amazon EC2 – EC2 is a wonderful magical kingdom for processing data.  It’s a pity that corporate masters tend to completely screw it up.
• I did a bunch of reading about Lean, Agile, Kanban, Motivation, Change, and just to branch out a bit, Design and Typography.
• Most recently, I’m in a new team that is using Scala.  I’m growing a complex love-hate relationship with Scala – I’ll have to write up my thoughts about it soon.
• And I put up a few bits and pieces on github

Phew.  Lots of scope for more posts.  Lets see if I can keep it up!

(Note, on the usual question of browser support – check out the webfonts wiki page, or the nifty “When can I use” page. The short answer is, @font-face is supported in *all* desktop browsers, as long as you can get a font in the right format.)

Anyway, the point of this post is not to describe how @font-face works – you can find that information all over the web, and there are many good free (and non-free) fonts out there.

But I did want to point folks to Tim Brown’s Web Font Specimen, a nifty way to look at web fonts. I found this when trying to wade through a pile of free fonts, looking for nice ones. The only problem was, it needed editing by hand to change the font specimen – too much work.
“That’d be easy to automate with Sinatra” I thought, so I went looking on github to see if anyone had done it, and found the web font specimen Sinatra mini-app, by Robert J Whitney. Which was neat, but still involved editing a sass file to specify the font you want to view.
This wasn’t quite what I wanted – (a) I’m not a big fan of Sass, I prefer something closer to CSS like Less, and (b) it only showed a single font, and needed configuration to change fonts, while I wanted to browse several.

So I forked it – see my shiny new Sinatra app on github.
It actually ended up being more of a reboot than a fork – I undid a pile of other people’s hard work, and reverted the sass and haml files back to their original css and html, then added some logic to scan font directories dynamically – all the details are in the code and the readme.
But this is how it looks, once you install some fonts and load up the main index page:

Click on a font, and it takes you to a the Specimen page – this is basically Tim Brown’s original page, unmodified (except for a ‘back’ link):

… And that’s about it.  I’m probably not going much further with this – it’s pretty handy as-is, for browsing your local font files and seeing what looks nice on different browsers.  If anyone wants any changes, feel free to hassle me – or fork the project yourself!

I’m using a couple of different applications that allow you to update them by e-mail – mostly, task management apps. I’m teetering between using Remember the Milk and Runway to manage my to-do tasks. Both of them have nice web interfaces, but adding a new idea/task via a web interface is a hassle – you have to be online, and you need to use a Browser UI – and even though both have nice mobile UIs, they are overkill for just adding a new “to be categorised” item.

Thankfully, both have magic e-mail addresses that can receive tasks as mail. And on an Android phone, it’s very simple indeed to fire off a new mail – you can compose an e-mail and hit “send” or “save as draft”, and when the phone next syncs mail, it’ll be sent or put into your ‘drafts’ folder on gmail. Sure, you can do the same with a standalone app such as Astrid, or the RTM-specific Android app (which I’ve never tried), but again, for basic “I need to remember this” tasks, a full app is overkill; and while Astrid will auto-sync, I trust the Gmail auto-sync rather more!

Anyway, to the shortcut business. Rather than loading up the whole gmail client every time, you can simply make a desktop shortcut which loads the “Compose Mail” interface. (Note, I have a HTC Hero, which has HTC’s custom Sense UI, so some of this may not apply to non-Sense phones). The steps are as follows:

1. Add the mail address to your gmail contacts, however you like (I use a desktop web browser generally, why bother doing this on the phone? Just add the contact in gmail, and wait for it to sync to the phone’s contacts list)
2. On the phone, press the “+” button on the bottom-right of the screen, or however you add icons in your Android OS
3. Select “Shortcut” (not “widget”, strangely)
4. Select “Person”
5. Choose the newly added contact
6. Select “send mail” – and a “send mail to XXX” icon will be put on your desktop

If you find you have a lot of shortcuts, it’s well worth putting them in folders – I have a folder called “Planning” that has my “send to RTM” shortcut, plus bookmarks for the web interface, links to google calendar, etc.

Note this is also how you make a shortcut to send a SMS or to call a person. Very handy for quick contacts for loved ones!

class TagPanel
  constructor: (width, height, paper) ->
    @width: width
    @height: height
    @paper: paper
    @boxes: []

  add_box: (box) ->
    box.display(@paper)
    @boxes.push box

  animate: (count) ->
    for i in [1...count]
      for box in @boxes
        box.step()

this.TagPanel: TagPanel # make global

compared with the JavaScript output:

(function(){
  var TagPanel;
  TagPanel = function TagPanel(width, height, paper) {
    this.width = width;
    this.height = height;
    this.paper = paper;
    this.boxes = [];
    return this;
  };
  TagPanel.prototype.add_box = function add_box(box) {
    box.display(this.paper);
    return this.boxes.push(box);
  };
  TagPanel.prototype.animate = function animate(count) {
    var _a, _b, _c, _d, _e, _f, _g, _h, _i, box, i;
    _a = []; _d = 1; _e = count;
    for (_c = 0, i = _d; (_d <= _e ? i  _e); (_d <= _e ? i += 1 : i -= 1), _c++) {
      _a.push((function() {
        _f = []; _g = this.boxes;
        for (_h = 0, _i = _g.length; _h <; _i; _h++) {
          box = _g[_h];
          _f.push(box.step());
        }
        return _f;
      }).call(this));
    }
    return _a;
  };
  this.TagPanel = TagPanel;
  // make global
})();

However, I think it fails (for me) for a few reasons; some of these may be resolved over time, some may not.
I think the main problem is that it's too different to Javascript. This is the same reason I'm not a fan of HAML or SASS, but quite like LessCSS – if you are too different to the core language, a number of things no longer work:

• Syntax highlighting and IDE support are minimal
• You aren’t learning enough about the core language
• You can’t easily use existing code snippets or examples, with mentally de-compiling them back to CoffeeScript

LessCSS, or in-language templating tools like Mustache or even good ol’ ERB, let you mix the old and the new; sometimes you IDE still gets screwed up by the differences from the core language, but you can generally work it out.

I have a few other minor issues with CofeeScript – I’m not such a big fan of Python-style indentation-as-code; I used to be more of a fan, and I could learn to love it again, but without IDE support it’s a bit of a pain. Also there are a few other things I’d quibble about; but they are just that, quibbles, most of it is just fine as a language.

But a key thing for me is that I’m still learning to do Javascript properly – after years of using it badly as a make-my-browser-clicks-work tool, I went to a talk by Doug Crockford, and realised I need to learn this language properly. And that means becoming familiar with the language itself, learning the right way and the wrong way to do things; using CoffeeScript encourages a few particular idioms, which work well, but I’m not learning beyond those idioms.

So I’ll take my code spiked in CoffeeScript and try to do it properly in JavaScript, probably working on TDDing it with JSpec or other such tools. Maybe when I’m completely confident in JavaScript I’ll come back to the sweet smell of the Coffee…

So why would you use it? Because it bypasses some of the horribleness of Javascript syntax, and implements a few handy patterns, like anonymous function creation, in much clearer code.

For example, the following Coffeescript:

square: (x) ->
alert 'squaring: ' + x
x * x

converts to the following javascript:

square: function square(x) {
  alert('squaring:' + x);
  return x * x;
}

There are many more examples at the Coffeescript home page.

I’m not 100% sold on this, as I’m still learning (proper) Javascript, and this amplifies some of my ignorance. For example, I really struggle with ‘this’ and functions/objects – and while Coffeescript automatically enforces some good Javascript behaviour, when I do something wrong it’s a bigger than usual struggle to work out why.
A longer example: I’ve been playing with Raphael (and JQuery) and decided to do some fiddling in Coffeescript:

app: {
  init: ->
    app.paper: Raphael('playspace',320,200)
    app.draw_circle()
    app.draw_square()

  # simple circle with event-based on-click event
  draw_circle: ->
    circle: app.paper.circle(50,40,30)
    circle.attr("fill","#f00")
    circle.attr("stroke","#fff")
    $(circle.node).click((e) -> app.kick(e))

  # square with object-based on-click event
  draw_square: ->
    square: app.paper.rect(150,40,50,30)
    square.attr("fill","#0f0")
    square.attr("stroke","#fff")
    $(square.node).click((e) -> app.punt(square,e))

  kick: (e) ->
    app.paper.text(e.pageX + 10, e.pageY + 10, "Raphaël\nkicks\nbutt!")

  punt: (obj,e) ->
    obj.translate(10,10)
}

# make it globally visible - not really needed, handy for debugging
if window?
  window.APP: app

$(app.init)

The javascript version is:

(function(){
  var app;
  app = {
    init: function init() {
      app.paper = Raphael('playspace', 320, 200);
      app.draw_circle();
      return app.draw_square();
    },
    // simple circle with event-based on-click event
    draw_circle: function draw_circle() {
      var circle;
      circle = app.paper.circle(50, 40, 30);
      circle.attr("fill", "#f00");
      circle.attr("stroke", "#fff");
      return $(circle.node).click(function(e) {
        return app.kick(e);
      });
    },
    // square with object-based on-click event
    draw_square: function draw_square() {
      var square;
      square = app.paper.rect(150, 40, 50, 30);
      square.attr("fill", "#0f0");
      square.attr("stroke", "#fff");
      return $(square.node).click(function(e) {
        return app.punt(square, e);
      });
    },
    kick: function kick(e) {
      return app.paper.text(e.pageX + 10, e.pageY + 10, "Raphaël\nkicks\nbutt!");
    },
    punt: function punt(obj, e) {
      return obj.translate(10, 10);
    }
  };
  // make it globally visible - not really needed, handy for debugging
  (typeof window !== "undefined" && window !== null) ? (window.APP = app) : null;
  $(app.init);
})();

Now, this works, it’s a lot more readable than the Javascript version, and it’s nifty how Coffeescript wrapped everything in an anonymous function so as to not pollute the global namespace (except where I did so deliberately).

But this was about my 8th try.  I had endless problems getting the ‘click’ handlers to hook into the ‘kick’ and ‘punt’ methods.  I tried (initially) using ‘this.kick’ but it failed as when the click handler is called, this is bound elsewhere. I tried storing ‘this’ in a variable, but had pain with the fact that somewhere in the mess of anonymous function wrappers, ‘this’ never seemed to mean the right thing.

In the end, I hit on storing the whole thing as an object ‘app’ – I was already going to make it part of ‘window.APP’ but didn’t want to prefix all calls with ‘APP’… but in the end, that’s kind-of what I did, only the global APP variable isn’t actually needed, just the local ‘app’ variable.

Anyway, I’m sure there is a better way to do this – but I haven’t found it, and this one works. Now I have to decide if I want to keep playing with Coffeescript, having worked all this out, or go back to ugly old Javascript, where at least there are hundreds of net resources to help me when I get stuck!

Many agile dev teams use Delcom USB build lights:
http://delcomproducts.com/productdetails.asp?productnum=804005

Coding for these is apparently quite complex… unless you are on a recent-ish Linux machine, in which case it’s amazingly trivial.

When you plug the light in, if you have the right modules loaded, you’ll get a set of magic files in a directory somewhere under:
/sys/bus/usb/drivers/usbled – on my PC with my light they show up under /sys/bus/usb/drivers/usbled/7-2:1.0, and you only need to care about 3 little virtual files:

-rw-rw-rw- 1 root root 4096 2010-02-12 16:26 blue
-rw-rw-rw- 1 root root 4096 2010-02-12 16:26 green
-rw-rw-rw- 1 root root 4096 2010-02-12 16:26 red

Those 3 files control the 3 components of the light. You can set or get the colour trivially from the shell, once you know the name:

$ cat /sys/bus/usb/drivers/usbled/7-2:1.0/blue
0
$ echo "1" > /sys/bus/usb/drivers/usbled/7-2:1.0/blue
# Blam!  The light goes blue!
$ cat /sys/bus/usb/drivers/usbled/7-2:1.0/blue
1
$ echo "1" > /sys/bus/usb/drivers/usbled/7-2:1.0/red
# and now the light goes purple - red + blue!

That’s really all you need to know – you can read, and write, the red/green/blue elements of the light. A little ruby wrapper I wrote makes for nicer colour names, and automatically works out the path to the light (ok, it just finds the first directory that looks right in the usbled path…):

#!/usr/bin/env ruby
class UsbLight
  COLOURNAMES = {
      :black => [0,0,0],
      :red => [1,0,0],
      :green => [0,1,0],
      :blue => [0,0,1],
      :cyan => [0,1,1],
      :magenta => [1,0,1],
      :yellow => [1,1,0],
      :white => [1,1,1]
    }

  def initialize
    @usbledpath = '/sys/bus/usb/drivers/usbled'
    @usbleddev = Dir.new(@usbledpath).find { |f| f =~ /^\d-\d.*/ }
    raise "Can't find device in #{@usbledpath}" unless @usbleddev
    @usbled = File.join(@usbledpath,@usbleddev)
    @lights = {}
    %w(red green blue).each do |light|
      @lights[light] = File.join(@usbled,light)
      raise "can't read #{light} light" unless File.readable?(@lights[light])
      raise "can't write #{light} light" unless File.writable?(@lights[light])
    end
  end
  def colour
    lights = [read_light("red"),read_light("green"),read_light("blue")]
    colour_pattern = COLOURNAMES.detect { |colour, pattern| pattern == lights }
    raise "bad colour combo: #{lights.inspect}" if colour_pattern.nil?
    colour_pattern[0]
  end
  def colour=(name)
    lights = COLOURNAMES[name.to_sym]
    raise "unknown lamp colour: #{name}" unless lights
    write_light("red",lights[0])
    write_light("green",lights[1])
    write_light("blue",lights[2])
  end
  def off
    self.colour = :black
  end

  private
  def read_light(name)
    File.new(@lights[name]).read.to_i
  end
  def write_light(name,value)
    open(@lights[name],"w") do |f|
      f.puts(value.to_s)
    end
  end
end

I have a Hudson watcher that uses this to set the light colour – but that’s a subject for another post!

What is Behaviour Driven Development?

BDD is a new-ish term used as a contrast to Test Driven Development – it was coined by Dan North in 2006, as described in his article at http://dannorth.net/introducing-bdd – there’s also a good introduction in wikipedia.

In a nutshell, BDD is all about describing the behaviour of an application, in a plain-text language that can be understood by end users, testers, and developers – and then hopefully automating acceptance testing, so you can prove that at any time, the application meets the BDD description.

How about Cucumber? and Selenium?

Cucumber, which grew out of the Ruby Rspec framework, is a tool to enable the writing of acceptance criteria in a controlled plain-text format, and then running those criteria via some ruby back-end code.

Selenium is a set of tools for automating tests in web browsers.  There are actually two main flavours of Selenium, Selenium IDE that runs in a browser-based gui, and Selenium RC, a client/server version with a java server and clients in a number of languages. We’ll use Selenium RC, with the Ruby front end (as it makes integrating with Cucumber very easy).  You can also use alternative tools such as Watir or Celerity to do similar things in different ways; each has it’s merits and limitations.

Ok, give me an example

I’ve used the Twitter home page as an example, as it has a bunch of asynchronous ajax behaviour, which is very hard to test without automating a web browser.

The scenarios for a related set of Twitter features are defined in a feature file, such as trending_topics.feature :

Feature: Twitter trending topics
In order to tap into the zeitgeist
as a web surfer
I should be able to see what's being discussed on twitter

Scenario: view popular topics
When I visit the Twitter home page
Then I should be able to see popular topics right now

Scenario: search for most popular topics
Given I am on the Twitter home page
And I see the most popular topic
When I search for the most popular topic
Then I should see results containing the most popular topic
And I should see a message indicating more results exist within 40 seconds

The first few lines are a preamble – they define the background of the feature.

The actual scenarios are runnable definitions of how the feature actually behaves.

Making the features runnable

There are several layers of code to make these features actually runnable.

First, we have step definitions – these are code that matches each “Given”, “When” or “Then” step with a regular expression or a plain text string, that identifies the matching ruby logic, and possibly extracts user parameters.  For example, the step “And I should see a message indicating more results exist within 40 seconds” is matched by ruby code like:

Then /^I should see a message indicating more results exist within (\d+) seconds$/ do |timeout|

- the regular expression matches the step, and pulls out the timeout parameter into a variable.

The full code of the steps file is as follows:

twitter_steps.rb

Given "I am on the Twitter home page" do
  @home_page.visit
end

Given "I see the most popular topic" do
  @most_popular = @home_page.most_popular_topic
  puts "Most popular topic: '#{@most_popular}'"
end

When "I visit the Twitter home page" do
  Given "I am on the Twitter home page"
end

When "I search for the most popular topic" do
  @home_page.search(@most_popular)
end

Then "I should be able to see popular topics right now" do
  @home_page.should have_popular_topics
end

Then "I should see results containing the most popular topic" do
  @results_page.tweets.each do |result|
     result[:tweet].downcase.should include @most_popular.downcase
  end
end

Then /^I should see a message indicating more results exist within (\d+) seconds$/ do |timeout|
  @results_page.wait_for_more_results(timeout.to_i)
end

As you can see, the step implementations are very simple – all the heavy lifting is done by page model objects – @home_page and @results page wrap all the logic related to two different web pages.  The twitter_steps.rb file just handles matching the steps, and tracking some state, such as the @most_popular variable.

twitter_home_page.rb defines the home page model:

class TwitterHomePage
  PAGE_URL = "http://www.twitter.com"
  def initialize(world)
    @world = world
    @browser = $selenium_helper.browser
  end

  def visit
    @browser.open PAGE_URL
    @browser.wait_for_page_to_load
    @browser.title.should == "Twitter"
  end

  def has_popular_topics?
    @browser.element? POP_TOPICS_LOCATOR
  end

  def most_popular_topic
    @browser.text FIRST_POP_TOPIC_LOCATOR
  end

  def search(topic)
    @browser.type SEARCH_BOX_LOCATOR, topic
    @browser.click SEARCH_BUTTON_LOCATOR
    @browser.wait_for_element RESULTS_HEADING_LOCATOR
  end

  private

  POP_TOPICS_LOCATOR = %Q{//div[@id = "trends"]//div[@class = "current"]}
  FIRST_POP_TOPIC_LOCATOR = "#{POP_TOPICS_LOCATOR}/ul/li[1]/a"
  SEARCH_BOX_LOCATOR = %Q{//input[@id="home_search_q"]}
  SEARCH_BUTTON_LOCATOR = %Q{//a[@id="home_search_submit"]}
  RESULTS_HEADING_LOCATOR = %Q{//div[@id="content"]/h2[@id="timeline_heading"]}
end

twitter_results_page.rb is for handling search results:

class TwitterResultsPage
  def initialize(world)
    @world = world
    @browser = $selenium_helper.browser
  end

  def tweets
    # ajax results, give them a chance to load
    @browser.wait_for_element(TIMELINE_LOCATOR)
    result_count = @browser.get_xpath_count(TIMELINE_LOCATOR).to_i

    (1..result_count).collect do |count|
      {
        :author => @browser.get_text(tweet_author_locator(count)),
        :tweet => @browser.get_text(tweet_text_locator(count))
      }
    end
  end

  def wait_for_more_results(timeout_secs)
    @browser.wait_for_element(MORE_RESULTS_LOCATOR,{:timeout_in_secs => timeout_secs})
  end

  private

  def tweet_author_locator(count)
    %Q{#{tweet_locator(count)}//a[contains(@class, "screen-name")]}
  end

  def tweet_text_locator(count)
    %Q{#{tweet_locator(count)}//span[contains(@class, "msgtxt")]}
  end

  def tweet_locator(count)
    %Q{#{TIMELINE_LOCATOR}[#{count}]}
  end

  TIMELINE_LOCATOR = %Q{//ol[@id="timeline"]/li}
  MORE_RESULTS_LOCATOR = %Q{//div[@id="new_results_notification"]/a[@id="results_update" and not(contains(@style, "display: none"))]}
end

The page models use a lot of xpath locators to find items in web pages; they work well on most modern browsers but might be inconsistent on IE6 – if you want to test on IE6 you might have to look into other ways to locate elements, such as finding them by ID.

There are some utility classes that set up the environment, user configuration, and the Selenium interface:

env.rb is the main entry point for Cucumber – it is loaded first, before any other ruby files, and it sets up globals and startup/shutdown code:

BASEDIR = File.join(File.dirname(__FILE__),"..") unless defined? BASEDIR
PRJDIR = File.join(File.dirname(__FILE__),"..","..","..") unless defined? PRJDIR
require 'spec/expectations' # rspec extras
require File.join(BASEDIR,'support/selenium_helper.rb')
require File.join(BASEDIR,'support/user_config.rb')

# globals - keep these to a minimum!
$user_config = UserConfig.new
$selenium_helper = SeleniumHelper.new($user_config)  # better a global than a singleton - still need something global as it's used in monkey-patching bits below

at_exit do
  $stderr.puts "global exit block - closing browser"
  $selenium_helper.shutdown
end

module MyWorld
  # add methods here you want accessible from all cucumber steps
end

World(MyWorld)

Before do
  @home_page = TwitterHomePage.new(self)
  @results_page = TwitterResultsPage.new(self)
end

The ‘MyWorld’ module is there as a starting point for your own extensions – generally I have often-needed functions in this module; see the cucumber documentation for more.

The ‘Before’ block is called before every scenario – here it just sets up the page objects, but other per-scenario stuff can also be added here.

Other than that, the main things included are a user config class, that loads user configuration from a file named for the user’s host name (so on “my_pc” it will load a config file called “config/my_pc.config”); see http://gist.github.com/224106 for code and a typical example.

… and the real work of loading Selenium is in the selenium_helper.rb file:

require 'selenium/client'
require 'selenium/rspec/spec_helper'
require File.join(BASEDIR,'support/user_config.rb')

if defined? JRUBY_VERSION
  # jruby has it's own process-handling code, as 'fork' is unreliable in java
  require 'java'
end

class SeleniumHelper
  attr_accessor :max_timeout, :browser
  def initialize(user_config)
    @user_config = user_config
    @max_timeout = 45  # maximum allowed timeout
    @selenium_port = user_config['selenium.port']
    @selenium_browser = @user_config['selenium.browser.name']
    @selenium_process = nil
    @browser = nil
    start_selenium
    start_browser
  end

  def shutdown
    stop_selenium
  end

  private

  def start_selenium
    selenium_jar_path = File.expand_path(File.join(PRJDIR,"lib","selenium","selenium-server.jar"))
    raise "Can't find #{selenium_jar_path}" unless File.exists?(selenium_jar_path)
    cmd = "java -jar #{selenium_jar_path} -timeout #{@max_timeout} -port #{@selenium_port}"

    if defined? JRUBY_VERSION # java magic to run a process
      @selenium_process = java.lang.ProcessBuilder.new(cmd.split(" ")).redirectErrorStream(true).start
      # spawn a thread to redirect background process to log file
      $stderr.puts "selenium process started in background"
      output_stream_to_log(@selenium_process.getInputStream, "selenium.log")
    else # not java - use fork
      @selenium_process = Process.fork do
          # Note: you need to redirect stdout this way
          # - if you try using "cmd > selenium.log" ruby spawns a subprocess, which you can't kill
         $stdout.reopen(File.new("selenium.log", "w"))
         exec cmd
      end
      $stderr.puts "selenium process started with pid #{@selenium_process}"
    end
    sleep 2  # give it a chance to start
    $stderr.puts "Selenium output sent to selenium.log"
  end

  def start_browser
    begin
      base_url = "http://localhost"
      @browser = Selenium::Client::Driver.new("localhost", @selenium_port, @selenium_browser, base_url, @max_timeout)
      @browser.start_new_browser_session
    rescue Exception
      stop_selenium
      raise
    end
  end

  def stop_selenium
    $stderr.puts "killing background selenium task"
    # could open http://localhost:selenium_port/selenium-server/driver/?cmd=shutDown - but this is more reliable!
    if defined? JRUBY_VERSION
      @selenium_process.destroy
      $stderr.puts "and waiting..."
      @selenium_process.waitFor
    else
      Process.kill("HUP", @selenium_process)
      $stderr.puts "and waiting..."
      Process.wait
    end
    $stderr.puts "dead."
  end

  def output_stream_to_log(inputStream, logfilename)
    Thread.new do
      File.open(logfilename,"w") do |f|
        output = java.io.BufferedReader.new(java.io.InputStreamReader.new(inputStream))
        while (line = output.readLine) != nil
          f.puts line
        end
        output.close
      end
    end

  end
end

Note you can just load selenium.jar from a command line, and save yourself much of the effort here – but this will load and unload it for you, which can keep things simpler. There’s some complexity involved in getting it to work both in JRuby (which has to use Java’s ProcessBuilder class) and in vanilla Ruby.

Building and running this example

To get this example up and running, you need:

• Ruby, or JRuby – native Ruby is faster, but if you are in the Java world, JRuby can be handy as it’s 100% java
• Ruby-gems, the ruby packaging system (included in JRuby)
• The ruby gems:
  • rspec (tested with version 1.2.6) – not essential, but adds many nice matchers and features to cucumber
  • cucumber (tested with version 0.3.9)
  • Selenium (tested with version 1.1.14)
  • selenium-client (tested with version 1.2.15)
• Java 1.5 or later (needed for the selenium server)
• The selenium-rc server (version 1.0.1 or later)
  • you really only need the selenium-server jar file, everything else is included in the ruby gems above
  • download selenium-rc from http://seleniumhq.org/download/ , unzip selenium-remote-control-???.zip, and extract the file ‘selenium-server.jar’

The file structure is as follows:

• Root folder – optionally contains a cucumber.yml file (see cucumber docs for more)
  • test/features – base directory for all feature files
    • /trending_topics.feature
    • /step_definitions
      • twitter_steps.rb
    • /support
      • env.rb
      • selenium_helper.rb
      • user_config.rb
      • /pages
        • twitter_home_page.rb
        • twitter_results_page.rb
  • lib/selenium/selenium-server.jar – the selenium server itself

The complete set of sample files (except the selenium jar file!) is available at http://sietsma.com/korny/cuke_sample.zip.

Running it!

To run all the scenarios in the feature file, assuming all the software is installed and in the path as required, run:

$ cucumber test/features/trending_topics.feature

The output should be similar to:

selenium process started with pid 25298
Selenium output sent to selenium.log
Feature: Twitter trending topics
  In order to tap into the zeitgeist
  as a web surfer
  I should be able to see what's being discussed on twitter

  Scenario: view popular topics                           # test/features/trending_topics.feature:6
    When I visit the Twitter home page                    # test/features/step_definitions/twitter_steps.rb:10
    Then I should be able to see popular topics right now # test/features/step_definitions/twitter_steps.rb:18

  Scenario: search for most popular topics                                     # test/features/trending_topics.feature:10
    Given I am on the Twitter home page                                        # test/features/step_definitions/twitter_steps.rb:1
Most popular topic: '#unseenprequels'
    And I see the most popular topic                                           # test/features/step_definitions/twitter_steps.rb:5
    When I search for the most popular topic                                   # test/features/step_definitions/twitter_steps.rb:14
    Then I should see results containing the most popular topic                # test/features/step_definitions/twitter_steps.rb:22
    And I should see a message indicating more results exist within 40 seconds # test/features/step_definitions/twitter_steps.rb:26

2 scenarios (2 passed)
7 steps (7 passed)
0m33.328s
global exit block - closing browser
killing background selenium task
and waiting...
dead.

Note sections in italics are actually green on a terminal, to indicate success. You can also format output as html, for prettier display, or for embedding in a Continuous Integration report.
—

There is far more that could be said on this topic – this is just the tip of the iceberg. Hopefully this is a useful starting point however!

As mentioned earlier, you can download all the code in this article from http://sietsma.com/korny/cuke_sample.zip.