Headless Cucumbers and Capybaras with Selenium and Hudson

Ludo van den Boom

Geschreven door Ludo van den Boom op

Nowadays software engineers can’t live without their favorite test frameworks. But what a software engineer does not want is having to put a lot of effort in keeping these test run. Neither do teams of software engineers want to spent much time on making their test results visible to the entire team.

More and more useful applications, libraries and plugins are appearing that take away some of the pain that is sometimes involved in making software tests useful for entire teams. Examples of excellent tools that we have at our disposal right now are Hudson for continuous integration, Cucumber for integration tests and Selenium for automated testing of web applications in an actual browser. But making all these play together nicely involves more and more setting up of build servers and configuration.

Our goal is to document the steps required to overcome one of the hurdles that we kept running into, namely running our full Cucumber test suite including all Selenium scenarios on our Hudson integration server.

Ingredients

We’re going to use the following ingredients in this post:

  • 1 Debian 5.0.4 ‘Lenny’ installation
  • 1 installation of Xvfb
  • 1 web browser (in our case iceweasel, Firefox rebranded by Debian)
  • 1 Ruby on Rails 2.3.7 application
  • 1 Capybara 0.3.8 gem
  • As many Cucumbers (0.7.3) as necessary

We won’t go into the details of creating a Rails application, installing the gems and writing the Cucumber scenarios here. Their manuals are perfect guides for this.

Installing required packages

We start with installing a couple of packages that allow us to run headless tests in a web browser.

Virtual framebuffer with Xvfb

“In the X Window System, Xvfb or X virtual framebuffer is an X11 server that performs all graphical operations in memory, not showing any screen output.” Source: http://en.wikipedia.org/wiki/Xvfb

$ apt-get install xvfb

Web browser

After installing Xvfb we can go ahead and install a web browser.

$ apt-get install iceweasel

Before we continue we want to configure our browser profile so that it doesn’t whine about tabs being closed or not resuming correctly after it has crashed. If we don’t do this, our test suite might crash or run forever.

$ cd ~/.mozilla/firefox/xxxxxxxx.default/
$ vim user.js

Now put the following two lines in the user.js file:

user_pref("browser.sessionstore.enabled", false);
user_pref("browser.sessionstore.resume_from_crash", false);

Test Display

Before we start running our tests we will ant to verify that all packages were installed successfully. To do so start a virtual framebuffer (an Xvfb session) on display 99 with screen 0:

$ Xvfb :99 -ac -screen 0 1024x768x16

In a different terminal window do:

$ DISPLAY=:99.0 iceweasel http://example.org

This will start our web browser in the virtual framebuffer and open the example.com homepage in this browser. Next up is makinga ‘screenshot’ so that we can see what is going on inside our virtual framebuffer.

$ xwd -root -display :99.0 -out xwdout

And subsequently view the screenshot using:

$ xwud -in xwdout

Do you see the example.org homepage? Then Xvfb and iceweasel were installed successfully and we are now ready to do some testing.

xwud

Running cucumbers

Before we integrate this setup into our continuous integration environment we do a test run to see if the cucumbers work with our new configuration. We can do this using the following command, keep in mind that we have to explicitly tell the cucumbers to use our virtual framebuffer display:

$ DISPLAY=:99.0 rake cucumber

If all goes well then we will see that all scenarios passed. If not all scenarios pass first check if they do run on in a non-headless situation.

Configuring Hudson

Now we have come to the point where we can tie all strings together. We need to add a new build step to the job that needs to run headless cucumbers. But, before adding this build step, we will create a start/stop script for our virtual framebuffer. This script can be used to start a buffer before running the scenarios, and stop the buffer after the scenarios completed. You can save this script to /etc/init.d. Make sure that the permissions are set so that the user that runs Hudson can execute it.

XVFB=/usr/bin/Xvfb
XVFBARGS="$DISPLAY -ac -screen 0 1024x768x16"
PIDFILE=/var/hudson/xvfb_${DISPLAY:1}.pid
case "$1" in
  start)
    echo -n "Starting virtual X frame buffer: Xvfb"
    /sbin/start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS
    echo "."
    ;;
  stop)
    echo -n "Stopping virtual X frame buffer: Xvfb"
    /sbin/start-stop-daemon --stop --quiet --pidfile $PIDFILE
    echo "."
    ;;
  restart)
    $0 stop
    $0 start
    ;;
  *)
  echo "Usage: /etc/init.d/xvfb {start|stop|restart}"
  exit 1
esac
exit 0

Source: http://marc.info/?l=tomcat-user&m=102335321103262&w=2

Final step is to add an ‘Execute shell’ build script to our Hudson job. You can use the following set of commands to run your cucumbers.

#!/bin/bash
export DISPLAY=:99
/etc/init.d/xvfb start
rake cucumber
RESULT=$?
/etc/init.d/xvfb stop
exit $RESULT

After adding this build step we save our job and let Hudson build it. If all goes well the cucumber scenarios should now be running as part of our continuous integration process. You can check the ‘Console output’ page of a Build in Hudson for hints when it didn’t run successfully.

Conclusion

If you have followed along with this post then you should have a working, headless setup by now. The advantage of this setup is that it is pretty light-weight and easy to setup. But this setup is not suited for you if you need to test across multiple browsers on different browsers. For such a setup you will want to have a look at virtual machines and Hudson slave agents.

Reacties

  1. iain zegt:

    Finally got around to trying it, and it works! Thanks guys, awesome!

  2. andy zegt:

    I can confirm that this works with on Ubuntu with Debian package Hudson and RVM. And using FireFox (Ubuntu brand).

    Although there were a few minor adjustments…

    PIDFILE=/var/hudson/xvfb_${DISPLAY:1}.pid

    becomes

    PIDFILE=/var/lib/hudson/xvfb_${DISPLAY:1}.pid

    and my “rake cucumber” is more complex due to RVM

    Nice work, thanks for the write up.

  3. Brad zegt:

    Can you describe what exactly “headless” is referring to?

  4. Headless means that there is no need for a physical display, all screens are output directly to memory. By using a headless setup we are still able to test our Selenium features even though our continuous integration server has no display attached to it, nor has it a real (X) desktop environment installed.

    It does not mean that we had to decapitate anyone for our cucumbers ;)

  5. Vidmantas zegt:

    You might also have to install fonts if “fixed” is missing:

    apt-get install xfonts-base xfonts-75dpi xfonts-100dpi

    Read more here: http://www.wonko.be/2008/01/09/running-openoffice-headless-on-debian

  6. [...] Headless Cucumbers and Capybaras with Selenium and Hudson « Kabisa Blog – March 7th ( tags: cucumber hudson selenium capybara headless testing rails automation howto setup ) [...]

  7. Shashank Soni zegt:

    I am new to this and wanted help.
    Is the procedure same for setting it up in MacOSX?
    Thanks.

Schrijf een reactie

Het e-mailadres wordt niet gepubliceerd. Verplichte velden zijn gemarkeerd met *


9 − = vijf



RT @theguildconf 90 days until The Guild! We are looking for sponsors to make this *awesome* edition happen! Get in touch with us!