As a professional software engineer you have every intention of making your project the best. You religiously practice TDD/BDD, write clean code, engage in pair programming and perform thorough code reviews. Inevitably you get to a point where you notice technical debt gathering in the small corners of your code base. A few refactoring rounds and a some upgrading of libraries and you’re back on track. Until you notice this big hairy monster glaring you right in the face.
The wonderful world of cucumbers and capybaras
Cucumber is advertised as Simple, human collaboration. Simply write your features and scenarios in plain English (or whatever language floats your boat):
1 2 3 4 5 6 7 8 9 10 Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two numbers Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen
The idea is that you can easily discuss features with customers, focussing on the behaviour or the application instead of the nitty gritty technical details. Sounds good! The next step involves a developer writing some regular expression powered steps to actually perform these steps somewhere. In the case of Ruby on Rails it’s common to use a library named Capybara. This acceptance test framework virtualizes a browser window and can take scripted instructions to click on links, fill out forms and inspect page content.
When you start out development for a new project this works great and at Sprint Demo 1 you can proudly show the customer that their user stories, along with several usage scenarios, is automatically tested and reported as working.
Fast Forward six months.
Trouble in paradise
Development is going fast. You’re adding features every sprint like a boss and your team is happy and the customer satisfied. Your cucumber tests have helped you tremendously, preventing you from deploying broken code to production several times. Yah for TDD/BDD!
The time has come that you need to do some optimalisation in your application. Several pages have become a bit overweight and need to be optimized for a better user experience. One of the things you consider is loading parts of the page on demand with AJAX. This is where things start to break down.
From your entire set of features, there’s this one scenario that now needs to load part of the page on demand with AJAX. Simply put, the page now shows a button, when clicked, the browser will fetch some HTML and update the page with the new content.
You quickly add a
save_and_open_page to inspect the page your test is running against and try to spot something obviously wrong. Nope, it all looks like it should. After banging our head onto your keyboard several times you learn that
You run the entire feature suite to make sure you didn’t break anything. Hmz, 73 failures?! An obscure error message about some element not being found, but
"" was found? Wait, what? Turns out that other features expect to see some content you’re now loading with AJAX. All those features need to be updated as well.
Now you have another problem! Some of those features check content that is hidden by default, but is made visible by a simple jQuery
Where to go from here?
In my opinion we’d be much better off if the defaults for acceptance test frameworks would include:
- Visbility is user visibility
- Use one efficient driver instead of different drivers with different tools, options and features.