Hello Storyteller!
Every time a commit is made, we run 19 builds consisting of nearly 5000 tests. A sizeable portion of those tests are regression tests that fire up a web browser and exercise the UI. Our regression tests are particularly valuable to our testing suite. They let us make very significant changes with confidence. Once our regression tests are green, we’re done (that’s the goal anyway). Here are some examples of some big changes we’ve made; none taking more than a few days start to finish.
- Converting from integer to GUID based primary keys
- Upgrading third party binaries with breaking changes
- Upgrading third party javascript libraries
- Turning on lazy loading with NHibernate
I’m going to briefly introduce Storyteller, the tool we use for all of our UI based regression testing. The rest of the post is a brief introduction to the StoryTeller UI, the tool we use for authoring our regression tests.
There’s a picture of the Storyteller UI with one of our tests loaded. Hopefully the key points are apparent in the image. The side bar on the left has the test outline and the middle panel has a better visual representation of the test. There’s some common IDE buttons at the top that are test related – run, cancel and save. Hitting the run button will execute the test and then it presents you with the results:
This is hopefully the image you see after Storyteller finishes executing a test. The green bar at the top indicates success and every assertion you make in the test is highlighted. It’s a quick, easy visual indicator of whether or not your test passed or failed.
This was just a quick, high level post on the Storyteller UI and how we’re using it to facilitate our testing here at Dovetail. There’s some really cool stuff that’s going on behind the scenes to make all of that possible. I’ll discuss all of that in a later post.
