Probably the best thing I’ve discovered with my recent playing is Travis CI. I’ve known about it for quite some time, even played with it for simple projects but never with anything with any real complexity. Given this project uses rails which wants a database for pretty much every test and that database has to be postgres because I’m using it’s jsonb support, plus capybara and phantomjs for good measure, this certainly isn’t a simple project to test.
Despite that, getting things running on Travis CI was quick and easy. The container based builds give you a well controlled environment out of the box and they already have a heap of software you’re likely to need ready to go. Other things can be easily provided using a simple .travis.yml file.
While it’s common to have a build server these days, it’s quite uncommon for them to be so well controlled and able to be rebuilt quickly and easily – most wind up being special little snowflakes in some way or other. So building on Travis CI is immediately a massive leap forward in testing practices.
The two things we do at LMAX that I can’t see how to do with Travis CI (yet) is creating a database of historic test results that can be easily queried (at the individual test level, not the build result level), and running a build in parallel with the tests automatically load balanced across a number of hosts. I wouldn’t expect either to be provided as part of the free service anyway but they seem like the next major areas of value it could offer (with significant effort required I imagine).
Still, it’s been a real luxury to have a full CI setup for hacking on a little side project. So thanks for the generosity Travis CI.