"There are a ton of excellent strategies for speeding up Rails test suites—aggressive use of stubbing/test doubles, decoupling logic from models, avoiding loading Rails entirely—but given the size of our codebase and the velocity with which we’re moving, most of these weren’t immediately feasible. We needed a build system that would allow us to parallelize our test suite so that the real time taken to run the suite was manageable.
Our SRE team went through several different continuous integration solutions in the last year before settling on Solano.
Each of the previous systems had some issue: instability, memory consumption, poor DB management, poor parallelization, painful web UI, you name it. What Solano gives us is an on-premise solution with excellent native support for fanning out tests to multiple threads, running them in parallel, and then assembling the results. It has a great web UI, CLI support, and impressive performance. Since we started using it, our deploy workflow has grown noticeably faster, and the number of wails and anguished GIFs from frustrated engineers is at an all-time low."
"The best way to ensure that local testing was possible was to normalize people’s dev environments. For this we chose Vagrant. This, combined with Chef, allows us to do our local dev in sandboxed Linux instances running locally via VirtualBox in a configuration as similar to production as possible. In addition to making dev environment setup much easier than it used to be, this ensures that each engineer has a consistent environment that is ready to run tests out of the box. The user SSHs into the local linux server and runs spec commands like they would on their host OS, and generally everything Just Works. Most people on our team combine this with Zeus, which allows the Rails environment to be preloaded for lightning fast (relatively speaking) test runs. Both Vagrant and Zeus have their share of issues, but in practice we’ve found them to be a huge time saver."
"Having a CI server building all commits across all branches was a huge first step, but to make this useful we needed to surface the outcome of these builds. This is where GitHub’s commit status API comes in. Every time our CI server begins a build, it pings GitHub’s commit status endpoint, and every time it completes a build it hits the endpoint again with the outcome. Now every open PR includes a yellow/red/green indicator for the branch in question, with a direct link to the build status page on our CI server. In practice this means more transparency, faster feedback cycles, and a guarantee that every branch merged into master has a passing test suite. This integration has been a huge help in keeping our master branch green, and has thus greatly reduced our deploy times (since engineers aren’t waiting on build failures to be resolved in master)."