This is Signal vs. Noise, a weblog by 37signals about design, business, experience, simplicity, the web, culture, and more. Established 1999 in Chicago. Visit the Product Blog for more information on our products.

The road to fast deploys

A few months ago I was focused on making our deployments much, much faster. I was tired of waiting 5-10 minutes when deploying Basecamp (Yes, I know others have it much, much worse, but that shouldn’t prevent me from wanting it to be even better for myself.) Fast deploys are really critical to rapid and agile iterations: if it is too painful to deploy your application, you’re more likely to wait until later than deploy sooner.

The first tweak I made was a custom Capistrano deployment strategy: FastRemoteCache. However, this still left a lot of data to be copied on each deploy, since we had Rails and all of the plugins used by our applications checked into our repository, directly.

Now, bundling your dependencies with your application does reduce the pain associated with managing different versions of those dependencies. No two of our applications are running precisely the same version of Rails, and having each app maintain it’s own Rails version is a pretty simple way to avoid dependency hell.

However, it means that each deploy has to copy all of rails, and all of its plugins, every time, even though they are very unlikely to change often. It would be nicer if those dependencies could be cached somewhere else, outside of the app, and just have the app point to them. Obviously, RubyGems is one candidate for that, since you can install multiple versions of a single library, but it would mean (first of all) building custom Rails gems, since we often run on Rails versions that aren’t sanctioned releases (yes, we’re that brave!), and (secondly) remembering to install the correct versions of the gems on all of our servers. (We’ve got, um, several of those.)

Caching external dependencies

This CachedExternals plugin basically lets you define your library dependencies in a YAML configuration file in your application. Then, on deploy, those dependencies are checked out into a location specific to the application (the “shared path”, for those with Capistrano-fu). If the needed revision of a library is already checked out there, it is used instead, and since that is the common case, you need move only a much smaller subset of data. By bundling the dependency info with the application, you keep that knowledge where it belongs, and you let the deployment process itself keep track of what needs to be updated, when it needs to be updated.

We use FastRemoteCache and CachedExternals to great effect internally. It has reduced our deploy time for some of our applications to a matter of some 15-30 seconds (though Basecamp is still a minute or two to deploy—better, but plenty of room for improvement, yet).

So, give it a spin and let us know how it works for you!. Better yet, submit patches for the things you think are missing!