Over the past two years, CommerceHub has been diving head-first into config management with Chef. We’ve come to realize that establishing a consistent workflow and common patterns when working with Chef across product teams can save us some pain.
Because Chef cookbooks are code, we needed to treat them as such, yet we lacked the infrastructure and process around working with Chef that we had with our application code. We didn’t have a consistent or automated way to perform static analysis or test our cookbooks. Additionally, individual users had their own processes for versioning and publishing cookbooks to the Chef server, which generally resulted in no versioning at all. We needed to invest the time to incorporate tools and processes into our Chef workflow, similar to our application code.
To address these issues, we decided to codify our workflow and process around some of the best practices in the Chef community with tools, and enforce them with a Continuous Integration process. The result is Sous Chef: an opinionated cookbook that helps set up a Continuous Delivery pipeline on top of Jenkins to perform static analysis, test, and then upload your Chef cookbooks to your Chef server. Sous Chef has an opinion about what your Chef workflow should look like, but is flexible enough that you can inject your own opinions and practices into the workflow. The cookbook starts by setting up a Jenkins server, configuring SSH keys, a knife configuration, and a handful of Jenkins plugins.
This may sound pretty straightforward and not like something that needs to be shared, but what happens next is what sets Sous Chef apart.
The cookbook provides an attribute-driven system for creating jobs on Jenkins to test your cookbooks. Each job is broken into steps representing distinct phases of the deployment pipeline. We stress convention over configuration, but allow the cookbook to be flexible enough to customize any step of a given job.
By default, a Sous Chef job is broken down into five steps:
Step 1: A bundle install is run to install any libraries needed for the testing.
Step 2: RuboCop performs static analysis using Ruby.
Step 3: Foodcritic performs static analysis for cookbook linting.
Step 4: Test Kitchen runs for integration testing, to ensure the cookbook converges and meets all expectations (as outlined in the tests).
Step 5: The cookbook is uploaded to your Chef server with the ‘–freeze’ flag.
Since Sous Chef controls when a cookbook gets added to the Chef server (after all the testing steps and freezing the version), we know that we are only uploading quality, properly versioned cookbooks to Chef.
At this time, we have ~20 internally written cookbooks across six different product teams. Leveraging the pipeline has helped us standardize our process for working on cookbooks across different teams, while providing an automated approach to testing and publishing cookbooks similar to the way we manage application code.