Today I released version 1.6.0 of Opscode’s apt cookbook. The cookbook itself needed better coverage for testing in Test Kitchen. This post will describe these additions to the cookbook, including how one of the test recipes can actually be used for actual production use. My goal is to explain a bit about how we go about testing with Test Kitchen, and provide some real world examples.
TL;DR – This commit has all the code.
First, the Kitchenfile for the project looked like this:
1 2 3
This is outdated as far as Kitchenfiles goes. It still has the empty array runtimes setting which prevents Test Kitchen from attempting to run additional tests under RVM. We’ll remove this line, and update it for supporting the configurations of the recipes and features we want to test. The cookbook itself has three recipes:
By default, with no configurations defined in a Kitchenfile, test-kitchen will run the default recipe (using Chef Solo under Vagrant). This is useful in the common case, but we also want to actually test other functionality in the cookbook. In addition to the recipes, we want to verify that the LWRPs will do what we intend.
I updated the Kitchenfile with the following content:
1 2 3 4 5
A configuration can correspond to a recipe (
but it can also be arbitrarily named. This is a name used by kitchen
cacher-client recipe isn’t present because
recipe[apt::cacher-ng] includes it, and getting the test to work,
where the single node is a cacher client to itself, was prone to
error. “I assure you, it works” :–). We’ll look at this later anyway.
With the above Kitchenfile, kitchen test will start up the Vagrant
VMs and attempt to run Chef Solo with the recipes named by the
configuration. This is a good start, but we want to actually run some
minitest-chef tests. These will be created inside a “test” cookbook
included with this cookbook. I created a cookbook named
This creates the cookbook scaffolding like normal. I cleaned up the contents of the directory to contain what I needed to start:
1 2 3 4 5 6
The metadata.rb is as you’d expect, it contains the name, a version, maintainer information and a description. The README simply mentions that this is a test cookbook for the parent project. The recipes are the interesting part. Let’s address them in the order of the configurations in the Kitchenfile.
First, the default recipe in the test cookbook. This is simply going
to perform an
include_recipe "apt::default". The way test kitchen
runs, it will actually have the following run list for Chef Solo:
test-kitchen sets up some essential things for Test Kitchen itself.
minitest-handler is the recipe that sets up minitest-chef-handler to
run post-convergence tests.
apt_test::default is the “test” recipe
for this configuration, and finally
apt::default is the cookbook’s
recipe for this configuration named “default”.
Had we not done anything else here, the results are the same as simply running test kitchen with the original Kitchenfile (with runtimes, instead of configurations defined).
Minitest: default recipe
There are now minitest-chef tests for each configuration. The default
recipe provides some “apt-get update” executes, and also creates a
directory that can be used for preseeding packages. We’ll simply test
that the preseeding directory exists. We could probably check that the
cache is updated, but since this cookbook has worked for almost 4
years w/o issue for
apt-get update we’ll trust it continues working
:–). Here’s the test (ignoring the boilerplate):
1 2 3
When Chef runs, it will run this test:
1 2 3
Next, Test Kitchen runs the
cacher-ng configuration. The recipe in
apt_test cookbook simply includes the
The run list in Chef Solo looks like this:
apt::cacher-ng recipe also includes the client recipe, but
basically does nothing unless the
cacher_ipaddress attribute is set,
or if we can search using a Chef Server (which Solo can’t, of course).
Minitest: cacher-ng recipe
The meat of the matter for the
cacher-ng recipe is running the
apt-cacher-ng service, so we’ve written a minitest test for this:
1 2 3
And when Chef runs the test:
1 2 3
Finally, we have our custom configuration that doesn’t correspond to a
recipe in the
lwrps. This configuration instead is
to do a real-world integration test that the LWRPs actually do what
they’re supposed to.
The recipe itself looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
apt recipe is included because otherwise, we may not be able to
apt-get update resource to execute when the new sources.list
is dropped off.
Next, we use Opscode’s very own apt repository as an example because
we can rely on that existing. When Test Kitchen runs, it will actually
write out the apt repository configuration file to
/etc/apt/sources.list.d/opscode.list, but more on that in a minute.
Finally, we’re going to write out an apt preferences file for pinning the Chef package. Currently, Chef is actually packaged at various versions in Ubuntu releases:
1 2 3 4 5
So by adding the Opscode APT repository, and pinning Chef, we can ensure that we’re going to have the correct version of Chef installed as a package, if we were installing Chef as a package from APT :).
When Chef Solo runs, here is the run list:
Notice it doesn’t have “
apt::lwrps”, since that isn’t a recipe in
the apt cookbook.
Minitest: lwrps recipe
The minitest tests for the lwrps configuration and recipe look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13
The first test simply asserts that the Opscode APT sources.list is present. We could elaborate on this by verifying that its content is correct, but for now we’re going to trust that the declarative resource in the recipe is, ahem, declared properly.
Next, we run the
apt-key command to show the available GPG keys in
the APT trusted keyring. This will have the correct Opscode Packages
key if it was added correctly.
Finally, we test that the package pinning for the Chef package is correct. Successful output of the tests looks like this:
1 2 3 4 5
The Real World Bits
The tests hide some of the detail. What does this actually look like on a real system? Glad you asked!
Here’s the sources.list for Opscode’s APT repository.
Next, the apt-key content:
1 2 3 4 5
And the grand finale, the pinning preferences:
1 2 3 4 5 6 7 8 9 10
I used Opscode’s bento box for Ubuntu 12.04, which comes with the ‘omnibus’ Chef package version 10.14.4(-2.ubuntu.11.04). In order to install the newer Chef package and demonstrate the pinning, I’ll first remove it:
Then, I install from the Opscode APT repository:
1 2 3 4
And the package is installed:
1 2 3 4 5 6 7 8 9
Currently the omnibus packages are NOT in the APT repository, since they do not have additional dependencies they are installed simply with dpkg. Don’t use this particular recipe if you’re using the Omnibus packages. Instead, just marvel at the utility of this. Perhaps instead, use the LWRPs in the apt cookbook to set up your own local APT repository and pinning preferences.
Test Kitchen is a framework for isolated integration testing in
individual projects. As such, it has a lot of features, capabilities
and also moving parts. Hopefully this post helps you understand some
of them, and see how it works, and how you may be able to use it for
yourself. Or, if you want, simply grab the
contents and stick them in your own cookbook that manages Chef package
installation and move along. :–)
All the code used in this post is available in the Opscode Cookbook’s organization “apt” repository.