Category: Code

Chef Development with Shef

I thought I’d do a post how to use Shef, the interactive chef console, for iterative cookbook development for those times when you just want to experiment without uploading anything to the server as it’s a workflow I use heavily and have found really useful.

Getting Started

To get yourself set up for developing with shef, you’ll need to perform the following steps:

  • Make sure you have a copy of your chef repository on a machine similar to that on which your cookbooks normally run
  • Make sure you’re on the latest version of the chef gem (a bunch of shef bugs were fixed post-0.10.4)
  • Create a file called solo.rb somewhere, and paste the following into it (changing cookbook_path to wherever you have your chef repo):
file_cache_path "/var/chef-solo"
cookbook_path "/home/myuser/chef/cookbooks"
  • Create a file called anythingyoulike.json, containing the following (changing mycookbook as appropriate):
{ "run_list": [ "recipe[mycookbook::default]" ] }
  • Please note, this run_list should contain all the cookbooks you want to be available to you while using Shef. It doesn’t have to be the entire cookbooks directory, but make sure that if your cookbook has any dependancies, you include them here.
  • Run the following command (changing file paths as applicable):
sudo shef --solo --config ~/solo.rb -j ~/anythingyoulike.json
  • You’re now in the shef console, and should see output like the following:
loading configuration: /etc/chef/solo.rb

Session type: solo

Loading..[Tue, 07 Feb 2012 10:49:36 +0000] INFO: Run List is []

[Tue, 07 Feb 2012 10:49:36 +0000] INFO: Run List expands to []

done.

[Tue, 07 Feb 2012 10:49:36 +0000] INFO: Setting the run_list to ["recipe[mycookbook]"] from JSON

This is shef, the Chef shell.

Chef Version: 0.10.8

http://www.opscode.com/chef

http://wiki.opscode.com/display/chef/Home

run `help' for help, `exit' or ^D to quit.

Ohai2u me@mydomain.com!

chef >

You’re now using the shef console!

You can see that the run_list we specified in the JSON file has been picked up by shef. In the context of shef, this run_list is the list of cookbooks available to it, none of then will actually be run until we tell it to.

Performing a Chef Run

Now, to actually run our cookbook, we want to enter the recipe context. This is done as follows:

chef >
chef > recipe
chef:recipe >

Next, we need to load our cookbook into the current context. Please note, you can only do this with cookbooks that are shown in the run_list. Otherwise shef won’t know where to find them.

We do that with the include_recipe command:

chef:recipe>
chef:recipe > include_recipe "mycookbook::default"
=> [#<Chef::Recipe:0x00000004b5bdb0 @cookbook_name=:mycookbook, @recipe_name="default", @run_context=#<Chef::RunContext:0x00000004c1a300

<snip>

chef:recipe>

The above command will give you a huge amount of output, as it’s basically loading all of the resources from the recipe we gave it.

Next, to perform a chef run, use the following command:

chef:recipe>
chef:recipe > run_chef

At this point, you’ll see the mycookbook::default recipe run, producing the same output as you’d expect to see during a normal chef run, with the same sort of errors too.

Advanced Debugging

Trace Logging

If the standard output of a chef run doesn’t give you enough information, you can also turn on more verbose logging by using irb’s trace facility. This is enabled by running the following command:

chef > tracing on

Breakpoints

One of the most awesome features of Shef is the ability to add breakpoints to your recipes. This allows you to pause chef runs, and step forwards and backwards through the run between breakpoints.

The chef wiki goes into a lot of detail on breakpoints here, so I won’t repeat all of what it says, just give an outline.

To add a breakpoint to your recipe, simply add the following:

breakpoint "foo"

Breakpoints will be ignored during the course of a normal chef run (ie using chef-client), so don’t worry if you forget to remove one from your code. If you’re running using Shef, however, when the run hits the breakpoint, it will be paused.

You can now check the state of the system, make sure the recipe has done what you expected it to so far, and then assuming you’re happy to continue, run the following command:

chef:recipe > chef_run.resume

The opscode wiki page I linked above goes into more detail on actions like rewinding the chef run, and stepping through the run pausing at the next breakpoint, so if you’re still reading this post after all that, I’d recommend you have a look.

Produce a members report for all your Mailman lists

I recently had cause to produce a report on the membership of all our Mailman mailing lists, so rather than doing it manually I knocked together the following handy bash script…change mailman location and output file as desired 🙂


OUTPUTFILE="/tmp/mailman_report"
CURRMONTH=`date +%m-%Y`
LISTS=`/usr/local/mailman/bin/list_lists | awk '{print $1}' | grep -v [!0-9]`
rm ${OUTPUTFILE}
echo "Mailman Report for ${CURRMONTH}" > ${OUTPUTFILE}
echo >> ${OUTPUTFILE}
for x in ${LISTS}
do
echo "Members of List ${x}:" >> ${OUTPUTFILE}
LIST_MEMBERS=`/usr/local/mailman/bin/list_members ${x}`
for mems in ${LIST_MEMBERS}
do
echo ${mems} >> ${OUTPUTFILE}
done
echo >> ${OUTPUTFILE}
done
/bin/mail -s "Mailman_Report_for_${CURRMONTH}" foo@foo.com -c blah@blah.com < ${OUTPUTFILE}

Creating DMG Files Without MacOS X

I’ve put together a script for creating DMG files without using OS X…it requires Linux, I’ve tested it on Kubuntu 7.10 but it should work on anything recent. The process will also be Wiki’d, but in the meantime, instructions are below for the curious!

Run the following commands:

# This gets and builds a patched version of Apple's diskdev_cmds package which will work on Linux
wget http://www.mythic-beasts.com/resources/appletv/mb_boot_tv/diskdev_cmds-332.14.tar.gz
wget http://www.ecl.udel.edu/~mcgee/diskdev_cmds/diskdev_cmds-332.14.patch.bz2
tar xzf diskdev_cmds-332.14.tar.gz
bunzip2 -c diskdev_cmds-332.14.patch.bz2 | patch -p0
cd diskdev_cmds-332.14
make -f Makefile.lnx

# Create symlinks to the mkfs and fsck commands for HFS+
sudo cp newfs_hfs.tproj/newfs_hfs /sbin/mkfs.hfsplus
sudo cp fsck_hfs.tproj/fsck_hfs /sbin/fsck.hfsplus

# Get and enable the hfsplus kernel module
sudo apt-get install hfsplus
sudo modprobe hfsplus

Now that's done, you can use the following handy bash script (must be run as root) I've written to create a DMG file which contains the contents of a directory you specify on the command line.

#!/bin/bash

# DMG Creation Script
# Usage: makedmg <imagename> <imagetitle> <imagesize (MB)> <contentdir>
#
# imagename: The output file name of the image, ie foo.dmg
# imagetitle: The title of the DMG File as displayed in OS X
# imagesize: The size of the DMG you're creating in MB (Blame Linux for the fixed size limitation!!)
# contentdir: The directory containing the content you want the DMG file to contain
#
# Example: makedmg foo.dmg "Script Test" 50 /home/jon/work/scripts/content
#
# Author: Jon Cowie
# Creation Date: 02/04/2008

if [ ! $# == 4 ]; then
	echo "Usage: makedmg <imagename> <imagetitle> <imagesize (MB)> <contentdir>"
else
	OUTPUT=$1
	TITLE=$2
	FILESIZE=$3
	CONTENTDIR=$4
	USER=`whoami`
	TMPDIR="/tmp/dmgdir"

	if [ ${USER} != "root" ]; then
		echo "makedmg must be run as root!"
	else
		echo "Creating DMG File..."
		dd if=/dev/zero of=${OUTPUT} bs=1M count=$FILESIZE
		mkfs.hfsplus -v "${TITLE}" ${OUTPUT}

		echo "Mounting DMG File..."
		mkdir -p ${TMPDIR}
		mount -t hfsplus -o loop ${OUTPUT} ${TMPDIR} 

		echo "Copying content to DMG File..."
		cp -R ${CONTENTDIR}/* ${TMPDIR}

		echo "Unmounting DMG File..."
		umount ${TMPDIR}
		rm -rf ${TMPDIR}

		echo "All Done!"
	fi
fi

Hope it’s useful!