Matt on Rails

For a while, I’ve been keeping bundled gems local to each project. (It’s pretty easy to do: bundle --path .bundle/gems the first time you bundle a project. I like putting it in .bundle because it’s already gitignored.) This results in a lot of duplicate gems across my system, but it has the advantage of keeping each project very self-contained, and I avoid undesirable side-effects (like rake getting upgraded) just because I bundle some code from github.

This leads to a pretty sparse set of system gems. But it also makes it a little challenging to bootstrap a new rails project, since there isn’t a current rails executable! How to avoid installing the latest rails to start a new project?

It’s pretty easy, when you think about it, but it is a little less straightforward.

mkdir -p ~/my/new/awesome_project
cd ~/my/new/awesome_project
echo source :rubygems            > Gemfile
echo gem 'rails', '~>3.1.0.rc4' >> Gemfile
bundle --path .bundle
bundle exec rails .

It’ll ask if you want to overwrite Gemfile, which is, of course, fine, because it’s just going to add more gems.

Don’t do it.

And not just because you should be using rspec.

rake db:schema:load test does this:

** Invoke db:schema:load (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:schema:load
** Invoke test (first_time)
** Execute test
** Invoke test:units (first_time)
** Invoke test:prepare (first_time)
** Invoke db:test:prepare (first_time)
** Invoke db:abort_if_pending_migrations (first_time)
** Invoke environment 
** Execute db:abort_if_pending_migrations
** Execute db:test:prepare
** Invoke db:test:load (first_time)
** Invoke db:test:purge (first_time)
** Invoke environment 
** Execute db:test:purge
** Execute db:test:load
** Invoke db:schema:load 
** Execute test:prepare
** Execute test:units
** Invoke test:functionals (first_time)
** Invoke test:prepare 
** Execute test:functionals

If you look closely, you’ll see that db:schema:load is in there twice. The first time, it’s loading into your dev environment. The second time, it’s loading into your test environment. Or, tries to: rake notices that db:schema:load was already run, so it skips it. You know, unnecessary extra work.

I am most likely to rake db:schema:load test (or spec) on a CI server, sometimes with a db:migrate mixed in. One-liner ci scripts are nice. But keep db:schema:load and test separated.

If you have db/schema.rb in source control like you should, and you keep it up to date, you still need to db:schema:load to avoid the ‘abort if pending migrations’ thing: rake db:schema:load ; rake test

If your db/schema.rb might be out of date, rake db:schema:load db:migrate ; rake test

If you’re a glutton for punishment: rake db:migrate test

So, what if you want to run jenkins locally on a mac? What if you want it to build from a private github repository, and you want it to use its own private key? And you’re on a mac?

Get Jenkins

First, you need to get jenkins. On my homebrew-enabled mac, I ran brew install jenkins. Without homebrew, you can just download the latest war file. (You will need java 6 to run jenkins.) Homebrew provides a plist for launching jenkins when you log in. If that’s adequate, then you can stop reading.

Create a service account

Next you need to create a user to run jenkins. This is the most challenging part of the process. (I used a script from a pastebin as reference.) Here’s what I did:

sudo mkdir /var/jenkins
sudo /usr/sbin/dseditgroup -o create -r 'Jenkins CI Group' -i 600 _jenkins
sudo dscl . -append /Groups/_jenkins passwd "*"
sudo dscl . -create /Users/_jenkins
sudo dscl . -append /Users/_jenkins RecordName jenkins
sudo dscl . -append /Users/_jenkins RealName "Jenkins CI Server"
sudo dscl . -append /Users/_jenkins uid 600
sudo dscl . -append /Users/_jenkins gid 600
sudo dscl . -append /Users/_jenkins shell /usr/bin/false
sudo dscl . -append /Users/_jenkins home /var/jenkins
sudo dscl . -append /Users/_jenkins passwd "*"
sudo dscl . -append /Groups/_jenkins GroupMembership _jenkins
sudo chown -R jenkins /var/jenkins

Before I ran through that, I did check that the ids were available. You can search for users with dscl . -search /Users uid 600 and groups with dscl . -search /Groups gid 600.

Also, I don’t think the uid and gid need to be the same, but most of the builtin service accounts (e.g. jabber) are, so I just went along with it.

Create the daemon

Mac OS uses launchd to control daemons and agents. It’s pretty easy to create a launch daemon. Create the file /Library/LaunchDaemons/org.jenkins-ci.plist with the following content, based on the plist from the homebrew jenkins formula. You may need to update the version number in the ProgramArguments.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>Jenkins</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/bin/java</string>
    <string>-jar</string>
    <string>/usr/local/Cellar/jenkins/1.414/lib/jenkins.war</string>
  </array>
  <key>OnDemand</key>
  <false/>
  <key>RunAtLoad</key>
  <true/>
  <key>UserName</key>
  <string>jenkins</string>
</dict>
</plist>

I think I had to load the daemon (sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist). Rebooting should work, too.

Create an ssh key

Like I said, I wanted jenkins to have its own ssh identity. This is fairly easy: sudo -u jenkins ssh-keygen

The new key is in /var/jenkins/.ssh/id_rsa.pub and can be copied to github, or wherever you have your source code.

Set up a build

Now you need to configure jenkins. Open http://localhost:8080/, click “Manage Jenkins”, “Manage Plugins”, then “Available”. I installed the git and github plugins. The git plugin gives you basic git functionality. The github plugin gives you links from the build info pages to github commit pages. After the install is complete, click the “Schedule a restart” button.

After jenkins restarts, configure git. From the “Manage Jenkins” page, “Configure System” and make sure the path for git is right. (It wasn’t for me: /usr/bin/git is the default, but homebrew put it in /usr/local/bin/git.)

Now, you can create your project. The rest of this post is a description of how I set up the build for my rails 3.0 project.

Click “New Job” in the menu on the left. Choose the “free-style” option. On the configuration page, set the github url prefix (e.g. https://github.com/user/project). Set the source code as git, with the repo url (e.g. git@github.com:user/project.git; if your url starts with git:// or http://, then you probably don’t need to do the ssh key stuff earlier). I set branches to build to “master”, but leaving it blank is sometimes useful. I chose to Poll SCM, with a schedule of “* * * * *”, though I didn’t set that up until after I had the build mechanics set up right.

The only build step is to execute a shell. Here’s what I’m using. There are a few things to note here. I’m using a system-installed rvm, so all the ruby commands are run through it. Jenkins can collect JUnit test results and produce some trend graphs, so I use the ci_reporter gem’s ci:setup:testunit to format the results JUnit-style.

/usr/local/rvm/bin/rvm 1.8.7 exec bundle --path .bundle/gems

cat <config/database.yml
development:
  adapter: sqlite3
  database: db/development.sqlite3
test:
  adapter: sqlite3
  database: db/test.sqlite3
END_CONFIG

/usr/local/rvm/bin/rvm 1.8.7 exec bundle exec rake db:bootstrap db:migrate ci:setup:testunit test --trace