8-bit internet rodent

Gravatar of porcupo


| Comments


I’ve recently been putting a more thought into test-driven infrastructure. http://serverspec.org/
has been on my radar for a while. Based on RSpec in the ruby world, serverspec provides a DSL for
spec testing servers.

The DSL is pretty minimal, but conveniently extendable.

Why Test Driven Infrastructure?

Test driven infrastructure allows you to specify outcomes, rather than the procedures that are supposed to
produce those outcomes.

In the below example for nginx, we’ll have to consider what the final outcome should be.

Example: nginx

In ones typical sysadmin world, you might get a request along these lines:

install web server (now!!)

From here, we proceed to install nginx, enable service, and start daemon.[1]
We’re pretending chef, puppet, et al. do not exist, and we’re running this by hand or via some sweet bash scripts over ssh with a for loop.

>:3 sudo apt-get install nginx
>:3 sudo update-rc.d nginx enable
>:3 sudo service nginx start

Note that we’re assuming a lot from the requester here.

This is all well and good, and most likely will produce the desired (or at least assumed) outcome. But
let’s say this request comes up a lot. You have a script that installs everything without interaction.
Works great! So far every request from the dev team for an nginx install went as expected.

If you’ve ever closed a ticket at this point, you know there’s a 33% chance this will all end in tears.

  • The requester meant nginx hosted port port 80 with various virtual hosts when he said “nginx”
  • The requester is not in engineering, or is working on a side project, etc., that does not conform to
    any recognizable standard you have
  • The requester assumed they would have sudo access and could configure nginx themselves
  • and so on and so on

Another TDI benefit? get the answers to these questions from the requester, removing even more ambiguity
from the request.

  • What port should nginx run on?
  • what should be in sites-enabled?
  • Should this service start on boot?

…and whatever other outcomes they might like.

nginx spec

I’ll start out with a simple example, and then dig into the details of setting everything up.

# spec/httpd_spec.rb
require 'spec_helper'

describe package('nginx') do
  it { should be_installed }

describe service('nginx') do
  it { should be_enabled  }
  it { should be_running  }

describe port(80) do
  it { should be_listening }

describe file('/etc/nginx/sites-enabled/porcupo') do
  it { should be_file }
  its(:content) { should match /server_name porcupo.net \*\.porcupo.net localhost/ }

Pretty self explanatory! Basically you assert the following:

  • nginx is installed
  • nginx service is running
  • nginx service is enabled on boot
  • something should be listening on port 80 (we’ll assume that’s nginx here)
  • the /etc/nginx/sites-enabled/porcupo file has a line matching the regexp server_name porcupo.net \*\.porcupo.net localhost
# run spec test
>:3 rake spec:kaiju.porcupo.net
Package "nginx"
should be installed

Service "nginx"
should be enabled
should be running

Port "80"
should be listening

File "/etc/nginx/sites-enabled/porcupo"
should be file
should match /server_name porcupo.net \*\.porcupo.net localhost/

Finished in 1.28 seconds (files took 6.03 seconds to load)
6 examples, 0 failures

The nitty- gritty

There are some dependencies required for all this:

>:3 tree .
├── Gemfile
├── Gemfile.lock
├── Rakefile
└── spec
    ├── kaiju.porcupo.net
    │   ├── base_spec.rb
    │   └── httpd_spec.rb
    └── spec_helper.rb

2 directories, 6 files

spec_helper.rb contains the test-independent glue for making all of this happen. Here’s what I’m using
in this instance.

# spec_helper.rb
require 'serverspec'
require 'net/ssh'
require 'highline/import'

options = Net::SSH::Config.for(host)
options[:user] ||= Etc.getlogin

set :host,        options[:host_name] || host
set :ssh_options, options
set :backend, :ssh
set :disable_sudo, true
set :env, :LANG => 'en_US.UTF-8'

note on sudo

sudo is enabled by default, and you can have serverspec prompt for or read a password. This
can be spotty, as you won’t quite know exactly when it’s using sudo along the way.
Preferably, you would enable this per feature:

describe command('whoami'), :sudo => true do
  it { should return_stdout 'root' }

Your Rakefile automates the processes

# Rakefile
require 'rake'
require 'rspec/core/rake_task'

task :spec    => 'spec:all'
task :default => :spec

namespace :spec do
  targets = []
  Dir.glob('./spec/*').each do |dir|
    next unless File.directory?(dir)
    targets << File.basename(dir)

  task :all     => targets
  task :default => :all

  targets.each do |target|
    desc "Run serverspec tests to #{target}"
    RSpec::Core::RakeTask.new(target.to_sym) do |t|
      ENV['TARGET_HOST'] = target
      t.pattern = "spec/#{target}/*_spec.rb"

Here’s the Gemfile:

~~~ ruby Gemfile
source “https://rubygems.org”

gem “serverspec”
gem “rake”
gem “rspec”
gem “highline”

Example: base system

A simple example for basic Ubuntu setup

The hot thing to do with rspec now is to use expect over should. I’ll try that below.

# base_spec.rb
require 'spec_helper'

describe "Hostname" do
  this_host = host("porcupo.net")

  it "should resolve to" do
    expect(this_host.ipaddress).to eq ''

  it "should be reachable on ports 22, 80" do
    expect(this_host).to be_reachable.with(:port => 22)
    expect(this_host).to be_reachable.with(:port => 80)

  it "should NOT be reachable on ports 25, 1234" do
    expect(this_host).not_to be_reachable.with(:port => 25)
    expect(this_host).not_to be_reachable.with(:port => 1234)

describe "OS" do
  it "should be Ubuntu 14.04 x86_64" do
    expect(os[:family]).to match /^ubuntu$/
    expect(os[:release]).to match /^14.04$/
    expect(os[:arch]).to match /^x86_64$/

This is only the beginning! I’ll be sure to add anything interesting.

Additional ideas

  • Use chef, puppet, fabric, saltstack, etc. to push files and configuring, using serverspec to test.
  • Use md5 sum of config files when checking
  • Have failure tests (apache should not be installed, etc.)
  • functional tests (hit API URL and expect a correct answer)
  • a more complex, role-based database of hosts to test.
  • more packages and package configs

Code also available on github

Internet Explorer in OSX

| Comments

If you’re unfortunate enough to work somewhere that say, uses some ancient
SAP application for your all corporate needs, you’ll probably run
into the need to fire up IE 6 to see your paystub.

Maybe your world is more sane; but you still need to test things in IE

ievms totally addresses this. Basically, it sets up a small vm with
whatever desired IE version is desired

From the above page, we can see installation is simple:

curl -s https://raw.github.com/xdissent/ievms/master/ievms.sh | env IEVMS_VERSIONS="7" bash

The smart thing to do here is to whip open Automator and make this an .app

  1. Open Automator
  2. File -> New
  3. Drag Run Shell Script from the left panel to the right.
  4. Paste this:

/usr/bin/VirtualBox --startvm "IE7 - WinXP"
  1. File -> Save (ex: /Applications/Internet Explorer.app)

vagrant and pump.io

| Comments

Starting what will be a little series on setting up pump.io for corporate use.

I throw in some basic vagrant too.

Part 1 will be setting up a reliable testing environment for pump.io. We’ll be using vagrant and some basic shell scripting to keep things easy.

My Vagrant configuration for pump.io is in my github vagrant repo for reference.

I’ll assume you have basic working knowledge of vagrant. If not, vagrantup.com has some solid beginner docs to get you up and running.

pump.io docs will probably not be winning awards any time soon. So I’m hoping I can help at least make this easier for them.

I used the following to whip up a vagrant box:

vagrant init test1 http://files.vagrantup.com/precise64.box

Once you’ve created a vagrant instance (with vagrant init), edit your Vagrantfile in the current directory. Here’s mine:

Vagrant.configure("2") do |config|
  config.vm.box = "test1"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"
  config.vm.network :forwarded_port, guest: 31337, host: 31337
  config.vm.network :public_network
  config.vm.provision :shell, :path => "bootstrap.sh"

The :forwarded_port variable is needed to provide some kind of external access to pump.io. This line will provide http://localhost:31337/ on your host machine.

I prefer using the :public_network setting, which will use a bridged adapter, giving the vagrant box it’s own IP on your network. This is preferable unless you don’t get along with your network admins.

The final line here points vagrant to a provisioning shell script. I’m using this for simplicity’s sake, but in an ideal world, we’d be setting this up in puppet or chef-solo.

ssh back home with icloud

| Comments

BTMM (back to my mac) works splendidly for a remote desktop solution, but let’s say this doesn’t quite work for you for whatever reason:

  • You don’t have adequate bandwidth
  • You have a 1440x900 screen at work and a 2560x1440 display at home
  • It’s a bit much
  • You’d take the commandline any day

Providing you enabled sshd on the remote end (System Preferences -> Sharing -> Remote Login) you can ssh over BTMM too.

OS X’s Terminal.app has knowledge of this built in, simply go to Shell -> New Remote Connection and choose form the list.

But if you’re not using Terminal, it’s easy enough to find:[^1]
~~~ bash
djo@kaiju ~/tmp >:3 echo show Setup:/Network/BackToMyMac | scutil

{ 501 : 000000000.members.btmm.icloud.com. } djo@kaiju ~/tmp >:3 ~~~ But don’t ever type that again! Use this information to add some reasonably meaty stuff to your ssh_config. In this example, my box at home is called ‘gravy’: ~~~ Host shark HostName gravy.000000000.members.btmm.icloud.com ~~~ [^1]: Graciously picked up from https://gist.github.com/skyisle/1856804