Eric Radman : a Journal

Local Continuous Integration

An entire industry has arisen to provide a platform for launching builds or tests a variety of architectures. One problem with operations that are triggered by a commit is that this isn't a development workflow. Unless tests can be initiated from the development environment, it is of no use while you have your editor open.

We are used to putting up with long and error-prone feedback cycle in software development, but I don't think we should. Can you imagine a spell-check that only operated once you had submitted a paper for review?

Network File Systems?

One possible solution is to run a network file system so that you can keep the same view of a repository across various platforms. This idea makes files immediately available but has many disadvantages

  1. Tedious to configure, an IP tunnel required for instances that are not on a local network
  2. Concurrent builds are not possible
  3. Edits on one host always interfere with builds on other hosts
  4. Full clean/rebuild always required

Propagate Edits

Another solution is to replicate a repository containing local edits with rsync. This has none of the downsides of setting up a shared file system 

#!/bin/sh -e
pwd

# Figure out our path relative to our home directory
home_repo="${PWD#/}"

# Generate a list of files to exclude base on .gitignore
exclude_list=/tmp/reposync-exclude.$USER
git ls-files --exclude-standard -oi --directory > $exclude_list

# Synchronize the current directory with a set of remote hosts
# Don't set file times since this confuses make(1)
while read host args; do
    echo "$host"

    # Sync regular files, not including files ignored by git
    # Don't set file times since this confuses make(1)
    rsync . $host:$home_repo/                 \
        --delete                              \
        --exclude-from=$exclude_list          \
        --links                               \
        --perms                               \
        --recursive                           \
        $args || true
done < ~/.reposync/hosts

In this implementation the host file may contain optional arguments to add to the rsync command line 

$ cat /home/eradman/.reposync/hosts
192.168.0.12
192.168.0.13 --rsync-path=/opt/homebrew/bin/rsync
192.168.0.5
192.168.0.6

A Word about Test Coverage

A few projects seek to achieve 100% test coverage—that is, automated tests that exersise or in some way verify every line of source code. This is a noble goal, but this terminology probably obscures what we mean since many behaviors can only be detected or can only be classified as incorrect in real-world use.

One example of this is trying to use sysctl(3) to detect the amount of RAM system has . This is tricky because HW_PHYSMEM will return a number, but on some platforms is the correct value and on other platforms it is only correct value if there is no 2GB or less of RAM! Some changes need to be confirmed manually.

Another variation is the ability to run automated tests in an environment that is subtly altered. This is one way to quickly verify that a test that is expected to fail responds with the correct assertion.

Minimal Frameworks

There is value in minimalist frameworks since they provides a way to share suggestions (in the form of bug reports) and allows for a way to standardize processes. At other times it may be better to make the utility in question so small that it only serves as the basis for assembling your own.