Eric Radman : a Journal

Automated FreeBSD Install

For FreeBSD, the ability to PXE boot BIOS and UEFI systems is not new, but documentation frequently assumes an NFS mounted root file system. In my view NFS is unworkable since file access is not logged, and a hung mount is difficult to diagnose.

By using TFTP to pivot to a minroot, then switching to HTTP the entire process can be audited and tailored for individual hosts. This methodology aims to borrow techniques used by an automated install for OpenBSD and Red Hat Linux.

DHCP Services

To PXE-boot FreeBSD there are three components we will need to three components 

# /etc/dhcpd.conf

subnet 192.168.0.0 netmask 255.255.255.0 {
    option routers 192.168.0.7;
    option domain-name-servers 192.168.0.3;
    range 192.168.0.64 192.168.0.254;
    next-server 192.168.0.2;

    host BL-6D-N150 {
        hardware ethernet 78:55:36:00:62:7d;
        fixed-address 192.168.0.8;
        option root-path "tftp://192.168.0.2/freebsd15";
        filename "loader15.efi";
        option host-name "bl-6d-n150";
    }
}

HTTP Services

The install media can be copied, or mount the ISO as a read-only file system 

vnconfig vnd1 /var/www/htdocs/pub/FreeBSD-15.0-RELEASE-amd64-dvd1.iso
mount_cd9660 /dev/vnd1c /var/www/htdocs/pub/freebsd15

Distribution files can now be pulled from this http://${next_server}/pub/freebsd15/usr/freebsd-dist/.

Copy loader.efi to /tftpboot/.

Memory File System

Sadly, FreeBSD does not ship a minroot, but we can fetch an image from the mfsBSD project. Mount this ISO in a location where the loader and access boot config and minroot 

vnconfig vnd0 /var/www/htdocs/pub/mfsbsd-15.0-amd64.iso
mount_cd9660 /dev/vnd0c /tftpboot/freebsd15

This provides everything required to boot 

.
|-- boot
|   |-- defaults
|   |   `-- loader.conf
|   |-- device.hints
|   |-- kernel
|   |   |-- acpi_*.ko
|   |   |-- ahci.ko
|   |   |-- kernel.gz
|   |   `-- linker.hints
|   |-- loader
|   |-- loader.conf
|   `-- lua
|       |-- cli.lua
|       |-- color.lua
|       |-- config.lua
|       |-- core.lua
|       |-- drawer.lua
|       |-- gfx-*.lua
|       |-- hook.lua
|       |-- loader.lua
|       |-- menu.lua
|       |-- password.lua
|       `-- screen.lua
|-- boot.config
`-- mfsroot.gz

boot/loader.conf provides the configuration for switching to a memory file system 

mfs_load="YES"
mfs_type="mfs_root"
mfs_name="/mfsroot"
ahci_load="YES"
vfs.root.mountfrom="ufs:/dev/md0"

For a standard build mfsroot.gz is about 70MB.

Building a Custom Miniroot

The mfsBSD root image can be modifed to kick off a custom install script.

First, fetch a copy of the base sets 

mirror="http://192.168.0.2/dist/freebsd/amd64/15.0-STABLE"
mkdir $HOME/15.0
cd $HOME/15.0
for f in MANIFEST base.txz kernel.txz; do
    fetch $mirror/$f
done

Get a copy of the latest mfsbsd sources 

fetch https://github.com/mmatuska/mfsbsd/archive/refs/heads/master.tar.gz
tar -zxf master.tar.gz
cd mfsbsd-master

Edit conf/rc.local.sample 

# sample rc.local
#
export PATH=/usr/local/bin:$PATH

# Find http server and host name from dhclient leases
for op in `awk '/^(  |  option )(next-server|host-name|domain-name) / { gsub("-", "_", $(NF-1)); print $(NF-1)"="$NF }' /var/db/dhclient.leases.*`
do
    eval "$op"
done

# Fetch package manifest to avoid confirmation
mkdir /usr/freebsd-dist
cd /usr/freebsd-dist
fetch http://${next_server}/dist/freebsd/`uname -m`/`uname -r`/MANIFEST

# Get bsdinstall(8) configuration by hostname and launch in a tmux session
fetch http://${next_server}/bsdinstall/${host_name}.cfg
echo "Starting autoinstall in tmux session"
tmux new-session -s autoinstall -d
tmux send-keys -t autoinstall:0 "bsdinstall script ${host_name}.cfg" C-m

Trim off tools that are not needed 

echo "tmux" > tools/packages.sample

Build: 

make clean-all BASE=$HOME/15.0 RELEASE=15.0
make BASE=$HOME/15.0 RELEASE=15.0
make iso BASE=$HOME/15.0 RELEASE=15.0

Install Script (UFS)

The installation bootstrap is based into the custom miniroot, but the host configuration is fetched from the installer set by the DHCP option 

PARTITIONS="nda0 { 64G freebsd-swap, auto freebsd-ufs / }"
export BSDINSTALL_DISTDIR="/usr/freebsd-dist"
export BSDINSTALL_DISTSITE="http://192.168.0.2/dist/freebsd/`uname -m`/`uname -r`"
export DISTRIBUTIONS="base.txz kernel.txz ports.txz"


#!/bin/sh
cat >> /etc/rc.conf <<CONF
ifconfig_DEFAULT=DHCP
sshd_enable=YES
ntpd_enable=YES
CONF

# set timezone
ln -s /usr/share/zoneinfo/US/Eastern /etc/localtime

# install packages
pkg install -y doas
echo "permit keepenv nopass :wheel" > /usr/local/etc/doas.conf
chmod 600 /usr/local/etc/doas.conf

# add users
pw useradd eradman -m -u 1000 -s /bin/sh -G wheel
install -d -o eradman -g eradman -m 700 /home/eradman/.ssh
echo "ssh-ed25519 AAAA...  eradman@t470s.eradman.com" > /home/eradman/.ssh/authorized_keys
chown eradman:eradman /home/eradman/.ssh/authorized_keys

# set passwords
sed -i -e 's/:passwd_format=sha512:/:passwd_format=blf:/g' /etc/login.conf
chpass -p '$2b$ ...' root
chpass -p '$2b$ ...' eradman

echo "rebooting in 10 seconds"
sleep 10
reboot

Install Script (ZFS)

For ZFS on root the setup has a more settings 

export ZFSBOOT_BOOT_TYPE="UEFI"
export ZFSBOOT_DISKS="nda0"
export ZFSBOOT_SWAP_SIZE="12g"
export ZFSBOOT_POOL_CREATE_OPTIONS=" "

export BSDINSTALL_DISTSITE="http://192.168.0.2/dist/freebsd/`uname -m`/`uname -r`"
export DISTRIBUTIONS="base.txz kernel.txz ports.txz"

export ZFSBOOT_DATASETS="
/ROOT   mountpoint=none
/ROOT/default        mountpoint=/
/tmp    mountpoint=/tmp,devices=off,exec=on,setuid=off
/usr    mountpoint=/usr,canmount=off
/usr/local  devices=off
/usr/ports  setuid=off
/usr/src
/var    mountpoint=/var,setuid=off,devices=off
/home   mountpoint=/home,setuid=off,devices=off
"