ZFS Quickstart

ZFS has always provided a wide range of useful features, and over time it's performance has improved. In 2023 OpenZFS 2.2 dramatically improved performance when using NVMe drives.

With some tuning, ZFS can provide good a platform for a database while providing better than 2-to-1 compression.

Rocky/Alma Linux Install

dnf install https://zfsonlinux.org/epel/zfs-release-2-2.el9.noarch.rpm
dnf config-manager --disable zfs
dnf config-manager --enable zfs-kmod
dnf install zfs
echo zfs > /etc/modules-load.d/zfs.conf

See also OpenZFS RHEL install.

Limit ARC memory use if the system is running a memory intensive application such as a database

echo 1073741824 > /sys/module/zfs/parameters/zfs_arc_max

FreeBSD

# /etc/rc.conf
zfs_enable="YES"

Limit ARC memory use

# /etc/sysctl.conf
vfs.zfs.arc.max=1073741824

ZFS does not require a partition table, but initializing a disk with GUID partition map will avoid spurious warnings and make it clear what kind of file system is on a device

gpart destroy -F nvd1          # Delete partition data
gpart create -s gpt nvd1       # New GUID partition table
gpart add -t freebsd-zfs nvd1  # Create and label partition

Create zpool and new volume on first partition

zpool create -O compression=lz4 zpool2 /dev/nvd1p1
zfs create -o mountpoint=/ci zpool2/ci

PostgreSQL Configuration

Since ZFS provides consistency guarantees that other file systems do not, namely we can disable full page writes since we do not need to guard against torn pages

ALTER SYSTEM SET full_page_writes=off;
ALTER SYSTEM SET wal_init_zero=off;
ALTER SYSTEM SET wal_recycle=off;
SELECT pg_reload_conf();

Automated Snapshot Managment

To automate snapshot retention make a new periodic snapshot and prune anything that is more than N days old

#!/bin/sh -e

today=$(date +"%Y-%m-%d")

for fs in zpool2/ci; do
   zfs snapshot $fs@$today

   for snap in $(zfs list -t snapshot -H -o name $fs | sort -r | tail +30); do
      zfs destroy $snap
   done
done

Run daily

15  20  *  *  * /usr/local/bin/zfs-snap.sh

NFS Export

While any local mount can be added to /etc/exports ZFS sharing allows mount points to be automatically by setting the shrenfs property on each volume

$ zfs sharenfs='-ro,-network 192.168.2.0/24' zpool1
$ zfs get sharenfs
NAME       PROPERTY  VALUE                        SOURCE
zpool1     sharenfs  -ro,-network 192.168.2.0/24  local
zpool1/ci  sharenfs  -ro,-network 192.168.2.0/24  inherited from zpool1

Virtual Machines

When running KVM or Bhyave local storage can be optimized by creating a device that is used directly the virtual machine. This is referred to as a zvol

zfs create -sV 100G -o volmode=dev zpool2/vm/mykube2