Minimalist Scripted Configuration
=================================

<ericshane@eradman.com>

















NYCBUG
June 2, 2021

Overview
========

• Step through the process of designing a framework for automating
  system configuration

• Understand the motivation for new minimalist configuration tools, one
  of which may work you

Project First Commit          Author             Date
------- --------------------- ------------------ ----------------------
judo    b23ef07 (tag: 0.1)    Kamil Cholewiński  Sun Oct 23 13:20 2016
rset    0b83212 (tag: 0.1)    Eric Radman        Wed Aug  8 00:58 2018
drist   c9735fd (tag: v1.00)  Solène Rapenne     Thu Nov 29 08:20 2018

Motivations
===========

1. Simple enough to be used at small scale
   (1 or more target hosts)

2. Complexity of the solution scales with the problem space
   (data + scripts)

3. Fast enough to test interactively
   (feedback in 1-5 seconds)

Agent-Server Model
==================

chef/puppet/salt/bcfg2:

✗ Development environment is not authoritative
✗ Need a separate staging environment
✗ Heavy client-side dependencies
✓ SSH access not required
✓ More advanced reporting capabilities

What are the advantages?

> Central status of systems (reporting)
> One place of authority (where did the change come from)
> Auditing
> Non-free offerings
> Windows

Gather-Fact Model
=================

ansible/salt-ssh:

1. Ship python modules, run remote probes
2. Generate execution plan
3. Package code modules and data
4. Pipe blob to remote python interpreter

✓ Dry run
✗ Difficult to debug
✗ Control flow is in YAML
✗ Long round-trip time

> Anyone can run your playbooks
> Someone else's fault if it doesn't work
> Can do small things
> Some degree of portability
> Even more portability if you're clever
> One-off tasks in a consistent way

Two-Stage Model
===============

pyinfra:

1. Execute remote scripts return information
2. Generate shell scripts
3. Execute scripts on remote host

✓ No remote dependencies
✓ Easy to debug
✓ Reasonably easy to extend
✓ Fast
✓ Multiple transports, not just SSH
✗ Python only

>

Remote Execution Model
======================

rset/drist/judo:

1. Copy files associated with host
2. Run scripts on remote host

------- --------------
rset    tar|ssh, http
drist   rsync
judo    scp

✓ Progressive status
✓ Any mix of scripting languages
✓ No need to collect facts
✓ Ship utilities, data, and configuration
✓ Fast

Oversimplification
==================

  ssh 10.5.5.1 < configure-wordpress.sh

What can this script do?

✓ Install packages
✓ Fetch/extract source
✓ Create directories, set permissions
✗ Install configuration files
✓ Start/enable services
✗ Apply part of a config

> No staging area
> "curl" in everything you want

Modularization Attempt
======================

  host=10.5.5.1
  ssh $host < www/letsencrypt.sh
  ssh $host < www/httpd.sh
  ssh $host < wordpress/mysql.sh
  ssh $host < wordpress/config.sh

Characteristics:

✓ Any scripting language
✗ No convention for running part of a configuration
✗ Performance degrades with each "module"

How can we solve the above limitations?

> Heredoc
> Heredoc uuencode!
> Use SSH control master to re-use connection

Installing Files
================

  cd /tmp/staging
  cp smtpd.conf /etc/mail/smtpd.conf
  rcctl restart smtpd

What can this script do?

✗ No convention for transferring files to a staging directory
✗ No diff
✗ No exit code indicating if the file changed

What other features would be useful for installing files?

>

Standard Utilities
==================

  SYNOPSIS
      rinstall [-o owner:group] [-m mode] source target

• Print diff or notice that a new file was created
• Optionally set owner and mode
• Fetch large files on-demand using HTTP
• Exit `0` only if file was installed or changed!

What other "standard" utilities might we need?

> rset also includes rsub(1) for modifying existing files
> manage users?
> portable means of installing packages
> Diff against baseline file

Host Routes
===========

rset - list of scripts and directories

  # routes.pln

  vm.eradman.com: etc/ wordpress/
    common.pln
    wordpress.pln

drist - script and files match hostname

  files
  files-vm.eradman.com/
  script
  script-vm.eradman.com/

Progressive Label Notation
==========================

Blocks of configuration can be selected individually
Labels names beginning with [0-9a-z] are excluded by default:

  root_tasks:
  →   crontab - <<-EOF
  →       ~ 1 * * * /usr/local/bin/renewcert
  →   EOF

Parameters apply to subsequent labels

  interpreter=/bin/sh -x


> Tab indentation?

SSH Tactics
===========

1. Always use `ssh-agent(1)`

2. Call home using wg(4)

3. Employ a jumphost if need be

  Host 10.0.0.20
    ProxyJump 192.168.0.2
    ConnectTimeout=5

4. Bootstrap using site69.tgz

> SSH is even more awesome than this
> Use signed keys

Conclusion
==========

_Source Control_ is a means of remembering and communicating the state
of a system.

_Interactive Testing_ depends on a tight feedback loop.

_Orchestration_ is a flamboyant term for scripting with configuration
data, scripts, and utilities already staged.










---
http://scriptedconfiguration.org/