The Sidecomment Web Stack

sidecomment.io is a service that allows readers to select text and create a ticket with suggestions or corrections that the site owner can respond to. Like a commenting platform such as Hyvor Talk, JavaScript is included from an external CDN, and is authorized by a site code.

This concept developed after I noticed how many people were willing to take the time to write an email to point out typos, broken links or errors on this online journal and other project pages.

Static Content and HTTP Routing: httpd/relayd

Since I'm using OpenBSD, httpd(8) is an my go-to daemon for static content

# httpd.conf

server "cdn.sidecomment.io" {
    log style forwarded
    listen on "*" port 8000

    root "htdocs/sidecomment.io/cdn"

    # Let's Encrypt
    location "/.well-known/acme-challenge/*" {
        root "/acme"
        request strip 2
    }

    location "/*" {
        directory auto index
    }
}

relayd(8) routes HTTP requests and provides TLS termination for https://

# relayd.conf
ext_addr = 0.0.0.0
table <httpd> { 127.0.0.1 }
table <puma> { 127.0.0.1 }

# TLS proxy all home services
http protocol "httpproxy" {
    tls keypair "sidecomment.io"

    match header set "X-Client-IP" value "$REMOTE_ADDR:$REMOTE_PORT"
    match header set "X-Forwarded-For" value "$REMOTE_ADDR"
    match header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

    pass request forward to <puma>
    pass request path "/*.*" forward to <httpd>
    pass request path "/.well-known/acme-challenge/*" forward to <httpd>
    pass request header "Host" value "cdn.sidecomment.io" forward to <httpd>
}

relay "https" {
    listen on $ext_addr port 443 tls
    protocol httpproxy
    forward to <httpd> port 8000
    forward to <puma> port 9292
}

relay "http" {
    listen on $ext_addr port 80
    protocol httpproxy
    forward to <httpd> port 8000
    forward to <puma> port 9292
}

Web Stack: Ruby/Sinatra/Haml

Any language can serve HTTP requests. I considered a few different frameworks, but I settled on Ruby because it gives me access to Haml templates. Sinatra is not perfect, but it is a very readable DSL. In addition to providing separate routes for HTTP request type, custom rules can be built

# main.rb

# test to see if the 'otp' query string is set
set(:otp) {|value| condition { params[:otp].nil? == value.nil? }}

get '/account', :otp => true do
end

get '/account' do
end

post '/account' do
end

cpan/gem/pip/npm/cargo-like systems are prone to regressions during upgrades since there is no release engineering that works in tandem with the operating system. For ruby-gems there are two strategies for working around broken builds:

# select an older version
gem33 install nokogiri -v 1.14.2

# build from top of tree
gem33 install specific_install
gem33 specific_install https://github.com/puma/puma.git

Application Server

Puma was designed to be a fast and efficient, but I was primarily interested in a server the reports useful error messages if it cannot start. This web application (like most?) is not designed to be thread-safe, so Puma must be set to use one thread per worker

# config.ru
require './main'
run Sinatra::Application

# puma.rb
app_dir = __dir__
port 9292
pidfile "#{app_dir}/run/puma.pid"
state_path "#{app_dir}/run/puma.state"
directory "#{app_dir}/"
stdout_redirect "#{app_dir}/log/puma.out", "#{app_dir}/log/puma.stderr.log", true
workers 3
threads 1, 1
activate_control_app "unix://#{app_dir}/run/pumactl.sock"

It usually possible to use cron to run periodic actions, or to deploy a dedicated service to handle these tasks, but the deployment of small applications can also be simplified by starting up a sidecar process. In worker mode the Puma configuration file is Ruby and is able to fork a process

# puma.rb
require_relative 'notify'

def log(stream, label, message)
  ts = Time.now.strftime('%m-%d-%Y %H:%M:%S')
  stream.puts "#{ts} #{label} #{message}"
end

before_fork do
  fork do
    Process.setproctitle('puma: sidecomment scheduler')
    loop do
      log $stdout, 'notify_tickets', notify_tickets.to_a
      log $stdout, 'notify_replies', notify_replies.to_a
      sleep 600
    rescue StandardError => e
      log $stderr, 'error', e.full_message
      next
    end
  end
end

Defining before_fork() allows the application to do things before the application is started, in this case start a subprocess that wakes up every 5 minutes. Logging to STDOUT and STDERR is captured in the standard puma logs.

The startup rc.d script supports Puma's graceful restart

#!/bin/ksh

daemon="puma33 -C /var/www/htdocs/sidecomment.io/puma.rb -e production"
daemon_flags="--control-url tcp://127.0.0.1:9293 --control-token ********"
daemon_user="www"
rc_bg=YES

. /etc/rc.d/rc.subr

rc_reload() {
    ${rcexec} pumactl33 ${daemon_flags} phased-restart
}

rc_stop() {
    ${rcexec} pumactl33 ${daemon_flags} stop
}

rc_check() {
    ${rcexec} pumactl33 ${daemon_flags} status
}

rc_cmd "$1"

Database: PostgreSQL

I prefer a seven day log rotation. log_filename is formatted using strftime(3)

# postgresql.conf
logging_collector = on
log_directory = 'log'
log_filename = 'postgresql-%a.log'
log_rotation_age = 1d
log_rotation_size = 0
log_truncate_on_rotation = on
log_min_duration_statement = 250ms
log_checkpoints = on
log_connections = on
log_lock_waits = on
log_temp_files = 0

pg_auto_failover was intended to be used as a coordination and supervision daemon, but it can also be used to set basic tuning parameters

PG_AUTOCTL_DEBUG=1 pg_autoctl do pgsetup tune --pgdata /var/postgresql/data/

# basic tuning computed by pg_auto_failover
track_functions = pl
shared_buffers = '251 MB'
work_mem = '16 MB'
maintenance_work_mem = '256 MB'
effective_cache_size = '755 MB'
autovacuum_max_workers = 3
autovacuum_vacuum_scale_factor = 0.08
autovacuum_analyze_scale_factor = 0.02

Mail Relay: smtpd

Sidecomment depends on email to validate site codes and user codes. A dedicated email relay is running smtpd(8)

# smtpd.conf

listen on egress port 25
listen on egress port 587

action "mbox" mbox alias <aliases>
action "relay" relay

match for local action "mbox"
match from any for domain sidecomment.io action "mbox"
match from src <trusted> for any action "relay"
match auth from any for any action "relay"

Hosts running the application are configured to forward to this mail relay