The Sidecomment Web Stack

sidecomment.io is a service that allows readers to select text and create a ticket with suggestions or corrections that the site owner can respond to. Like a commenting platform such as Hyvor Talk, JavaScript is included from an external CDN, and is authorized by a site code.

This concept developed after I noticed how many people were willing to take the time to write an e-mail to point out typos, broken links or errors on this online journal and other project pages.

Static Content and HTTP Routing: httpd/relayd

Since I'm using OpenBSD, httpd(8) is an my go-to daemon for static content

# httpd.conf

server "cdn.sidecomment.io" {
    log style forwarded
    listen on "*" port 8000

    root "htdocs/sidecomment.io/cdn"

    # Let's Encrypt
    location "/.well-known/acme-challenge/*" {
        root "/acme"
        request strip 2
    }

    location "/*" {
        directory auto index
    }
}

relayd(8) routes HTTP requests and provides TLS termination for https://

# relayd.conf
ext_addr = 0.0.0.0
table <httpd> { 127.0.0.1 }
table <puma> { 127.0.0.1 }

# TLS proxy all home services
http protocol "httpproxy" {
    tls keypair "sidecomment.io"

    match header set "X-Client-IP" value "$REMOTE_ADDR:$REMOTE_PORT"
    match header set "X-Forwarded-For" value "$REMOTE_ADDR"
    match header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

    pass request forward to <puma>
    pass request path "/*.*" forward to <httpd>
    pass request path "/.well-known/acme-challenge/*" forward to <httpd>
    pass request header "Host" value "cdn.sidecomment.io" forward to <httpd>
}

relay "https" {
    listen on $ext_addr port 443 tls
    protocol httpproxy
    forward to <httpd> port 8000
    forward to <puma> port 9292
}

relay "http" {
    listen on $ext_addr port 80
    protocol httpproxy
    forward to <httpd> port 8000
    forward to <puma> port 9292
}

Web Stack: Ruby/Sinatra/Haml

Any language can serve HTTP requests. I considered a few different frameworks, but I settled on Ruby because it gives me access to Haml templates. Sinatra is not perfect, but it is a very readable DSL. In addition to providing separate routes for HTTP request type, custom rules can be built

# main.rb

# test to see if the 'otp' query string is set
set(:otp) {|value| condition { params[:otp].nil? == value.nil? }}

get '/account', :otp => true do
end

get '/account' do
end

post '/account' do
end

cpan/gem/pip/npm/cargo-like systems are prone to regressions during upgrades since there is no release engineering that works in tandem with the operating system. For ruby-gems there are two strategies for working around broken builds:

# select an older version
gem33 install nokogiri -v 1.14.2

# build from top of tree
gem33 install specific_install
gem33 specific_install https://github.com/puma/puma.git

Application Server

Puma was designed to be a fast and efficient, but I was primarily interested in a server the reports useful error messages if it cannot start. This web application (like most?) is not designed to be thread-safe, so Puma must be set to use one thread per worker

# config.ru
require './main'
run Sinatra::Application

# puma.rb
app_dir = __dir__
port 9292
pidfile "#{app_dir}/run/puma.pid"
state_path "#{app_dir}/run/puma.state"
directory "#{app_dir}/"
stdout_redirect "#{app_dir}/log/puma.out", "#{app_dir}/log/puma.stderr.log", true
workers 3
threads 1, 1
activate_control_app "unix://#{app_dir}/run/pumactl.sock"

It usually possible to use cron to run periodic actions, or to deploy a dedicated service to handle these tasks, but the deployment of small applications can also be simplified by starting up a sidecar process. In worker mode the Puma configuration file is Ruby and is able to fork a process

# puma.rb
require_relative 'notify'

def log(stream, label, message)
  ts = Time.now.strftime('%m-%d-%Y %H:%M:%S')
  stream.puts "#{ts} #{label} #{message}"
end

before_fork do
  fork do
    Process.setproctitle('puma: sidecomment scheduler')
    loop do
      log $stdout, 'notify_tickets', notify_tickets.to_a
      log $stdout, 'notify_replies', notify_replies.to_a
      sleep 600
    rescue StandardError => e
      log $stderr, 'error', e.full_message
      next
    end
  end
end

Defining before_fork() allows the application to do things before the application is started, in this case start a subprocess that wakes up every 5 minutes. Logging to STDOUT and STDERR is captured in the standard puma logs.

The startup rc.d script supports Puma's graceful restart

#!/bin/ksh

daemon="puma33 -C /var/www/htdocs/sidecomment.io/puma.rb -e production"
daemon_flags="--control-url tcp://127.0.0.1:9293 --control-token ********"
daemon_user="www"
rc_bg=YES

. /etc/rc.d/rc.subr

rc_reload() {
    ${rcexec} pumactl33 ${daemon_flags} phased-restart
}

rc_stop() {
    ${rcexec} pumactl33 ${daemon_flags} stop
}

rc_check() {
    ${rcexec} pumactl33 ${daemon_flags} status
}

rc_cmd "$1"

Database: PostgreSQL

I prefer a seven day log rotation. log_filename is formatted using strftime(3)

# postgresql.conf
logging_collector = on
log_directory = 'log'
log_filename = 'postgresql-%a.log'
log_rotation_age = 1d
log_rotation_size = 0
log_truncate_on_rotation = on
log_min_duration_statement = 250ms
log_checkpoints = on
log_connections = on
log_lock_waits = on
log_temp_files = 0

pg_auto_failover was intended to be used as a coordination and supervision daemon, but it can also be used to set basic tuning parameters

PG_AUTOCTL_DEBUG=1 pg_autoctl do pgsetup tune --pgdata /var/postgresql/data/

# basic tuning computed by pg_auto_failover
track_functions = pl
shared_buffers = '251 MB'
work_mem = '16 MB'
maintenance_work_mem = '256 MB'
effective_cache_size = '755 MB'
autovacuum_max_workers = 3
autovacuum_vacuum_scale_factor = 0.08
autovacuum_analyze_scale_factor = 0.02

Mail Relay: smtpd

Sidecomment depends on e-mail to validate site codes and user codes. A dedicated email relay is running smtpd(8)

# smtpd.conf

listen on egress port 25
listen on egress port 587

action "mbox" mbox alias <aliases>
action "relay" relay

match for local action "mbox"
match from any for domain sidecomment.io action "mbox"
match from src <trusted> for any action "relay"
match auth from any for any action "relay"

Hosts running the application are configured to forward to this mail relay