OSPF and DNS NodePort

After setting up a basic Kubernetes Lab, a dynamic routing protocol provides effective routing at all layers

Node-to-node
Core switch
Edge router

To make name resolution work, we will configure a stub zone and port forwarding for DNS.

OpenBSD Router and OSPF, OSPFv3

OSPF can be run over any interface. Here we will advertise on the interface vlan2

# /etc/ospfd.conf
router-id 192.168.0.3

area 0.0.0.2 {
  interface vlan2 {
    metric 20
  }
}

The configuration for OSPFv3 (for IPv6) is exactly the same. EIGRP is also a fine option if supported by the network switches.

Kube Nodes

A number of routing daemons are avaialble for Linux. This example uses FRR, to redistribute the connected network on cni0

cat > /etc/frr/frr.conf <<CONF
hostname $(hostname -s)

router ospf
  ospf router-id $(hostname -I | awk '{ print $1 }')
  network 192.168.2.0/24 area 0.0.0.2
  redistribute connected
exit

interface $(route --inet6 -n | awk '/::\/0/ { print $NF; exit }')
  ipv6 ospf6 instance-id 0
  ipv6 ospf6 area 0.0.0.2
exit

router ospf6
  ospf router-id $(hostname -I | awk '{ print $1 }')
  area 0.0.0.2 range fd00:52::/64
  redistribute connected
exit
CONF

Now each Kubernetes node is a rounter visible to all other nodes

mykube2$ ip -6 route | grep fd00
fd00:52::/64 dev enp0s5 proto kernel metric 100 pref medium
fd00:f4::/64 nhid 19 via fe80::216:3eff:fe29:b8d9 dev enp0s5 proto ospf metric 20 pref medium
fd00:f5::/64 dev cni0 proto kernel metric 256 pref medium
fd00:f6::/64 nhid 20 via fe80::216:3eff:fe64:835a dev enp0s5 proto ospf metric 20 pref medium
default via fd00:52::7 dev enp0s5 proto static metric 100 pref medium

Juniper EX

One advantage of using a dynamic routing protocol is that core switching, as well as the router on the edge can route this traffic

set protocols ospf area 0.0.0.2 interface irb.82
set protocols ospf import into-ospf
set protocols ospf3 area 0.0.0.2 interface irb.82
set protocols ospf3 import into-ospf
set policy-options policy-statement into-ospf term static then accept

PicOS

Even small PicOS switches from fs.com have full routing support

set protocols ospf area 0.0.0.2
set protocols ospf network 192.168.2.0/24 area "0.0.0.2"
set protocols ospf interface vlan-82
set protocols ospf6 area 0.0.0.2
set protocols ospf6 interface vlan-82 area "0.0.0.2"

Forwarding DNS using a NodePort Sevice

Kubernetes allows a special kind of service to be defined that only modifies forwarding rules

apiVersion: v1
kind: Service
metadata:
  name: kube-dns-external
  namespace: kube-system
spec:
  selector:
    k8s-app: kube-dns
  type: NodePort
  ports:
  - name: dns-udp
    port: 53
    targetPort: 53
    nodePort: 30053
    protocol: UDP
  sessionAffinity: None

$ kubectl get svc --namespace kube-system
NAME                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
kube-dns            ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   19h
kube-dns-external   NodePort    10.103.147.139   <none>        53:30053/UDP             19h

Unbound is now able to use any Kubernetes node to answer nameserver queries!