monotux.tech

Mom, can we have anycast DNS?

Alpine Linux Raspberry Pi BGP DNS

After discovering diskless mode in Alpine Linux, I wanted to use it everywhere! But where to start…maybe it’s time to setup a BGP announced anycast DNS cluster with all the moving parts? All of this to avoid just using the reasonable tool1 for the job!

Table of Contents

Overview

We have anycast DNS at home

The goal is to have at least two Raspberry Pi 4Bs for internal DNS, so in case one dies I will still have DNS at home. I was initially planning to use something like keepalived and using a L2 VIP, but then I remembered how I solved it in Kubernetes – BGP! Everybody loves BGP, right?

When googling how to tie my BGP announcements to a service health probe, I discovered ExaBGP which was a perfect fit for this project.

In short:

When writing this draft I was planning to add a section which a reasonable person could stop at before descending into madness – but then I remember that our starting point is setting up anycast DNS, at home, using BGP. Any reasonable reader have already closed the tab :-)

Most of this entry is actually setup using ansible, in particular:

And a confusing visual description of the setup:

diagram

Limitations

I originally intended to setup a highly available DNS cluster. In my home network this isn’t realistically possible, as

This ended up being a semi-load balanced2, failover type of a setup – which I am still happy with as a hardware failure shouldn’t stop the service. I started listing all the steps required to make this closer to highly available and just writing that (very long) list was a chore.

Networking

As ExaBGP doesn’t manipulate routing tables or such, I was initially unsure how I should setup the VIP for each DNS server. I tested both adding the VIP to my loopback interface, and to setup a dummy interface…and decided to go with a dummy interface.

In Alpine this is easy, but requires manually installing iproute2 and loading the relevant kernel module.

# /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

auto dummy0
iface dummy0 inet manual
  link-type dummy
  address 10.0.64.1/32

Package and kernel module:

# apk add iproute2
# grep -q dummy /etc/modules || echo 'dummy' >> /etc/modules

DNS

I’m using knot-resolver again, mostly due to reasons but also due to it’s builtin support for DNSTAP.

First of all, install the basic dependencies:

# apk add knot-resolver knot-resolver-mod-dnstap knot-resolver-mod-http

As of 2026-03-26, Alpine Linux v3.23 ships knot-resolver v5.7.6 which uses the older, Lua based configuration. My setup looks something like this:

net.listen('::1', 53, { kind = 'dns' })
net.listen('127.0.0.1', 53, { kind = 'dns' })

-- VIP to announce
net.listen('10.0.64.1', 53, { kind = 'dns' })

modules = {
  'hints > iterate',
  'stats',
  'predict',
}

modules.load('dnstap')
dnstap.config({
  socket_path = "/run/dnscollector/dnstap.sock",
  -- yes, you have to log queries and responses...
  client = {
    log_queries = true,
    log_responses = true,
  },
})

-- this is too much already
cache.size = 100 * MB

-- send internal domains to our router
internalDomains = policy.todnames({ "home.arpa." "10.in-addr.arpa." })
policy.add(policy.suffix(policy.FLAGS({'NO_CACHE'}), internalDomains))
policy.add(policy.suffix(policy.STUB({'10.0.0.1'}), internalDomains))

-- RPZ handled outside of this entry
-- policy.add(policy.rpz(policy.DENY, "/etc/knot-resolver/rpz.db"))

policy.add(policy.all(
    policy.FORWARD({ "1.1.1.1", "8.8.8.8", "8.8.4.4" }))
)

There are some quirks with setting up the DNSTAP socket, as knot-resolver doesn’t have a runtime directory setup by default, just a PID file. We will deal with this in our dns-collector setup.

ExaBGP

ExaBGP is currently only available in the edge repository (“testing”), so we need to add it to our system:

# grep -q testing /etc/apk/repositories || \
    echo '@testing https://dl-cdn.alpinelinux.org/alpine/edge/testing' \
    >> /etc/apk/repositories

Now we can install packages from testing without having to run testing:

# apk add exabgp@testing py3-docopt@testing

Then we can bind ExaBGP to the kresd service:

# grep -q "rc_need" /etc/conf.d/exabgp || \
    echo 'rc_need="kresd"' >> /etc/conf.d/exabgp

Now, ExaBGP will only start if the knot-resolver service is running. We will also run a healthcheck in ExaBGP that will check if the DNS server is up, and only then will we announce the VIP.

The healthcheck configuration is quite simple:

# /etc/exabgp/healthcheck.conf
name = kresd
rise = 3
fall = 2
command = nc -uz 127.0.0.1 53
ip = 10.0.64.1/32

My ExaBGP is also quite simple:

# /etc/exabgp/exabgp.conf
neighbor 10.0.0.1 {
    router-id your-router-id-here;
    local-address your-local-ip-address-here;
    local-as 65011;
    peer-as 65001;

    api {
        processes [watch-service];
    }
}

process watch-service {
    # https://github.com/Exa-Networks/exabgp/wiki/Health-Checks
    run python -m exabgp healthcheck --config /etc/exabgp/healthcheck.conf;
    encoder text;
}

DNS-collector

Next part of this Rube Goldberg-machine is the DNS-collector, which will receive DNSTAP information from knot-resolver and expose a prometheus metrics endpoint which we can scrape and store the information in a time series database. In my case I am fronting this endpoint with Caddy so I can use automatic TLS and basic auth in one go, but the example below will only cover the DNS-collector bits!

In order to make knot-resolver DNSTAP functionality work, we need to create the socket knot-resolver expects to write DNSTAP data to before starting knot-resolver. We will solve this using OpenRC dependencies.

DNS-collector isn’t (as of 2026-03-26) packaged for Alpine Linux, but since its only a golang binary it’s easy to install and manage. In my ansible playbook I’ve built some logic for checking versions, downloading and unpacking it, but essentially it’s just a matter of:

# wget https://github.com/dmachard/DNS-collector/releases/download/v2.2.1/DNS-collector_2.2.1_linux_arm64.tar.gz
# tar zxf DNS-collector_2.2.1_linux_arm64.tar.gz
# chmod 0750 dnscollector
# mv dnscollector /usr/local/bin

Then I have defined an OpenRC service for dnscollector in /etc/init.d/dnscollector:

#!/sbin/openrc-run

depend() {
  need net
  # Make sure we start this before knot-resolver
  before kresd
}

name=$RC_SVCNAME
command="/usr/local/bin/dnscollector"
command_args="-config /etc/dnscollector.yaml"
command_user="kresd"

supervisor="supervise-daemon"
pidfile="/run/${RC_SVCNAME}.pid"

# Create a directory for dnscollector if it doesn't exist, and assign correct
# ownership of it
start_pre() {
    checkpath --directory --owner kresd:kresd /run/dnscollector
}

Then, finally, a basic configuration for my use-case:

# /etc/dnscollector.yaml
global:
  server-identity: "dns01.home.arpa"
  telemetry:
    enabled: true
pipelines:
  - name: "sock-input"
    dnstap:
      sock-path: /run/dnscollector/dnstap.sock
    transforms:
      # https://github.com/dmachard/DNS-collector/blob/main/docs/transformers/transform_normalize.md
      normalize:
        qname-lowercase: true
        qname-replace-nonprintable: true
        add-tld: true
        add-tld-plus-one: true
        rr-lowercase: true
        quiet-text: true
    routing-policy:
      forward:
        # https://github.com/dmachard/DNS-collector/blob/main/docs/loggers/logger_prometheus.md
        - name: prometheus
          prometheus:
            listen-ip: 0.0.0.0
            listen-port: 8080

Those familiar with OpenTelemetry agent configuration should feel right at home with dns-collector! With the dns-collector setup to expose metrics, you can use this Grafana dashboard for metrics, and this dashboard for logs (even though it’s not covered how to setup logs here)

By now, the dependency chain should be:

diagram

Bonus: RPZ adblocking

Response policy zone (RPZ) is a…

…mechanism to introduce a customized policy in Domain Name System servers, so that recursive resolvers return possibly modified results. By modifying a result, access to the corresponding host can be blocked.

It allows us to block ads using a standardized format that works with most current DNS servers, including knot-resolver!

I like using the big oisd.nl block list, which is available in RPZ. We will setup a cronjob that will download this blocklist once a day, validate it and use it if works. Since I am using Alpine Linux in diskless mode I have to commit the new file to system state as well.

First of all, install the dependencies and ensure cron is started at boot:

# apk add bind-tools curl
# rc-update add crond

Then, the script:

#!/bin/sh
# Usage: rpz-update.sh [URL] [DEST_PATH] [SERVICE]

set -eu

RPZ_URL="${1:-https://big.oisd.nl/rpz}"
DEST_PATH="${2:-/etc/knot-resolver/rpz.db}"
SERVICE="${3:-kresd}"

WORK_DIR="/tmp"
TMP_FILE="${WORK_DIR}/rpz_download_$$.tmp"
LOG_TAG="rpz-update"

# Please the markdown linter and make these oneliners multiple lines...
log()  { 
    echo "$(date '+%Y-%m-%d %H:%M:%S') [INFO]  $*" \
    | tee -a /var/log/rpz-update.log
    logger -t "$LOG_TAG" "$*" 2>/dev/null || true; 
}

warn() { 
    echo "$(date '+%Y-%m-%d %H:%M:%S') [WARN]  $*" \
    | tee -a /var/log/rpz-update.log >&2; 
}

die() { 
    echo "$(date '+%Y-%m-%d %H:%M:%S') [ERROR] $*" \
    | tee -a /var/log/rpz-update.log >&2
    rm -f "$TMP_FILE"; exit 1; 
}

download_file() {
    log "Downloading RPZ file from: $RPZ_URL"

    if command -v curl >/dev/null 2>&1; then
        curl -fsSL --max-time 30 --retry 3 --retry-delay 5 \
            -o "$TMP_FILE" "$RPZ_URL" \
            || die "curl failed to download $RPZ_URL"
    elif command -v wget >/dev/null 2>&1; then
        wget -q --timeout=30 --tries=3 \
            -O "$TMP_FILE" "$RPZ_URL" \
            || die "wget failed to download $RPZ_URL"
    else
        die "Neither curl nor wget found. Install one with: apk add curl"
    fi

    [ -s "$TMP_FILE" ] || die "Downloaded file is empty."
    log "Download complete ($(wc -c < "$TMP_FILE") bytes)."
}

validate_rpz() {
    log "Validating RPZ zone file..."

    if command -v named-checkzone >/dev/null 2>&1; then
        zone_name=$(grep -i '^@ IN SOA' "$TMP_FILE" | cut -d' ' -f4)
        zone_name="${zone_name:-rpz}"
        named-checkzone "$zone_name" "$TMP_FILE" >/dev/null 2>&1 \
            || { warn "named-checkzone reported errors."; }
    else
        die "bind-tools is not installed, giving up validation"
    fi

    log "Validation passed."
}

deploy_file() {
    dest_dir=$(dirname "$DEST_PATH")

    [ -d "$dest_dir" ] || die "Destination directory does not exist: $dest_dir"

    if [ -f "$DEST_PATH" ]; then
        cp "$DEST_PATH" "${DEST_PATH}.bak" \
            && log "Backed up existing file to ${DEST_PATH}.bak"
    fi

    mv "$TMP_FILE" "$DEST_PATH" \
        || die "Failed to move file to $DEST_PATH"

    chmod 640 "$DEST_PATH"
    chgrp kresd "$DEST_PATH"

    log "File deployed to $DEST_PATH"
}

restart_service() {
    log "Restarting service service: $SERVICE"

    rc-service "$SERVICE" restart \
        || die "rc-service failed to restart $SERVICE"

    log "Service $SERVICE restarted successfully."
}

lbu_commit() {
    log "Committing system state"

    /usr/sbin/lbu commit \
        || die "Could not commit state!"

    log "State committed."
}

main() {
    log "=== RPZ update started ==="
    log "  URL:     $RPZ_URL"
    log "  Dest:    $DEST_PATH"
    log "  Service: $SERVICE"

    download_file
    validate_rpz
    deploy_file
    restart_service
    lbu_commit

    log "=== RPZ update completed successfully ==="
}

main

Then, make the script executable:

# chmod +x path/to/rpz.sh

Finally, add it to your crontab!

# From /etc/crontabs/root
5 4 * * * sleep $(shuf -i 10-300 -n 1) && path/to/rpz.sh

The shuf command returns a random number between 10-300, which feels like good netiquette to avoid being a part of a thundering herd.

Bonus: Configuring BGP multi-path routing

Continuing the example configuration in this entry, below will enable equal-cost multi-path routing (ECMP), which will allow some level of load balancing over all nodes announcing the VIP.

router id 192.0.8.1;

protocol direct {}

protocol kernel {
    ipv4 {
        import none;
        export all;
    };
    # Allow ECMP
    merge paths yes;
}

template bgp k8s {
    local as 65001;
    source address 192.0.8.1;
    ipv4 {
        import all;
        export none;
    };
}

protocol bgp node01 from k8s {
    neighbor 192.0.8.10 as 65010;
}

protocol bgp node02 from k8s {
    neighbor 192.0.8.11 as 65010;
}

protocol bgp node03 from k8s {
    neighbor 192.0.8.12 as 65010;
}

template bgp dns {
  local as 65001;
  source address 10.0.0.1;

  ipv4 {
    import all;
    export none;
  };
}

protocol bgp dns01 from dns {
  neighbor 10.0.0.2 as 65011;
}

protocol bgp dns02 from dns {
  neighbor 10.0.0.3 as 65011;
}

Conclusion

I am not sure if writing this was a good idea, but I had some fun setting this up and it seems to Work In My Homelab(tm)!

To summarize:

  1. Yes, I am looking at you, Pihole! ↩︎

  2. I am not sure why, but my router isn’t properly load balancing the nodes but is sending most of the queries to the same machine. Unsure if the same issue as in scottstuff.net/posts/2025/01/11/linux-ecmp-not-balanced/ (scottstuff.net is an awesome blog) ↩︎