Alessandro Ghedini /dev/random

Multi-VLAN DHCP+DNS home network setup with dnsmasq

Last week my ISP had an outage, and I discovered to my dismay that local DNS resolution on my home network stopped working too. My ISP having outages is nothing new, but the broken local DNS is, as the whole point of setting it up in the first place was to allow local services to keep working during the not-infrequent ISP oopsies.

My home network is largely run over Unifi hardware, with a Gateway Pro acting as, well, the network’s gateway. It comes with its own local DHCP and DNS setup based on dnsmasq which has worked fairly well in the past, so it’s not entirely clear whether this new failure mode is something that was introduced in a recent update (as I said, I’ve weathered ISP outages before without noticing the issue).

Looking a bit into it, it appears that the dnsmasq instance responsible for local DHCP and DNS is configured to use the gateway’s WAN interface, which seems a little strange, with the result that Unifi’s WAN fallback mechanism seems to disable it when the upstream connection breaks (thus breaking DNS as well).

I did a fair amount of searching and found a few mentions of similar issues from other people, though there didn’t seem to be a solution.

I decided that instead of trying to fix the issue locally on the gateway itself I would just deploy my own dnsmasq installation on separate hardware to prevent future updates from breaking it again.

Easier said than done.

Mo VLANs Mo Problems

Installing dnsmasq on my own host was easy enough, however I soon ran into some issues that didn’t exist in the previous setup.

You see, my network has a number of VLANs to keep local hosts tidy and organized. For example, one VLAN (with disabled Internet access) hosts all of my IoT tchotchkes which helps prevent major security issues, and the host running Home Assistant is connected to both the IoT VLAN for managing these devices, as well as the main network to allow users to interact with it.

With the Gateway Pro setup the Home Assistant host would get two DHCP leases, one for each VLAN, and more critically, its domain name would resolve to one or the other IP address depending on which VLAN the DNS query came from.

This didn’t work in my naive setup, as it turns out that a single dnsmasq instance can only associate a particular hostname with a single DHCP lease file entry, which means that the multi-VLAN host would always resolve to a single IP address regardless of the source VLAN (with only one lease having the correct hostname in the lease database, and the others listing * as the hostname).

The solution was running separate dnsmasq instances for DHCP (one per VLAN) plus a dedicated dnsmasq instance for DNS only, which is actually pretty much how the Unifi gateway setup works as well. The DHCP instances never listen on port 53, they just hand out addresses and use dnsmasq’s dhcp-script hook to write individual /etc/hosts-compatible files into a shared directory.

The DNS instance then uses hostsdir=/run/dnsmasq/hosts.d to pick up all those per-IP hosts files and resolve them. Since each file is keyed by IP address rather than hostname, there’s no conflict when the same device has leases on multiple VLANs, you just get multiple files for different IPs of the same host.

The pieces

Each DHCP instance is enabled as a systemd template unit (dnsmasq-dhcp@<VLAN>.service) so they can be managed independently, and each instance’s configuration is generated from my Ansible inventory.

It looks something like this (with some cruft removed for brevity):

domain=example.com

interface=

port=0

dhcp-authoritative
dhcp-leasefile=/var/lib/dnsmasq/leases-
dhcp-script=/usr/local/bin/dhcp-script.sh
script-on-renewal

Where renders to the netif's name, and to the actual netif device. Also note port=0 which means this instance only does DHCP, never DNS.

The dhcp-script.sh hook runs whenever a lease is added, renewed, or deleted, and writes one file per IP address into /run/dnsmasq/hosts.d/, containing the full hostname (with domain) plus the short name.

I couldn’t find an example of a similar setup anywhere, and the Unifi’s dhcp-script seems to be a binary so I couldn’t quite inspect what it does, so I ended up vibecoding my own script:

And for completeness, the DNS-only instance config is as follows:

domain=example.com

bind-dynamic
bogus-priv
hostsdir=/run/dnsmasq/hosts.d
localise-queries
no-hosts
no-resolv

local=/example.com/
server=127.0.0.1#5053

The hostsdir directive makes dnsmasq load all files from that directory as if they were static host entries, and localise-queries ensures each client gets resolved with the correct domain based on which interface it’s coming from.

dnscrypt-proxy

Finally upstream DNS resolution is handled by dnscrypt-proxy (largely to get all the nice modern goodies like DNS over HTTPS), which I run as a systemd socket-activated service on 127.0.2.1:5053.

The default Debian package listens on 127.0.0.1:53, so I just had to override the socket unit file to change port:

[Socket]
ListenStream=
ListenDatagram=
ListenStream=127.0.0.1:5053
ListenDatagram=127.0.0.1:5053

This avoids conflicting with dnsmasq’s own DNS on port 53.

Anyways, I hope this will be useful to the unfortunate souls (human or otherwise) that decide that hosting their own dnsmasq is “probably just going to take 10 minutes” in the future.

Building Debian packages using Linux namespaces

In the past few days I have been messing around with Linux namespaces, and developed a little tool (pflask) that automates the creation of simple Linux containers based on them (a sort of chroot(8) on steroids if you will).

While the whole raison d’être behind this project was “just because”, and many more mature solutions exist, I decided that it’d be nice to find an actual use case for this (otherwise I tend to lose interest pretty quickly) so I wrote a lil (and rather dumb) pbuilder clone that uses pflask instead of chroot.

The nice thing about pflask is that, differently from e.g. LXC, it doesn’t need any pre-configuration and can be used directly on a vanilla debootstrap(8)ed Debian system:

$ sudo mkdir -p /var/cache/pflask
$ sudo debootstrap --variant=buildd $DIST /var/cache/pflask/base-$DIST-$ARCH

Where $DIST and $ARCH are e.g. unstable and amd64.

Once that’s done just run pflask-debuild on the package sources:

$ apt-get source somepackage
$ cd somepackage-XYX
$ pflask-debuild

The script will take care of creating a new container, chroot(2)ing into it, installing all the required dependencies, building and signing the package (it also runs lintian!).

The main difference from pbuilder is that pflask will mount a copy-on-write filesystem (using AuFS) on the / of the container so that any modification (e.g. installation of packages) can be easily discarded once the container terminates (similarly to what cowbuilder(8) does, modulo the hardlinks hack).

Additionally, thanks to the mount namespace created inside the container, all of this will be isolated from the host system and other containers, so that multiple packages can be built simultaneously on the same base debootstrapped directory.

Another possibility would be that of disabling the network inside the container using a network namespace, in order to prevent the package build system from downloading stuff from Internet while at the same time maintaining the network active on the host system, but I haven’t done any experiment in this direction yet.

Note though that all of this is rather crude and experimental, but as a little hack it seems to work rather well (YMMV).

Demoscene - The Art of the Algorithms

Although existing art media have been transformed in the digital age, the advent of computers has brought new art forms into being. In the past, visual arts and music required both intellectual and physical skills, but in the present, computer programming permits people to make art just by using their minds. Moleman 2 presents a subculture of digital artists working with both new and old computing technology who push their machines to their limits.

Kernel module to disable ptrace()

I don’t really know why I ended writing this, but it all started as a way to do some Linux module coding.

Anyway, all this module does is overwriting the Linux syscall table, and replacing the ptrace() syscall with a custom one (which does nothing but printing a message).

Now, I’m quite sure there are better ways of doing this, so take the whole code just as a humble example of Linux module development.

The code is also on GitHub, as usual. There you can also find a Makefile to compile the code:

$ make

Then load the module with:

$ sudo insmod noptrace2.ko

And look at the output of dmesg:

$ dmesg | tail -1
[25374.003588] [noptrace2] ptrace syscall disabled

Which means everything went as expected.

x86 booting code

I have written a little snippet of code that magically boots on x86 hardware (not sure if it works on x86_64 as well). It’s a few lines of assembly code, just for fun.

It compiles with nasm(1):

$ nasm -o boot.bin boot.asm

It works pretty well under qemu(1), but I have not tried it on bare metal hardware yet. Not that it does anything fancy: it boots, prints a string and loops forever.

The code on GitHub includes a Makefile to make the iso image generation (using genisoimage(1)) and the vm start-up, easier.

With a simple make run it compiles the code, generates the iso and starts the virtual machine.

Enjoy :)