Updated: August 25, 2023 at 4:15 PM
Originally Posted: August 17, 2023 at 12:00 PM
Edited on 2023-08-25 to add how to start the jail on each boot.
The latest iteration of my home firewall has a spare interface. Many switches these days have a dedicated management interface, and I wanted something similar for my firewall. I want an interface that I can plug in, get an IP via DHCP, and I can ssh into and it’ll “just work” even if there’s a misconfiguration, or an IP conflict.
are a way to isolate an interface such that it won’t interfere (or receive
interference) from other interfaces. Some may have just used epair and
assigned IPs, but this could cause conflict, while using a unix domain
socket keeps things isolated, and FreeBSD ships w/ everything needed to
make this work. A package like
socat isn’t needed.
Configuring the host was straight forward, make a directory for the socket:
And then add a line to
/etc/inetd.conf to listen to incoming connections
on the socket and launch
/var/mgmt/mgmt.ssh.sock stream unix nowait root /usr/sbin/sshd sshd -i
-i option to
sshd tells it to run in “
inetd” mode, which means that
the stdin and stdout of the process is the socket to use for communication.
The next harder part was getting a jail configured that would accept incoming connections and forward them to the unix domain socket.
First the jail configuration, which goes in
/etc/jail.conf or similar location:
host.hostname = mgmt; # Hostname
path = "/usr/jails/mgmt/root"; # Path to the jail
mount.fstab="/usr/jails/mgmt/fstab"; # mount spec
mount.devfs; # Mount devfs inside the jail
devfs_ruleset = "101";
exec.start = "/bin/sh /etc/rc"; # Start command
exec.stop = "/bin/sh /etc/rc.shutdown"; # Stop command
There isn’t anything special in this. It’s pretty standard jail
configuration, the differences are the vnet configuration, and the
devfs_ruleset is necessary in order to expose the bpf interface
used by dhclient. This required the following lines in
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $mydevfsrules_jail
add path 'bpf*' unhide
Note that after adding the above lines, you need to run:
service devfs start
to load the rules (per devfs(8) man page).
Note: I learned my mistake not to number my blocks immediately after
the standard defaults (from
/etc/defaults/devfs.rules). I had done that
befure but there’s now a conflict, so I skip ahead a bit to get a unique range.
Now I needed to make a number of directories for the jail:
mkdir -p /usr/jails/mgmt/root
mkdir -p /usr/jails/mgmt/etc
mkdir -p /usr/jails/mgmt/var/mgmt
mkdir -p /usr/jails/mgmt/tmp
I needed to setup the
fstab for the jail:
# Device Mountpoint FStype Options Dump Pass#
/ /usr/jail/mgmt/root nullfs ro 0 0
/usr/jail/mgmt/etc /usr/jail/mgmt/root/etc nullfs rw 0 0
/usr/jail/mgmt/var /usr/jail/mgmt/root/var nullfs rw 0 0
/var/mgmt /usr/jail/mgmt/root/var/mgmt nullfs ro 0 0
/usr/jail/mgmt/tmp /usr/jail/mgmt/root/tmp nullfs rw 0 0
This is a little bit more tricky, It first
nullfs mounts the root system.
I’m using ZFS boot environments, so this is a pretty clean FreeBSD install
without much host specific data. It then mounts some jail specific directories
var and finally mounts the shared directory w/ the
unix domain socket to the host system. Also note that a couple of the mounts
are read-only to prevent the jail from modifying the system.
etc directory was populated from the system
tar -cf - -C /etc . | tar -xf - -C /usr/jails/mgmt/etc
Then the jail was configured, first
# Management port
# necessary as devd can't be run in a jail
inetd_enable="YES" # Run the network daemon dispatcher (YES/NO).
The key part of this configuration that took me a while to figure out was
synchronous_dhclient line. It used to be that
netif would start
dhclient, but in order to better handle USB ethernet devices and other
removable interfaces, it was moved to
devd. The only problem is that
devd hasn’t been jail’ified, and you can’t run it to get things like
link notifications that would normally launch dhclient. Setting this to
yes, makes sure it gets launched when the jail starts.
And then the following line was added to
ssh stream tcp nowait root /usr/bin/nc nc -N -U /var/mgmt/mgmt.ssh.sock
This is the part that will forward incoming connections to the ssh port on to the unix domain socket.
Now that everything is configured, a simple
jail -c mgmt will get
the jail running and accepting connections.
To get the jail to start every boot, add the following to the host’s
jail_enable="YES" # Set to NO to disable starting of any jails
This was testing and deployed on a FreeBSD 14-CURRENT build as of August
8th, 2023, or more specifically, from
main-n264621-09c20a29328, but it
should work on all currently supported FreeBSD releases.