.. | ||
examples | ||
scripts | ||
interactive_setup.sh | ||
README.md |
How to Create a Custom Container on UnifiOS 3.x+
This is a guide that shows you how to create your own container on UnifiOS 3.0+, and how to install custom services in your container (such as pihole or adguard home).
Starting with UnifiOS 3.0, podman/docker support has been removed due to a kernel change. However, you can still create a container with systemd-nspawn, which is what this guide will focus on.
Table of Contents
Instructions
Step 1. Create the Container
The following commands are all perfomed on the Unifi router in SSH.
-
We first need to install systemd-container and debootstrap. We will use debootstrap to create a directory with a base debian system, and then use systemd-nspawn to boot the container.
apt -y install systemd-container debootstrap
-
Next, we use debootstrap to create a directory called
debian-custom
with a base debian system in/data/custom/machines
.mkdir -p /data/custom/machines cd /data/custom/machines debootstrap --include=systemd,dbus unstable debian-custom
- This process can take up to 10 minutes to download and install all the packages.
- The container folder will be 390MB after installation, but can increase to 1GB+ after installing many services (storage management is up to you).
- Note: Instead of debootstrap, you can also use pacstrap or other distributions' tools to create an Arch Linux, Fedora, or other container instead of a debian container (see examples here).
-
Finally, let's bring up a shell on this container, set the root password, and enable the networking service. Run each command one-by-one.
systemd-nspawn -M debian-custom -D /data/custom/machines/debian-custom passwd root systemctl enable systemd-networkd echo "nameserver 1.1.1.1" > /etc/resolv.conf echo "debian-custom" > /etc/hostname exit
- The first command should put you in a shell in the container. If it doesn't work, something went wrong.
- Note the password will be hidden when you are typing it out in the
passwd root
command, and you will be asked to type it twice. - We also set the default nameserver to 1.1.1.1 in resolv.conf. You can change this to your own DNS or omit this command if you plan to configure the DNS later.
- The hostname can be set to whatever you want, here debian-custom is used as an example.
Step 2. Configure the Container
Now that the container is created, let's configure it. Make sure you are back on the host OS and not in the container.
-
First, we will link the container to
/var/lib/machines
so we can control it withmachinectl
.mkdir -p /var/lib/machines ln -s /data/custom/machines/debian-custom /var/lib/machines/
-
Next, we will create a
debian-custom.nspawn
file in/etc/systemd/nspawn
to configure parameters for the container (such as network, bind mounts, etc). Here I use vim to create the file as a personal preference.mkdir -p /etc/systemd/nspawn vim /etc/systemd/nspawn/debian-custom.nspawn
-
For a container that has access to all the host network interfaces and full capabilities to do anything to the system, here is an example nspawn configuration file. Note it is important to set
Boot=on
so systemd boots up inside the container.[Exec] Boot=on Capability=all ResolvConf=off [Network] Private=off VirtualEthernet=off
-
For a more isolated container configured with a macvlan bridge, follow Step 2A below instead before running the container.
-
For other options, see the nspawn manpage here.
-
-
After you've configured your nspawn file, let's boot up the container and see if it works.
machinectl start debian-custom machinectl enable debian-custom
- If the container booted up, you can check
machinectl status debian-custom
for information (hint: press 'q' to exit the status log). - The second enable command will enable the container to start on boot.
- If the container booted up, you can check
-
Now that the container is running, we should be able to open a shell or login to it.
- Typing
machinectl shell debian-custom
should open a shell to the machine and bypass login. Typingexit
in this shell will exit back to the host Unifi OS. - Typing
machinectl login debian-custom
will give you a login prompt like a normal Linux system. In most cases, you can just usemachinectl shell
to bypass the login for easier access. If you do use the login instead of the shell, you can exit the container by holding theCtrl
key and pressing the]
key 3 times.
- Now that you have access to your own container, you can install whatever services you want within it like a normal Linux system (see examples below). Make sure you ran
machinectl enable debian-custom
so the container starts on boot.
Step 2A: Configure the Container to use an Isolated MacVLAN Network
This configuration is only needed if you want to isolate the container's network with a macvlan bridge. The following steps are all performed on the host OS.
-
Download the 10-setup-network.sh script to
/data/on_boot.d
and configure it with your VLAN and IPs for your container and gateway. This script will create a brX.mac interface as a gateway bridge for containers to communicate with.mkdir -p /data/on_boot.d && cd /data/on_boot.d curl -LO https://raw.githubusercontent.com/peacey/unifios-utilities/nspawn/nspawn-container/scripts/10-setup-network.sh vim 10-setup-network.sh
- Modify
VLAN
to an existing VLAN network that you want your container to be on. The default is VLAN 5. Make sure this VLAN network is created in Unifi first with a unique subnet and IP (do not use the same IP as you will use for IPV4_IP or IPV4_GW in this script). - Modify
IPV4_GW
to set the gateway interface's IP. The default is 10.0.5.1/24, but you can use whatever subnet you want as long as it's different than any Unifi subnet. - Modify
IPV4_IP
to your preferred container IP. The default is 10.0.5.3, but you can use whatever you want as long as its on the same subnet as the gateway subnetIPV4_GW
. - Also modify
IPV6_GW
andIPV6_IP
if you need IPV6 support. Leave them empty for no IPV6 support.
- Modify
-
Create or modify your
/etc/systemd/nspawn/debian-custom.nspawn
file with the following parameters. This will tell nspawn to isolate the network and create a macvlan interface in the container from our VLAN bridge. This interface will be called mv-br5.[Exec] Boot=on ResolvConf=off [Network] MACVLAN=br5
- Change br5 to brX where X = VLAN number you used in
10-setup-network.sh
.
- Change br5 to brX where X = VLAN number you used in
-
Configure your container to set the IP and gateway you defined in
10-setup-network.sh
by creating a network file in the folder/etc/systemd/network
under your container's directory. Name this filemv-brX.network
where X = VLAN number you used (e.g.mv-br5.network
).cd /data/custom/machines/debian-custom/etc/systemd/network vim mv-br5.network
-
The following is an example configuration based on the default settings in
10-setup-network.sh
.[Match] Name=mv-br5 [Network] IPForward=yes Address=10.0.5.3/24 Gateway=10.0.5.1 Address=fd62:89a2:fda9:e23::3/64 Gateway=fd62:89a2:fda9:e23::1
-
Make sure to change
Name
to the correct VLAN. -
Change
Address
andGateway
accordingly if you changed the settings in10-setup-network.sh
. -
You can remove the last 2 lines with IPv6 addresses if you don't need IPv6.
-
-
Run the
10-setup-network.sh
script, start the container, open a shell on the container, and check the network.chmod +x /data/on_boot.d/10-setup-network.sh /data/on_boot.d/10-setup-network.sh machinectl reboot debian-custom machinectl shell debian-custom ip addr show ping -c4 1.1.1.1
- You should see the correct IP defined on mv-br5. If no IP has been assigned, make sure you enabled and started the systemd-networkd service and check again:
systemctl enable --now systemd-networkd
- If you still don't see any IP on mv-br5, then double-check you're using the correct VLAN and put the configuration in the correct location. You can also check
journalctl -eu systemd-networkd
for any errors. - If pinging 1.1.1.1 doesn't work from within the container, double-check you set the correct container IP in your 10-setup-network.sh.
- You should see the correct IP defined on mv-br5. If no IP has been assigned, make sure you enabled and started the systemd-networkd service and check again:
-
The script
10-setup-network.sh
in/data/on_boot.d
needs to be started on boot.- If you've installed the udm-boot service, it should automatically run any scripts in
/data/on_boot.d
and no further setup is needed. - If you prefer not to use udm-boot and instead use your own systemd boot service, here is an example systemd service to run this script at boot. Save it to
/etc/systemd/system/setup-network.service
in the host OS (not container) and then enable it withsystemctl enable setup-network
.
- If you've installed the udm-boot service, it should automatically run any scripts in
Step 3: Configure Persistence Across Firmware Updates
When the firmware is updated, /data
(which contains our container storage) and /etc/systemd
(which contains our boot scripts) are preserved, but /var
and /usr
is deleted by the firmware update script. Any additional debian packages that are installed in the host OS like systemd-container are also deleted. This means we need to reinstall the systemd-container package and re-link our container to /var/lib/machines (for machinectl access) when the firmware is upgraded. This can be accomplished with a simple boot script that checks to see if this package is installed on boot.
-
Download the 0-setup-system.sh script into /data/on_boot.d.
mkdir -p /data/on_boot.d && cd /data/on_boot.d curl -LO https://raw.githubusercontent.com/peacey/unifios-utilities/nspawn/nspawn-container/scripts/0-setup-system.sh chmod +x /data/on_boot.d/0-setup-system.sh
-
Download the backup dpkg package files for systemd-container and dependencies into
/data/custom/dpkg
. These packages will only be used as a backup install in case the Internet is down after the first boot after an update.mkdir -p /data/custom/dpkg && cd /data/custom/dpkg apt download systemd-container libnss-mymachines debootstrap arch-test
-
The script
0-setup-system.sh
in/data/on_boot.d
needs to be started on boot.- If you've installed the udm-boot service, it should automatically run any scripts in
/data/on_boot.d
and no further setup is needed. - If you prefer not to use udm-boot and instead use your own systemd boot service, here is an example systemd service to run this script at boot. Save it to
/etc/systemd/system/setup-system.service
in the host OS (not container) and then enable it withsystemctl enable setup-system
.
- If you've installed the udm-boot service, it should automatically run any scripts in
Step 4: Install Custom Services
Services can be installed in the container like any linux system. For debian containers, you can use apt or other manual methods. Follow the Debian/Linux guide for your particular software that you want to install.
- Common examples
FAQ
-
How do I access a folder from the host OS in the container (e.g. the /data directory)?
-
Edit your
.nspawn
config file and add the following[Files]
section. You can specify multipleBind=
orBindReadOnly=
lines to bind mount multiple directories. See nspawn manpage for more details.[Files] Bind=/data:/data
-
-
I am getting security errors trying to run certain privileged commands in the container.
- Edit your
.nspawn
config file and addCapability=all
under the[Exec]
section to unlock all security capablitlites. You can also permit or restrict capabilities for enhanced security (see the nspawn manpage for more information).
- Edit your
-
Some programs complain of a missing /lib/modules folder and can't access kernel modules.
- Edit your
.nspawn
config file and addBindReadOnly=/lib/modules
to the[Files]
section. This will bind mount the /lib/modules folder from the host OS. You might also want to try unlocking all capabilitites or specific ones (as in the above question) if you're still having issues with permissions using certain modules.
- Edit your
-
iptables doesn't work in the container after I installed it.
-
You need to use iptables-legacy and not iptables-nft because the host OS is still using the legacy iptables. If using a Debian container, you can switch to the legacy iptables with the following commands executed from within the container. Also, you won't be able to see or modify the host's iptables entries if you're using a private/macvlan network for the container.
update-alternatives --set iptables /usr/sbin/iptables-legacy update-alternatives --set ip6tables /usr/bin/ip6tables-legacy
-