Quick and Dirty Containers with systemd
Docker and other containerization technologies are making the rounds in the
Linux community, but a lowly hero lurks beneath now every major Linux
systemd has arrived with mixed reviews -- you either love it or you hate it -- but one thing stands for certain: it gets the job done. Regardless of your emotions projected towards it, systemd is likely here to stay for a while, so you might as well exploit as many features as you can out of it.
systemd-nspawn containers are akin to FreeBSD Jails more than
Docker containers. They're basically just a fancy
with some handy built-in integrations with systemd. You can start, stop,
enable, and disable the containers as if they were regular services.
Keep in mind, however, that by its own admission,
systemd-nspawn is an experimental feature that hasn't been
thoroughly tested or audited. There are no guarantees of security or
stability; it's probably best to keep them out of production for the
time being. That said, here at BlackieOps we've been using
systemd-nspawn containers for Jenkins, Stash, and Jira for a
while now without any issues.
Creating the container
We will be working on a fresh install of CentOS 7, but this process is
possible on any system using systemd. The only difference will be the first
package installation step. Obviously, on Debian you will not be using
yum, and on Fedora your repo names and release versions will
be different (and you'll be using
First step is to "install" a new root filesystem into a directory.
# yum -y --releasever=7 --nogpg --installroot=/var/lib/machines/cool-container \ --disablerepo='*' --enablerepo=base install systemd passwd yum centos-release
This will create a directory,
/var/lib/machines/cool-container, and populate it with a new root
filesystem and a couple core packages.
Second, we need to enter into the container to set up some basic things
root password. We can use the
systemd-nspawn command directly for this:
# systemd-nspawn -D /var/lib/machines/cool-container
This drops you to a shell in the container without actually
"booting" anything inside of it (think of it as like
chroot-ing from a recovery mode. From here, we can set the
root password so we can log in later.
When you're done, just
^D out as usual and you'll be
dropped back to your host machine.
Aside: kernel auditing and containers
If you ignore this section and continue trying to boot the container, you will likely get a warning before the container starts about the kernel auditing subsystem. There are supposedly odd bugs that can surface if auditing is enabled, so we're just going to disable it. If this worries you, feel free to inspect the issue further, but since this is not a production system it's probably fine.
We just need to add a flag to the kernel parameters in our bootloader. This
will vary between distros, but for CentOS it's as easy as editing the
/etc/sysconfig/grub file and changing the
GRUB_CMDLINE_LINUX variable by appending
to the list of parameters.
After editing the parameters, we'll need to regenerate our GRUB configuration:
# grub2-mkconfig -o /etc/grub2.cfg
Configuring the base system
We now have a skeleton of a container installed, but we still need to actually configure what's inside of it, and get it prepped to start automatically, or at least as a service from systemd.
Since we now have access to the
root account, we can fully
"boot" the container:
# systemd-nspawn -bD /var/lib/machines/cool-container
-b flag is short for
--boot and basically
systemd-nspawn will search for an init binary and
execute it. You'll see the standard boot log fly by, and then be
dropped at a standard PTY login prompt. Log in with the root credentials
you set up previously, and now we can start installing things as if we were
on a brand new machine.
Once you have your container set up and everything is running and
configured, you can exit by "shutting down" the container as if
it was a physical machine:
whatever you usually use).
Managing the container
/var/lib/machines prefix at the beginning may have
seemed arbitrary, in fact it was intentional -- containers in this
directory will be auto-discovered by systemd and we can enable and manage
To have your container start with everything else when your host boots:
# systemctl enable systemd-nspawn@cool-container
And as you can perhaps guess, we can start and stop our container just as any other service:
# systemctl start systemd-nspawn@cool-container # systemctl stop systemd-nspawn@cool-container
Accessing the container
Accessing a running container can be a bit tricky; one option is to install
openssh in the container and have it run on a non-standard
port (as containers share the host's network interfaces).
Alternatively, you can access the machine through
machinectl without arguments will list all
running containers (and other VMs, etc). Interestingly, the older version
machinectl on CentOS does not allow us to use the
login argument (so you may want to install
openssh)... If you're on Fedora (or a different more
up-to-date distro), we can use
machinectl login command:
# machinectl login cool-container
... which will drop us at that familiar PTY prompt.
Since we don't necessarily want to halt the container to escape from this prompt, there is a panic button to disconnect: hit escape three times within a second (i.e., fast).
systemd-nspawn is an interesting technology
that shows promise. Its ubiquity through the proliferation of systemd
means containers are quite portable, easy to set up, and well-integrated
directly into the OS's init system.
Would I use it in production? Probably not. It's a very green technology and its immaturity is worrisome enough that my sleep cycles would be lessened dramatically by its deployment. For production "containers", FreeBSD Jails still provide the best security and featureset.
systemd-nspawn is staying on my internal
infrastructure, running my Atlassian stack, Jenkins, etc.; and it is
running those internal services quite well. But until its features are more
solidified and someone has verified it is at least moderately secure, it
won't be finding its way to my production stack for a few years