OpenBSD has had for a couple of years its own virtual machine system, vmm. Despite being at a fairly early stage [1], its development is proceeding at a fast pace, it is a joy to configure and use – OpenBSD style, and I’ve been using it to run Alpine Linux VMs for a while.

It took me some time to get Ubuntu running on it, but the final solution is straightforward. So here are my notes on how to do this.

Problem

I wanted some Ubuntu/Debian-like VM on my OpenBSD 6.4 machine because of “reasons” (some Machine Learning packages don’t like OpenBSD nor Alpine Linux).

For those who don’t know, vmm is OpenBSD’s very own virtual machine manager. As with most software coming from OpenBSD’s developers, it is well-thought, and it is simple to configure, with a very clear syntax.

The main obstacle for installing Ubuntu/Debian on vmm is that the OS should support installation via serial console.

Plenty of guides exist for configuring Alpine Linux on vmm (it supports serial console installation out of the box), but AFAIK only a couple are dedicated to Ubuntu/Debian:

  • [2] for Ubuntu cloud images – an overkill for what I need,
  • [3] for Debian.

The latter option, unfortunately, requires making the actual installation via qemu, and then importing the image from vmm. I thought there should be a more direct way of doing this, and indeed there was.

I’ll give the final solution first, and then enumerate failed attempts – should anyone wish to try different solutions.

Solution

The solution was to use the minimal installer for Ubuntu 18.04. I tried many versions/distros: this is the only one that did not cause any troubles.

To get it running, we will need to:

  • edit the installation ISO image to support the serial console,
  • proceed with the standard installation on vmm.

Note that once the installation image is configured for serial console installation, the actual configuration and installation on vmm is straightforward.

Preparing the ISO image

We first need to enable serial console installation for the ISO image. This requires installing xorriso (run pkg_add xorriso). The following steps are adapted from [4]. First download the minimal installer image, mini.iso, from the link above. With root privileges, run:

vnconfig vnd0 mini.iso
mkdir old new
mount /dev/vnd0c old
cp -r old/* new
umount old
vnconfig -u vnd0

This mounts the ISO image read-only and copies its content into directory ./new.

Then edit the files as follows:

new/isolinux.cfg:

# D-I config version 2.0
# search path for the c32 support libraries (libcom32, libutil etc.)
serial 0 115200
console 0
path
include menu.cfg
default vesamenu.c32

new/txt.cfg:

default install
label install
        menu label ^Install
        menu default
        kernel linux
        append vga=normal initrd=initrd.gz --- console=ttyS0,115200n8

new/adtxt.cfg:

label expert
        menu label ^Expert install
        kernel linux
        append priority=low vga=normal initrd=initrd.gz --- console=ttyS0,115200n8

For the installation, we will be using the default “install” mode.

Finally, create the new image, mini-serial.iso:

$ xorriso -as mkisofs -r -J -joliet-long -l -cache-inodes -partition_offset 16 \
	-A "Ubuntu" -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 \
	-boot-info-table -o mini-serial.iso  ./new

I henceforth refer to mini-serial.iso as image.iso.

Install

Installation steps are pretty standard and distro-independent [5].

We first create the disk where the VM will be installed.

$ vmctl create disk.img -s 30G

Note that the actual size used by the newly created disk.img is actually fairly low (the space is dynamically allocated):

$ du -h disk.img
96.0K

vmm has a very nice configuration syntax. For the installation, we will include the following in /etc/vm.conf:

vm "ubuntu" {
	memory 2G
	cdrom "/path/to/image.iso"
	disk "/path/to/disk.img"
	interfaces 1
	local interface tap
	owner user
	disable
}

Where user is replaced by your user; this allows such user to start/stop/pause the VM without root privileges.

Now for the installation:

$ vmctl start ubuntu -c

And follow the installation. Simple as that.

After the installation, you can start the VM with the same command. However, I’d recommend setting up an ssh server on the machine, and to launch the VM without the console:

$ vmctl start ubuntu

Authentication via public key is a must. Note that this also allows X forwarding:)

Optionally, you can now remove the parameter cdrom from /etc/vm.conf.

Failed attempts

The steps for installing Ubuntu or Debian are more or less as follows:

  • edit the ISO image to support serial console installation,
  • install the thing with vmm.

Simple enough, huh? Well, I tried so many different Ubuntu&Debian distros, until I found one that worked (see above).

Since along the way I solved some problems that brought me closer (but not close enough) to installing other Ubuntu versions, I trace my steps here in case someone (e.g., my future self) encounters them.

Debian

I stopped trying with Debian distros because:

  • the installer wouldn’t see the installation cdrom, and even when opening a shell I could not find the device,
  • the installer wouldn’t see my network interface.

Debian can be installed with the qemu method [3], but that’s out of scope.

In “Alternative method” I reference a guide which managed to install Debian on vmm by first adding the virtio modules on the image.

Ubuntu server

As for Ubuntu distros, I did make some progress. For instance, with Ubuntu server 16.04.5, I managed to get quite far with the installation process.

To configure a Ubuntu ISO, one can proceed as we did above for the minimal Ubuntu install: extract the image’s content, edit its files, and create a new ISO with serial console activated. (Note that this time the files to edit are in new/isolinux.) However, once the installation is launched, it usually does not recognise the cdrom. The solution to this is to escape to a shell and to mount the (virtual) CD manually. In my case this meant running: mount /dev/sr0 /cdrom.

After this, the installation will proceed as normal, but it will get stuck with error:

An Installation step failed. You can try to run the failing item again from the
menu, or skip it and choose something else.

The failing step is: Install the system

Rather unhelpful. I asked people on #ubuntu, I checked the network connection was working, and I tried various disk partitioning methods without luck: I could not move past this error.

Alternative method

Update 01/11/2018

On #openbsd, I was pointed out to a guide for installing Debian on vmm [6]. This guide uses a boot image – rather than an ISO, which still poses the issue that cdrom and network interface are not recognised.

To solve this, they suggest uncompressing the image, and adding the virtio modules before installation.

Concluding remarks (actually, rather OT)

vmm is awesome. It is clear, well-structured, and competitors such as VirtualBox have already much to envy (e.g., a clean text-based configuration file and sensible command line options). Clearly, there are several things in vmm’s TODO list (e.g., [7]), but I see it as very promising software, and I can only thank its developers sincerely. Props also go to services like OpenBSD Amsterdam, who’ve been offering OpenBSD VMs using vmm for long: their effort certainly motivates vmm’s development.

References

[1] https://www.reddit.com/r/openbsd/comments/7qdsya/vmm4_production_use_and_performance

[2] https://gist.github.com/reyk/6d369c5c0bd0c76f4906f83933f3bb71

[3] https://forums.bunsenlabs.org/viewtopic.php?id=4709

[4] http://pcengines.info/forums/?page=post&id=51C5DE97-2D0E-40E9-BFF7-7F7FE30E18FE

[5] https://www.openbsd.org/faq/faq16.html

[6] http://www.netzbasis.de/openbsd/vmd-debian/index.html

[7] https://www.openbsd.org/papers/asiabsdcon2018-vmm-slides.pdf