Setting up a device with illumos (OmniOS)

While working at Oxide Computer Company, I came across the need to have an illumos distribution installed on a "real device" to be able to contribute to propolis, part of the bhyve hypervisor (as nested virtualization isn't yet supported).

For folks who have previously worked on illumos, that intro probably makes perfect sense, but for everyone else: basically, I needed to install illumos on physical hardware! Regardless of your reasons, this guide walks you through:

- Downloading an illumos distribution (in my case, OmniOS)

- Setting up networking, users, and ssh access for development

Since many (most? all?) illumos distributions don't yet support cross-compilation, this would also function as a reasonable "getting started" guide for bootstrapping development on illumos itself.

Prerequisites

This guide assumes you have access to the following:

- An x86-64 machine, onto which you want to install an illumos distro

- A USB stick (at least a couple GB, to hold the OS image)

Downloading illumos

Although the illumos docs link to a list of many possible distributions, today, we'll be sticking with OmniOS.

First, you'll want to download an image of OmniOS. You can pick which variant you want (current, LTS, bloody), which allows users to make a tradeoff between "recency" and "stability". In my case, I chose the "Current Stable" release, downloading the corresponding usb-dd file.

Once the file was downloaded, I connected a USB stick to my machine, identified the name of my USB device, and copied it there. This can be done in a few steps:

1) Identify the name of your USB stick. The easiest way to do this most Unix-y systems is the lsblk command, which prints out the name and mountpoint of all devices. In my case, this device showed up as "sdb", which meant it existed in my filesystem as "/dev/sdb".

Note: If these devices have a number after their name (like "sdb2"), this refers to a partition within the device, not the whole device. Don't use this value! Instead, plan to refer to the device itself in the commands below.

2) Make sure that the USB stick isn't mounted. lsblk has a "MOUNTPOINT" column which indicates where in the filesystem the device is mounted, if any. Running "$ umount <MOUNTPOINT>" will put the device in a state where it may be overwritten.

3) Copy the OS image to your USB stick. This can be done with the following command:

# On my machine, this was:

# dd if=~/Downloads/omniosce-r151036m.usb-dd of=/dev/sdb

$ dd if=~/Downloads/{downloaded-image-file}.usb-dd of=/dev/{USB device}"

If you've gotten this far, you have a bootable USB stick, ready to run illumos.

Running the Installer

After prepping this USB stick, you can plug it into your target machine (on which you intend to install illumos) and reboot it. You'll need to access the boot menu for your device to instruct it to boot from the USB stick, which is specific to your device - this likely means hitting a key such as F12 while powering on.

Once you've booted, the system should automatically enter an installer, where you can select your devices, hostname, and timezone. These steps are fairly self-explanatory, with "enter + arrow keys" to control (although, since it confused me, I'll mention explicitly: Use space-bar to toggle checkboxes during device storage selection). 

After following those instruction steps, you should be able to unplug the USB stick, reboot, and automatically start illumos. If successful, you should end up at a root shell (default root password is empty, so you can just press "Enter" to log in).

A Note About Services

illumos controls most of the system using the concept of SMF services, which are visible with the svcs command. If you bump into issues, I recommend using the following commands to observe the system and ensure the prerequisite services are up and running:

# Observe all running services

# Pipe to grep to search for a specific service, like "dns".

$ svcs -a

# List more information about a service ("what is it, what does it depend on, etc").

$ svcs -l <name of service>

# For any services which are "disabled" or "offline", try to turn them on.

$ svcadm enable <name of service>

# The magic incantation to "turn it off and back on again".

$ svcadm restart <name of service>

Add a User, Set Passwords

First things first, we can add a new user to the system, using the command useradd. 

The simplest form of this command is just:

$ useradd -m <your user name>

Where the "-m" flag requests that a new home directory also be created for this user.

Now that you've created yourself, you can also set a password:

$ passwd <your user name>

And also set one for root:

$ passwd root

To perform privileged operations, you can login as root or use sudo, although pfexec is the preferred way to run privileged commands on illumos.

Networking

Let's get connected to the network! (these steps are loosely taken from the OmniOS general administration guide)

1) Identify your networking interfaces. "dladm", or the "data link administration" tool is the primary way we can do this. For now, we'll just look at the physical devices that exist on this machine:

$ dladm show-phys

LINK        MEDIA       STATE     ...

e1000g0     Ethernet    Unknown   ...

So in this case, we have an Ethernet data link, though it doesn't seem set up yet.

2) Create an IP interface for that device. "ipadm", or the "IP administration" tool is how we'll set this one up. First, we'll create an IP interface on top the data link device, then we'll give it an IP address.

# For me, this was: ipadm create-if e1000g0.

$ ipadm create-if <device>

3) Give it an address. You can either set the device up with "dynamic network" (DHCP) or a "static network", depending on your environment. DHCP means you'll avoid colliding with other IP addresses on the same network, but your IP address will periodically change. In contrast, with a static address, your IP address will be stable, but you'll need to check that you avoid collision.

This can be executed with ONE of the following commands:

# The DHCP way:

$ ipadm create-addr -T dhcp e1000g0/v4

# The static way:

$ ipadm create-addr -T static -a 192.168.1.10/24 e1000g0/v4static

You should be able to observe the new address with:

$ ipadm show-addr

Which should show your IP address under the "ADDR" column.

4) Set up DNS resolution.

At this point, you'll probably notice that commands like ping 8.8.8.8 are working correctly, but pings which require name resolution (like ping google.com) aren't. To resolve this, we need to instruct the device how to translate these human-readable names into IP addresses by adding a nameserver to the resolv.conf file.

You can either supply a nameserver of your own choice (like 8.8.8.8, owned by Google), or you can ask your router to figure it out for you, by supplying "192.168.1.1". We'll do the latter:

# Add the nameserver to the DNS resolver's config file.

$ echo 'nameserver 192.168.1.1' >> /etc/resolv.conf

# Make a backup of your current "/etc/nsswitch.conf" file (optional).

$ cp /etc/nsswitch.conf{,.bak}

# Overwrite the nsswitch.conf file with nsswitch.dns, the sample

# DNS configuration file. 

$ cp /etc/nsswitch.{dns,conf}

At this point, you should be good to go - although I'd recommend rebooting the device with "reboot" to check that your changes here are persistent.

SSH

After setting up networking and a user, you should be able to ssh into the machine by IP address, like the following:

smklein@linux:~$ ssh <username>@<IP Address>

For development purposes, this is major progress! We'll go through a couple more steps to make this more ergonomic for development on a local network, but these are optional.

Stop hard-coding IP addresses

Within a local network, Multicast DNS (or mDNS) is a mechanism for resolving local hostnames to IP addresses. If you followed the steps for giving your illumos device a DHCP address, using mDNS is a great way to keep ssh-ing into the device with the same name, even though the underlying IP may change.

Let's observe what dns services are enabled on the illumos system:

smklein@omnios:~$ svcs -a | grep dns

disabled    17:29:15 svc:/network/dns/install:default

disabled    17:51:43 svc:/network/dns/multicast:default

online      17:29:17 svc:/network/dns/client:default

As we can see, the multicast DNS service is not yet enabled. Let's turn it on:

smklein@omnios:~$ svcadm enable /network/dns/multicast

Now, on a different device, we can try accessing the device with "hostname.local" instead of the IP address:

smklein@linux:~$ ssh smklein@omnios.local

Stop entering passwords

ssh-keygen is a handy tool to automate login across machines, we can give it a shot on our host machine (referred to as "linux" in my case).

# Generate a keypair.

smklein@linux:~$ ssh-keygen -t rsa

# Ensure a ".ssh" directory exists on the illumos machine.

smklein@linux:~$ ssh smklein@omnios.local mkdir -p .ssh

# Copy the public key to your illumos machine

smklein@linux:~$ cat ~/.ssh/id_rsa.pub | ssh smklein@omnios.local 'cat >> .ssh/authorized_keys'

Afterwards, you should be able to ssh without a password.

Troubleshooting

If this still doesn't work, try adjusting access to the necessary ssh directories on your illumos machine:

smklein@omnios:~$ chmod 700 .ssh && chmod 640 .ssh/authorized_keys

You did it!

Congrats, you have an illumos machine up and running, ready for development.