Running a Q35 Virtual Machine Is, In Fact, Possible Under TrueNAS SCALE

What an adventure this was.

The context here is interesting, to say the least. I have a friend who needed to run VMware ESXi for school, but didn't have the immediate hardware available. The compromise was to virtualize ESXi on a bare metal TrueNAS SCALE server.

Why not just install ESXi bare metal, you ask? Well, the CPU is a Ryzen 5 5600G, and we were looking to utilize the Radeon integrated graphics for hardware transcoding, and didn't want to deal with the likely-complicated GPU passthrough it would involve. So, nested hypervisors it was.

We tested two different versions of ESXi, version 6.7 and 7.0. We started by creating our VMs the normal way using the built-in TrueNAS web GUI. ESXi version 6.7 would kernel panic during initialization; probably because the machine type was i440fx and not Q35. Q35 is crucial because it is a more modern machine type and supports nested virtualization. Version 7.0 did not kernel panic, but it complained that there was no supported network interface card. There is no support for VirtIO or e1000 drivers in this version of ESXi.

This is where the limitations of TrueNAS SCALE started to creep up. Unlike Proxmox, KVM/QEMU in TrueNAS SCALE does not have a option to choose Q35 as a machine type, i440fx is the only option. Secondly, there was no choice for an emulated Realtek NIC or even vmxnet3, VMware's paravirtualized NIC they use in their hypervisors. Luckily, these limitations are only a problem in the web GUI.

So, where do we go from here? After hours of research, I learned that every action in TrueNAS SCALE is controlled via the TrueNAS Websocket API. This API is also accessible via the command line. This allowed me to go under the hood and figure out how to manually create a q35-based virtual machine. I also got help from virsh to make a network interface supported by VMware's trademark hypervisor.

Here is how I did it.

1. You will need to create a new virtual machine from the command line. Open the TrueNAS API CLI by running cli in the TrueNAS root shell and run the following command:

service vm create cpu_mode="HOST-PASSTHROUGH" cpu_model=null name="esxi" machine_type="q35" memory=1024 arch_type=x86_64

Then exit the CLI with exit.

2. Open up the web GUI for TrueNAS and navigate to the VM "esxi" under Virtualization.
   Edit the CPU cores, threads, and memory to be something reasonable and save the changes.
   Next, click on "Devices".
   Add a DISPLAY device and make it 1024x768.
   Add a CDROM and point it to the path where the ESXi installer ISO is located.
   Add a DISK for the OS installation and make it 32GB.
   Add a second DISK as big as you need for your ESXi storage pool. Make sure both disks are AHCI.
   Lastly, add a NIC with the adapter type e1000. We won't be using it, but we have to have one so QEMU can do the witchcraft that makes attaching a NIC possible in the first place.
   Write down the name of your TrueNAS host's NIC in the field "Nic to attach" (It will be something like enp6s18). We will need it for step 5.
3. Go back to the shell and create an alias for virsh so you can easily connect to the QEMU libvirt socket.

alias virsh='virsh -c "qemu+unix:///system?socket=/run/truenas_libvirt/libvirt-sock" $1'

I recommend including the alias in your .zshrc or aliases file for future convenience. The next command will do it for you automagically.

echo $'alias virsh=\'virsh -c "qemu+unix:///system?socket=/run/truenas_libvirt/libvirt-sock" $1\'' >> ~/.zshrc

4. List all the virtual machines with virsh list --all and write down the name of the virtual machine we created. It will be something like 1_esxi. Virsh also likes to call this the "domain".
5. Start the virtual machine via web GUI, or alternatively run midclt call vm.start {id} where {id} is the number that's in front of _esxi.
   Quickly attach a new vmxnet3 network interface to the virtual machine with the below command before ESXi loads all the way. Replace {VM name} and {host NIC} with that you recorded in steps 2 and 4.

virsh attach-interface --domain {VM name} --source {host NIC} --type direct --model vmxnet3 --mac 1a:22:33:44:55:66 --config --live

It should return with "Interface attached successfully." You can also double check and list attached network interfaces with virsh domiflist {VM name}. You should see the vmxnet3 and e1000 devices on the list.

Voila, now you have a nested hypervisor that's running on supported virtual hardware! There is only one drawback: The vmxnet3 NIC does not persist between shutdowns of the virtual machine. If you shut down the VM, you will have to repeat step 5 every time you boot it back up. You may be able to automate re-adding the NIC with a script or alias, but I did not try that. You could also try to install the drivers for the e1000 nic, but I have not tested that.

The most ironic part of this story is that we couldn't use the Radeon graphics for video transcoding, anyway. C'est la vie!