Why I can't boot my virtual machines on FreeNAS after installing them?

I'm using FreeNAS-11.3-U2. I tried to install Archlinux Virtual machines on my it 3 times. When I reboot after installation is completed, I can boot into the virtual machine, but everytime I restart the virtual machines, I could only see the "UEFI Interactive Shell". Someone else experienced this?

2 Answers

Apparently the EFI implementation lacks persistent loader storage. It's a known limitation of eg. VirtualBox, but I'm not sure if that's what FreeNAS uses under the hood. Anyway, OS's loader is configured during installation, but then EFI "forgets" about it and falls back to EFI shell.

You may be able to work around it by installing rEFInd or similar bootloader and making sure its executable on EFI System Partition is called EFI\BOOT\BOOTX64.EFI. This is a default loader path for x64 systems when no loader is saved in EFI.

You could also fiddle with kernel's EFI stub and try to place kernel itself under that path. initramfs needs to go there too. Updating will be tricky, I'll leave figuring it out (yourself or with Arch wiki's help) as an excercise to you.

Third option is to create an EFI shell script startup.nsh which boots appropriate loader. Shell will interpret this script on boot unless interrupted.

Q: "Someone else experienced this?"

A: Yes. I did. In TrueNAS-12.0-U2.1 CORE, TrueNAS VMs use the bhyve virtual machine software. The UEFI interactive shell started when the installation media wasn't found. I've put the installation media into the directory on the system disk

/export/ISO/images/freebsd/12.2/FreeBSD-12.2-RELEASE-amd64-dvd1.iso

and copy&paste this path into the configuration of the VM. This is not the proper way, apparently. The path to the installation media should be available in the clicky menu of the configuration window. But, the system disk, i.e. the directory /export in my example, is not available in the clicky menu. Only /mnt and the selected ZFS pool are.

The problem was solved when I created a new ZFS dataset in the selected pool, e.g

pool1/ISO 1047828875 3881541 1043947334 0% /mnt/pool1/ISO

and put the installation media into this directory, e.g.

/mnt/pool1/ISO/FreeBSD-12.2-RELEASE-amd64-dvd1.iso

Then, this path to the ISO was available in the clicky menu and, when selected, the VM booted it. Among the VM devices, it's necessary to change device order and boot from the CD, of course (change it to disk after the installation).