Unable to boot ZFS system

[Rajil Saraswat]

Hello,

I have a Gentoo based system running on ZFS. At the moment I have Grub working with a entry like so,

linux   /@/vmlinuz-4.9.95-gentoo root=ZFS=rpool/ROOT/gentoo ro spl.spl_hostid=0xa8c06f01 console=tty0 console=ttyS0,115200
echo    'Loading initial ramdisk ...'
initrd  /@/initramfs-4.9.95-gentoo.img

I install refind and modified /boot/refind_linux.conf as

"Boot with standard options"  "zfs=rpool/ROOT/gentoo ro spl.spl_hostid=0xa8c06f01 console=tty0 console=ttyS0,115200 initrd=/boot/initramfs-4.9.95-gentoo.img"
"Boot to single-user mode"    "zfs=rpool/ROOT/gentoo ro spl.spl_hostid=0xa8c06f01 console=tty0 console=ttyS0,115200 single"
"Boot with minimal options"   "ro root="

rEFInd shows a boot entry to me for initramfs-4.9.95-gentoo.efi, says Starting vmlinuz-4.9.95-gentoo.efi but then freezes the system.

Does rEFInd work with ZFS and is my syntac for /boot/refind_linux.conf correct?

Thanks

[Godotless]

Works for me:

$ cat /boot/refind_linux.conf 
"Boot with standard options"  "dozfs=force root=ZFS=SSD/ROOT/Debian rw quiet splash vt.handoff=7"
"Boot to single-user mode"    "dozfs=force root=ZFS=SSD/ROOT/Debian rw quiet splash vt.handoff=7 single"
"Boot with minimal options"   "dozfs=force root=ZFS=SSD/ROOT/Debian rw"

or in refind.conf manual stanza

menuentry "Akua Debian" {
        icon EFI/refind/icons/os_elementary.png
        volume ZFSROOT
        loader ROOT/Debian/@/vmlinuz
        initrd ROOT/Debian/@/initrd.img
        options "dozfs=force root=ZFS=SSD/ROOT/Debian rw quiet splash vt.handoff=7"
}

I had to fiddle with the paths - since the actual root is ROOT/Debian ... but it wouldn't work if I didn't add the /@/ path component. Those paths are symlinks into boot/ (latest version of kernel and initrd).

zfs list
...
SSD/ROOT                      30.7G  61.7G    23K  none
SSD/ROOT/Debian               11.5G  61.7G  11.5G  legacy
SSD/ROOT/FreeBSD               949M  61.7G   949M  /mnt/FreeBSD
SSD/ROOT/Void                 18.2G  61.7G  18.2G  legacy
...
 # ls -l /boot/efi/EFI/refind/drivers_x64/
total 239
-rwx------ 1 root root 83222 Sep 16 18:13 btrfs_x64.efi
-rwx------ 1 root root 69384 Sep 17 09:41 ext4_x64.efi
-rwx------ 1 root root 90336 Apr 25  2018 zfs_x64.efi

Zpool's name is SSD ... ZFSROOT is the partition name set with gdisk (gptfdisk).

Hope this helps.

[Roderick W. Smith]

You don't say where your kernel and initrd file are located. The use of /@/ in file paths noted by Godotless is required for some configurations that use Btrfs, and might also be needed if the kernel and initrd are on a ZFS partition and you're using a third-party ZFS driver; however, I've never tried putting a Linux kernel and initrd on ZFS, so I can't be certain of that.

Another point is that your configuration is explicitly naming the initrd file in the refind_linux.conf file. This is inadvisable when the initrd file's name is likely to change, as it seems to be for you. (Your initrd filename includes a version number, which implies the kernel and initrd filenames are linked by filename.) When using kernel auto-detection, rEFInd matches the kernel and initrd file automatically, and creates the initrd={filename} option based on the matches it finds. Thus, you don't normally need to include an initrd={filename} option in refind_linux.conf. In fact, adding that option can create problems, since if you get it wrong, the computer may hang when booting. (The kernel EFI stub loader does essentially no error recovery, so when something goes wrong, it hangs.) Note that even a trivial typo in the filename can cause this sort of hang, which is why manual boot stanzas can be so tricky to get working.

[Toan Pham]

however, I've never tried putting a Linux kernel and initrd on ZFS, so I can't be certain of that.

Roderick, my kernel and initrd are on zfs's root filesystem. Do you have plans for refind to automatically search for all zfs datasets for valid kernel and initrd boot options? I am using zfs 100% of the time for linux and I am curious the best way to incoorporate auto detection with Refind! So far, the best approach is that I have a special script that generates a bootable kernel stub (with kernel, initrd and boot params), and stores that stub in ESP partition @ /efi/zfsboot/kernel.efi. This is a non conventional way, and is not pratical. Please advise if it is possible for Refind to search within all zfs datasets for possible boot options.

thank you,

tp

[Roderick W. Smith]

rEFInd currently scans all the filesystems it can read for EFI binaries (including kernels with the EFI stub loader), assuming the filesystems aren't omitted based on the scanfor line in refind.conf. There are two potential pitfalls:

• No ZFS driver comes with rEFInd, so you'd need to rely on a third-party driver, such Pete Batard's EFI driver set.
• rEFInd doesn't scan the whole filesystem, just certain directories, such as /boot and most subdirectories (to one level) of /EFI. If, from the EFI's point of view, your kernels exist in /@/boot or some other unusual location, you'll probably have to add them using also_scan_dirs in refind.conf.