8

The Arch Linux ZFS wiki page explains grub-compatible pool creation, as does this page about booting Fedora, but I have not been able to create a pool that is readable by Grub. The Arch Linux wiki page about Installing Arch Linux on ZFS highlights certain bugs but doesn't really explain how to overcome them.

The linked pages explain that Grub supports a subset of zpool features and cannot read a pool that uses features that it doesn't support. They go on to explain how to configure a suitable pool but I have been unable to make it work. The supported feature subset does not appear to be documented anywhere.

I am using a virtual machine to test with Grub 2.02 and Arch Linux kernel 4.16.13-1-ARCH which is the most recent and is compatible with the current zfs-linux package version (zfs-linux-0.7.9.4.16.13.1-1). I am not (yet) trying to make a bootable system, only to prove that Grub can read the zpool. Here is what I have tried:

First, like the arch wiki page suggests, by disabling unwanted features:

# zpool create \ -o feature@multi_vdev_crash_dump=disabled \ -o feature@large_dnode=disabled \ -o feature@sha512=disabled \ -o feature@skein=disabled \ -o feature@edonr=disabled \ testpool mirror \ /dev/disk/by-id/ata-VBOX_HARDDISK_VB{5f2d4170-647f16b7,f38966d8-57bff7df}

which results in these features:

testpool feature@async_destroy enabled local testpool feature@empty_bpobj active local testpool feature@lz4_compress active local testpool feature@multi_vdev_crash_dump disabled local testpool feature@spacemap_histogram active local testpool feature@enabled_txg active local testpool feature@hole_birth active local testpool feature@extensible_dataset active local testpool feature@embedded_data active local testpool feature@bookmarks enabled local testpool feature@filesystem_limits enabled local testpool feature@large_blocks enabled local testpool feature@large_dnode disabled local testpool feature@sha512 disabled local testpool feature@skein disabled local testpool feature@edonr disabled local testpool feature@userobj_accounting active local

Then, like the fedora example, by enabling wanted features:

zpool create -d \ -o feature@async_destroy=enabled \ -o feature@empty_bpobj=enabled \ -o feature@spacemap_histogram=enabled \ -o feature@enabled_txg=enabled \ -o feature@hole_birth=enabled \ -o feature@bookmarks=enabled \ -o feature@embedded_data=enabled \ -o feature@large_blocks=enabled \ testpool mirror \ /dev/disk/by-id/ata-VBOX_HARDDISK_VB{5f2d4170-647f16b7,f38966d8-57bff7df}

which results in these features:

# zpool get all testpool | grep feature testpool feature@async_destroy enabled local testpool feature@empty_bpobj active local testpool feature@lz4_compress disabled local testpool feature@multi_vdev_crash_dump disabled local testpool feature@spacemap_histogram active local testpool feature@enabled_txg active local testpool feature@hole_birth active local testpool feature@extensible_dataset enabled local testpool feature@embedded_data active local testpool feature@bookmarks enabled local testpool feature@filesystem_limits disabled local testpool feature@large_blocks enabled local testpool feature@large_dnode disabled local testpool feature@sha512 disabled local testpool feature@skein disabled local testpool feature@edonr disabled local testpool feature@userobj_accounting disabled local

In each case, I loaded some content:

# cp -a /boot /testpool

And then, rebooted into Grub:

grub> search --set --label testpool grub> ls / @/ grub> ls /@ error: compression algorithm 80 not supported . grub> ls /@/ error: compression algorithm inherit not supported .

I tried enabling/disabling some features, most notably lz4_compress. I also tried creating a dataset on the pool. Nothing I tried worked inside Grub.

I expected to be able to list /boot or /@/boot.

Errors encountered include

  • compression algorithm inherit not supported
  • compression algorithm 66 not supported
  • compression algorithm 80 not supported
  • incorrect dnode type

How should a ZFS zpool be created in order for it to be readable by Grub?

Improve this question

2 Answers 2

Reset to default
6

Grub cannot reliably perform a directory listing of a zpool due to a bug as confirmed via the mailing list:

listing directory contents in Grub is broken, I have a patch lying around that fixes that specific issue. if you get strange error messages (e.g., something like invalid BP type or compression algorithm), it's likely this issue.

This problem is present in Grub installed from ArchLinux and also Fedora 28. Ubuntu, however, appears to have patched their Grub to fix this (confirmed with Ubuntu 16.10).

Grub can, however, read the files required to boot. You can create a zpool that Grub will boot like this:

zpool create -m none "$ZPOOL" "$RAIDZ" "${DISKS[@]}"

where the variables define the name of the pool, e.g mypool, the raid level, e.g. mirror and the disks, e.g /dev/disk/by-id/... (two disks being required for a mirror).

You need to create a dataset

zfs create -p "$ZPOOL"/ROOT/archlinux

And you need to set the dataset's mount point:

zfs set mountpoint=/ "$ZPOOL"/ROOT/archlinux

You can then boot it with Grub using commands like:

insmod part_gpt search --set --label mypool linux /ROOT/archlinux@/boot/vmlinuz-linux zfs=mypool rw initrd /ROOT/archlinux@/boot/initramfs-linux.img boot

I scripted this to test it in a VirtualBox machine. I installed Arch from the ISO onto a normal ext4 root and then used that to install a new ZFS root onto two mirrored virtual disks. The above is a summary - see the script for full details.

Improve this answer
6
  • Dunnow how you got this woirking, but on recent centos grub2-probe / fails to detect zfs when all of the features are enabled for a pool. It even states the exact feature it stumbles on (and zfs detection fails after that) and then, sure, it cannot boot when asked. This is an interesting post, but all the grub directives you're providing are pretty much regular ones required for a boot from zfs. Commented Jan 31, 2019 at 19:22
  • And yeah - from my opinion - this has nothing to do with ability to actually list zpool/zfs contents from grub [rescue shell]. Commented Jan 31, 2019 at 19:25
  • @drookie my systems work with all listed features showing as either enabled or active (the listed features being those identified in the question). Nothing is disabled. I don't use grub-probe though and I hand roll my own grub.cfg. Commented Feb 1, 2019 at 10:26
  • Could you please show your grub version ? Commented Feb 1, 2019 at 16:52
  • 2
    Sure, @drookie, my Grub version shows as 2.03 on the Grub screen. I did build it myself from source because I also use these crypto extensions, I specifically built from this tag (not that that would make any difference to the ZFS support). Commented Feb 3, 2019 at 19:23
1

The supported feature subset does not appear to be documented anywhere.

The only reliable place where I could find the list of features supported is GRUB's source code (permalink to today's version). Basically all write-only features plus those explicitly listed there.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.