Experiment: Specify external GRUB for chainloading

[stevefan1999-personal]

This is a spun-off from #4041 (comment). From the current partition scheme of Talos:

Talos uses these partitions with the following labels:

1. EFI - stores EFI boot data.
2. BIOS - used for GRUB’s second stage boot.
3. BOOT - used for the boot loader, stores initramfs and kernel data.
4. META - stores metadata about the talos node, such as node id’s.
5. STATE - stores machine configuration, node identity data for cluster discovery and KubeSpan info
6. EPHEMERAL - stores ephemeral state information, mounted at /var

While we are obviously not going to fix #6325 soon, so we have to retain the current structure. But what if we let the user designate a tiny partition to store a first stage GRUB? I think 4 to 16MB is usually enough, so that we can load the LVM GRUB modules from the first stage and then chainload to the second stage bootloader inside the LVM. It might not necessarily be LVM, as ZFS and other volume virtualization tech may also apply here, LVM is the only sane choice for GRUB to chainload to.

What it effectively means is something like this:

1. CHAIN - a small, specialized partition to install GRUB and allows chainloading to Talos, which in such an extreme use-case, resided on a LVM volume. Technically this should NOT be touched by any others after creation as its sole purpose is to just chainload into Talos's own bootloader with extra sauces.

2.1. EFI - stores EFI boot data.
2.2. BIOS - used for GRUB’s second stage boot.
2.3. BOOT - used for the boot loader, stores initramfs and kernel data.
2.4. META - stores metadata about the talos node, such as node id’s.
2.5. STATE - stores machine configuration, node identity data for cluster discovery and KubeSpan info
2.6. EPHEMERAL - stores ephemeral state information, mounted at /var

3. DATA - anything else out of Talos's control

I think this would be a pretty interesting experiment for the Talos team to invest time with. Not that this would not need so much more code, would not have to destroy existing partition scheme (since LVM virtualized the disk concept to begin with), have little to no side effect for existing users since it is recursively backward compatible (maybe add a little bit of bootloading time, but Talos is already fast), while letting new user to test out a new way of installing Talos, and to solving #4041, #6325, and #6325 altogether.

It is also possible to modify existing Talos installation to add the new chainload partition, as we could nicely cut out 16MB off the EFI partition and install enough LVM stuff to make it a first stage GRUB bootloader.

[stevefan1999-personal]

We are so close...

(base) PS F:\talos\lvm-test> talosctl -n 192.168.28.96 disks --insecure
DEV        MODEL          SERIAL   TYPE   UUID   WWID                                   MODALIAS      NAME   SIZE     BUS_PATH                                                                                                        SUBSYSTEM          SYSTEM_DISK
/dev/sda   Virtual Disk   -        HDD    -      naa.600224804258e053204016d36f055afc   scsi:t-0x00   -      136 GB   /LNXSYSTM:00/LNXSYBUS:00/ACPI0004:00/VMBUS:00/3b9c49e7-b137-421a-bb87-025a9b7f151a/host0/target0:0:0/0:0:0:0/   /sys/class/block   
(base) PS F:\talos\lvm-test> talosctl apply-config --insecure -n 192.168.28.96 --file controlplane.yaml
error applying new configuration: rpc error: code = InvalidArgument desc = configuration validation failed: 1 error occurred:
        * specified install disk does not exist: "/dev/mapper/talos-lvol0"
        *

Maybe we should not load from /sys/class/block from then on.

I'm not really installing a specialized GRUB loader just yet. I'm using SystemRescue that has its own GRUB on the ISO that can scan onto others. Technically speaking you can use another drive (such as flash card or a pen drive) to serve as that chainloader as well

Sidenote: I just reckon the Xpenology project actually did that to some extent -- at least ARPL did something similar

[stevefan1999-personal]

Oh look:

https://github.com/siderolabs/go-blockdevice/blob/d9313ea908f3790dba29f7432501335825c863ff/blockdevice/util/disk/disks.go#L114-L120

Maybe we can hack together a kernel parameter to skip it from being filtered...See this