Merge ~juliank/grub/+git/ubuntu:resilient-boot into ~ubuntu-core-dev/grub/+git/ubuntu:ubuntu

FWIW, I followed os-prober's design on where to mount the ESPs and mounted them to /var/lib/grub/esp, os-prober was doing sth similar in /var/lib/os-prober.

I'm happy now, please review.

Use cases:

* New install:
* Upgrade:

If you have not preseeded grub2/efi_install_devices, we'll transition the device that is currently mounted to /boot/efi into the setting, so that you don't get asked. Doing this on install also keeps existing installers working, they can then add the debconf magic to get resilient boot support.

* User changed /boot/efi to point elsewhere not yet configured as an ESP

We take the current debconf settings, append the device that /boot/efi points to to the list of enabled things, and then prompt the user if that's correct.

* User unmounted /boot/efi

If the user unmounted /boot/efi during upgrade/install, they'll get prompted for which ESPs to install. If the user unmounted /boot/efi otherwise, the device that was mounted there will be mounted to /var/lib/grub/esp as for any other ESP.

* User does not have an ESP-type partition, or the wanted one does not have an ESP identifier

This might happen if the system supports booting from any FAT partition, not just those with magic ESP identifiers. Our installer does not create such partitions, though. The user will see an empty list, or well, at least not the partition they want.

* User mounted sth to /boot/efi that is not an ESP

Similar to before, except the user will get prompted on each upgrade.

Maybe I should open a FFe bug for this, but it's also a kind of bug fix?

@raharper could use some review of the debconf keys. The logic for installers is to continue to mount one ESP to /boot/efi, and add all other disks using their /dev/disk/by-id/ alias to the grub2/efi_install_devices key.

I see that you commented in the spec that you were moving the invocation of multiple targets into a wrapper script instead of modifying the C code. FTR this comment did not result in any notifications to me.

The problem with doing this in a wrapper script, and why I originally insisted on this being done in grub-install, is that grub-install is also the tool that on EFI systems populates the BootXXXX and BootOrder nvram variables. If we are not passing an ordered list of target devices to grub-install, then it does not know what to do with the nvram contents wrt recording primary and fallback options. So doing this in a wrapper script cannot actually provide the bootloader resiliency that we are looking for here.

Now the thing is that you cannot specify an order in debconf prompts either, so while the installer can do such a thing, and while you can manually set the selection, as soon as you get reprompted because you changed your ESPs, the order will be changed anyway (I believe it's alphabetically by device id).

The only guarantee we could make is that you can mount /boot/efi to what your preferred ESP is, and otherwise it's going to be hell.

But looking at the code, it actually seems to delete any other boot entry for ubuntu, so that's not really helpful. This confuses me, because I'd expect to be able to install two instances of ubuntu on two hard disks and not have them constantly delete each other's bootloader entries.

In any case we still need the wrapper script - only the block that runs grub-install needs to be changed to pass the devices to grub-install.

However, there is another problem with passing a list of devices to grub-install: We can't tell you which device failed to install, so if we reprompt you on a failure, you won't know what you need to do. That's why grub-pc runs them in a loop too.

I'm not sure how to proceed. Modifying grub-install to take a list of devices and loop over them and install to all of them seems out of the question, as it has a lot of global state that needs to be stored in a per-device array, function signatures need to be changed to accept arrays of new structs instead of multiple arguments, and getting this sorted out before focal does not seem feasible.

So I was thinking we could pass it a list of all ESPs for each invocation, and then we only remove entries that do not belong to any of those ESPs. Then each grub-install invocation will only change it's own boot entry, essentially. That should be strictly a subset of the other work, and hence have a short turnaround time.

Still does not solve that you don't have an *ordered* list in the debconf entry (multiselect entries are not orderable by the user). But I guess we can give it a distinction between primary and fallback ESPs at the very least, if /boot/efi is mounted.

I'm also confused about the previous check to EFI/ubuntu exists on the ESP, because I want people to use the grub-multi-install wrapper to do the initial installation too, until we have this sorted out properly in grub-install (which let's face it, is a lot of work).

Because I can add a hack to grub-install now reasonably quickly, but I don't see that getting a clean solution, because that's touching a lot of places, and that's (a) a lot of work and (b) a huge effort to maintain downstream.

I made it super ugly now, but this seems like it works reasonably OK. It just is undefined which ESP is first in the boot order. OK, to be fair, the last entry it adds will be the first one, but don't have to tell people that...

I can start making it nice, but it's good to have a fallback that's 90% there but super ugly.

Needs

* Rebasing against didrock's changes
* Reinstate the check that we installed grub before / should be installing grub, but this time

cjwatson | Testing for /boot/grub/@FIRST_CPU_PLATFORM@/core.efi might work

Thanks, a few minor things that I need fixing up before landing.

Upstreaming is a whole other story and I haven't reviewed the changes for that in mind.

Replied to diff comments.

Re upstreaming - that's not even really on the radar. Not even the efivar.c bit we have from Debian is upstreamed yet - upstream uses efibootmgr still :/