Using Ubuntu Live Disk with rEFInd.

[JH Park]

Hello.
I must say thank you, Roderick W. Smith, for your great effort.
With rEFInd, my Linux life is better than ever.

Here's what I want to do.
I want to make a Ubuntu Live Installation flash drive(Using Ubuntu 17.10 beta ISO image), using rEFInd. Actually, I want several images(like Kubuntu, Ubuntu Budgie, etc) all together on this flash drive.

I formatted a flash drive like this.

sdc                                                          
├─sdc1 vfat   ESP_MULTI 70D8-21C9  
├─sdc2 ext4   UbuntuExt e08985be-b761-44e5-aacc-1c30597fb9fa
└─sdc3 ext4   KUBUNTU  418756d2-a70c-4a33-9f4f-d7f079534513

I installed rEFInd to sdc1, and it was successful.
Then I extracted Ubuntu ISO to sdc2.

mkdir /tmp/install_cd
mkdir /tmp/installer

sudo mount ubuntu-17.10-beta2-desktop-amd64.iso -o loop /tmp/install_cd
sudo mount /dev/sdc2 /tmp/installer

sudo rsync -a /tmp/install_cd/* /tmp/installer

sudo umount /tmp/install_cd
sudo umount /tmp/installer

After that, I rebooted, and rEFInd could find Ubuntu Live disk on UbuntuExt volume.
I chose it, and I chose "Try Ubuntu ...".

But I falied to boot to the live session, and I've got this error message.

"mounting /dev/loop0 on //filesystem.squashfs failed no such device"

I tried some options as far as I know, but could not succeed.
Can you help me to figure this out?

Thanks for reading.

[Roderick W. Smith]

I've done what you're attempting, but in a slightly different way:

1. Partition the USB drive with an ESP and separate partitions for each .iso file you want to use. Make the partitions for the .iso file at least as large as the files themselves.
2. Put rEFInd on the USB drive's ESP, using the fallback filename. I did this manually, but using refind-install --usedefault /dev/sdb1 (or whatever the ESP's device file is) should work, too.
3. Copy the ISO-9660 driver (iso9660_x64.efi, assuming this is for an x86-64 system) to the EFI/BOOT/drivers or EFI/BOOT/drivers_x64 subdirectory of the USB drive's ESP.
4. Use dd to copy each .iso file to its matching partition on the USB drive. For instance, sudo dd if=image.iso of=/dev/sdb2.

When you boot the USB drive, rEFInd will be able to read the ISO-9660 filesystems and launch the EFI boot loaders they contain. This works fine for me for several Linux distributions I've tried (mainly Ubuntu and its variants, but I'm pretty sure I've done this with OpenSUSE and/or Fedora or CentOS). Some distributions might not work, though. This approach does not work with the Microsoft Windows installer I tried.

The biggest problem with this approach, in my experience, is that managing the OS installers is difficult; if you no longer need one and so delete it, but want to put in a slightly larger one in its place, you won't be able to easily re-use the space, since you'd need to move one or more partitions to make the freed space larger. GParted won't move ISO-9660 images (or at least, not the Frankenstein's Monster ISO-9660 format used by most Linux installers); and even if it would, the time taken for the move would be significant.

Another caveat is that, if you want to support Secure Boot, you'll need to add Shim or PreLoader to the mix, and you'll have to add appropriate keys to each target computer's MOK on a boot-by-boot basis. This would be true of any use of rEFInd for this purpose, of course.

Despite these drawbacks, it can be handy to have a bunch of installers on one medium. If it's big enough, you can simply put new .iso images later on the disk, until you eventually free up enough contiguous space to add new images earlier. Careful planning (like putting LTS images at the start or end of the disk) can help, too.

[Mikhail Morfikov]

The instructions to use separate partitions for each iso file works well, but only in the case of ubuntu live images. I also have some debian live images and in their case the situation is a little bit different.

I have Secure Boot enabled with my own keys configured, I signed the rEFInd binary on the pendrive's ESP partition as well as the iso driver. I also added the debian and canonical certs to the shim's MOK database via mokutil.

When I boot from the pendrive, it shows me the OS menu and I can choose which iso file to boot. I currently have 4 partitions for the iso images -- 2 for ubuntu (lts+lates) and 2 for debian (gnome+mate). The ubuntu images boot without problem, but when I choose any of the debian image, for a fraction of second I get a rEFInd screen which says:

error: Secure Boot forbids loading module from (hd0,gpt5)/boot/grub/x86.64-efi/part.acorm.mod

There are several lines like this one on the screen. After this screen disappears, another shows up which says:

rEFInd - Booting OS

and then the Debian's grub screen shows up.

So the image seems to work, and when I select from the grub menu to boot the live system, it boots and works well. But when I choose the second image, the same happens, I mean the first Debian's image is booted instead of the second one. So it's unable to boot the second image, and no matter what Debian's image I choose, it always boots the first one.

Here's the quick screen change: https://youtu.be/iUvJoDdhnxc

So what can be wrong with it?

[Roderick W. Smith]

Unfortunately, this use case is very much a "bleeding edge" setup. The .iso image files have evolved over the years to handle more and more use cases, and in the process they've become barely-maintainable Frankenstein Monsters. Initially, they were mastered with the assumption that they'll be burned to an optical disc. Then a hackish extension enabled them to be written to a USB flash drive (a whole drive, not just a partition). Some now work when written to partitions of a USB flash drive, but that functionality is not yet universal or 100% reliable, as you've discovered.

There may be a workaround that a user could implement, but I don't know what it is. Chances are it would involve modifying the GRUB configuration on the image that isn't booting correctly. You may want to ask about it on a Debian forum, since in your case it's a Debian-specific problem. There MIGHT be a solution within rEFInd, involving having rEFInd launch the Debian kernel directly with the appropriate options; but that solution would likely be no easier than modifying the GRUB configuration, and might in fact be harder. Thus, I'd focus on the GRUB workaround -- but I can't guarantee you'll be able to find something even then.

[Mikhail Morfikov]

I'll try to ask on some Debian ML, because it's weird that Ubuntu works just fine and Debian has issues...

[JH Park]

Thank you.
Actually I did just as you wrote, until #2. I did copy the iso9660 driver, also.
But I have never tried 'dd' to the partitions.

I'll try later and let you know what happens.
To have rEFInd in a flash drive is a big benefit for me.

[Jsf]

Your trial would have been successful if you had used FAT partitions.

[JH Park]

@Jsf
Thank you for your comment. I did try Fat from the beginning, but I failed.

@Roderick W. Smith
It was successful. I can't thank you enough. I should have asked you before I spent hours in front of my monitor. :-o
I tried Ubuntu, Ubuntu Budgie, Kubuntu, and Archlinux, and all works fine. All Ubuntu flavours are 17.10 beta2, and Arch is 2017.10 release.

[Jsf]

Newest and older Ubuntu & Ubuntu like and also many other distros works when just copied (whole main iso content) iinto fat partition. So is windows 10 installer.
Look here for all the methods in general (<Using manual="" formatting="">)
https://wiki.archlinux.org/index.php/USB_flash_installation_media
Now rEFInd scans every partition that it has driver for and to make its job as a boot manager it just has to find *.efi in proper directories ((fat driver built-in), linux kernels in proper directories (ext4 or other drivers) or iso-partitions (iso driver needed) - the method Roderick has described above.

When copying to fat partition one could also, as a bonus, use available free space on every partition.
However I didn't know about ISO method and see now that it works so will try with coming new Windows edition.
Linux or unix distros normally I just copy - it is faster and more safe. And if you take for example PCLinuxOS where nearly all iso content is one huge squashfs file copying with USB 3.0 media take just a few seconds. Files could be quickly deleted later and new copied.
For some problematic isos this method fails so then we could dd iso to partition.

What is interesting now is question whether rEFInd could load configured by paths iso images from hard drive like grub2:
https://www.howtogeek.com/196933/how-to-boot-linux-iso-images-directly-from-your-hard-drive/
It would allow for booting Linux live CDs or even installing Linux on another hard drive partition without booting from a USB drive at all.

[Roderick W. Smith]

In theory, rEFInd could boot a .iso file from a supported filesystem IF a loopback driver for EFI existed. Such a driver would take a file and make it look like a disk device or partition to the firmware. Loopback drivers exist for many OSes, and writing one for EFI shouldn't be all that hard for somebody familiar enough with EFI drivers. In fact, writing such a driver has been on my own "to-do" list for a while, but it's a bigger challenge than other desirable changes to rEFInd (like adding the in-program tag hiding feature for rEFInd 0.11.0), and my own time is limited, so I've not yet tackled this task.

Note that there's the caveat that the booted OS (an Ubuntu installer, Windows installer, etc.) would have to know enough to read the .iso file, too; or it would have to load everything it needs as part of its basic boot process (like from an extra-large initrd file). Given that many Linux distributions can install from a .iso file booted via GRUB, this seems like it shouldn't be a problem for them; but there might be some subtle thing I don't understand that would prevent it from working if rEFInd were to launch an OS installer in this way.

[3guesses]

This is a fairly old thread, so I was just wondering if any further progress had been made with regards to booting .iso files using rEFInd? Roderick Smith suggests that it would not be that difficult to implement. Grub4Dos has this incredibly useful feature but unfortunately does not support EFI, so an EFI alternative is HIGHLY desirable for use on machines with UEFI-only BIOSes. It's way beyond my ken, otherwise I'd have a go myself 8-) A good project for 2020 for someone so inclined?

[Roderick W. Smith]

I wrote that "...writing [a loopback driver] for EFI shouldn't be all that hard for somebody familiar enough with EFI drivers" (emphasis added). That's a rather big caveat. Note that I did not write any of the drivers included with rEFInd, although I have tweaked them a bit. Thus, for me personally, writing such a driver would be harder than it would be for some others, since I'm not all that familiar with EFI driver design.

[おとめ座 ひなたぼっこ]

I am some what late to the party?
The solution is quite simple.
Lets try what I have been discovered.
Just format the partition as EXT4, then copy the content inside the Live Iso to it. Make new Grub bootx64 with built-in ext4 module. It works 100%. See my github since 2017Grub for live linux