How can I install several Linux distributions on a USB stick?

A few pieces of advice and comments on jc__'s tutorial

First of all, many thanks to jc__ (again) for posting his tutorial on how you can achieve the goal stated in my initial post. I can confirm the method works, despite that you might have to do some additional troubleshooting when the time to boot the installed OSes comes, which is why I am making this reply to enumerate a few things you might want to consider while following the tutorial.

0. If you pick older distributions, carefully check you can boot them at all

I learned the hard way that Fedora 8 is too old to boot on more recent hardware such as my current laptop (for reminders, Fedora 8 was released around 2007; my laptop was built in the 2010s). I don't recall the exact details (as I learned this thanks to one of my colleagues), but it seems the kernel has trouble dealing with my hardware and is unable to properly mount everything, which is why I eventually dropped Fedora 8 for Fedora 14 (which works fine).

But how can you check if a distribution will boot properly on your machine ? A simple approach consists in booting from USB with a Live version of the target distribution. If everything boots normally with the Live version, then you should have no trouble installing the distribution through a VM, cloning its partition on your USB stick and boot it from the USB.

In my case, I used YUMI - Multiboot USB Creator to test multiple Live versions of older Fedora releases on my laptop. Of course, this advice only applies if you need, like me, to use older distributions.

1. Consider using FAT32 or ext3/ext4 for the data partition

At some point, my Ubuntu 16.04 had trouble dealing with the data partition when it was formatted in NTFS. Eventually, I re-formatted it as FAT32 - an older format with which any Linux distribution should be okay. ext3/ext4 should be okay too if you exclusively use Linux (as far as I know, Windows cannot deal with these filesystems), though I haven't tested it myself.

I now use the data partition as a way to easily share data between the OSes on the USB and the OSes on the host computer.

2. Consider using a secondary USB stick for partition cloning

Having a second USB stick can be very practical for the first steps, as you can use it to store the images produced by clonezilla. When clonezilla ask you where you will put the /home/partimag/ folder, select the first option in the menu (just above ssh_server) while not having your second USB stick plugged into your machine, then plug it, wait around 5 seconds and press Enter, then select your second USB stick when clonezilla enumerates your options for the /home/partimag/ folder. The subsequent operations will be identical to those enumerated by jc__.

This approach does not - of course - change the outcome and has the merit of not involving your host computer at all with the partitioning of the USB stick and the cloning/restoration of the partitions where the target OSes are located. You can try this if you are unsure about toying around with disk partitioning.

3. If booting your USB OS fails after loading kernel, rebuild initrd/initramfs image

Depending on the target OSes, you might run into issues similar to those I presented in some comments of this topic, such as having timeout with queues in dracut while booting Fedora 24.

In my case, except for the particular case of Fedora 8 (cf. item 0), I fixed the problem by rebuilding the initrd image. Depending on your distribution and how recent it is, this image can also be prefixed by initramfs rather than initrd. No matter the name, the purpose of the file is the same: having a transient filesystem at boot time before mouting the actual filesystem. If this file is corrupted, erroneous or missing, you cannot properly boot your system.

Hopefully, any Linux distribution comes with an utility to rebuild this image. From my (short) experience:

• Older distributions: mkinitrd
• Recent Ubuntu distributions: mkinitramfs
• Recent Fedora distributions: dracut

NOTE: when (re-)building the image, each utility will take the /etc/fstab file into account. Take advantage of this to already link a swap partition with your USB OS. In /etc/fstab, you can use an UUID to denote a partition, so use fdisk and blkid to get the UUID of the desired swap partition and update the /etc/fstab accordingly.

To use one of the three commands I listed to rebuild the image, you need first to chroot into the OS which you want to rebuild the initrd/initramfs image. I basically follow the same approach as in this tutorial, which consists in mounting the partition of the buggy OS (so, one of our USB OSes) on your host computer and using a few binding commands to properly chroot in it. You can create sub-folders in your /mnt folder if needed. In my case, to chroot into my Fedora 14, I used the following commands:

sudo mkdir /mnt/usb_f14 sudo mount /dev/sdb3 /mnt/usb_f14 sudo mount --bind /proc /mnt/usb_f14/proc sudo mount --bind /dev /mnt/usb_f14/dev sudo mount --bind /sys /mnt/usb_f14/sys sudo chroot /mnt/usb_f14 

Don't forget to check beforehand what /dev/sdX your partition is with fdisk. Once you are rooted in your USB OS, move to the /boot folder. If you wish to, rename the already present image (if any) to keep it (you can simply prefix it with old-, for instance) before you create the new image. Then, run

ls /lib/modules/ 

This will list the different kernels you have on your USB OS (most of the time, you should have only one). Indeed, any of the command I listed above requires you to give the kernel you will use to build the image. Then, depending on the utility you have access to, you should run one of the three following commands (by my-kernel, I denote one of the kernels listed by ls /lib/modules/; note that you can use other names for your images if you wish)

• mkinitrd initrd-my-kernel.img my-kernel
• mkinitramfs -o initramfs-my-kernel.img my-kernel
• dracut initramfs-my-kernel.img my-kernel

Depending on your OS, there might be some messages about missing modules. As long as the image is produced, you can ignore them (I got some while rebuilding the image for my Fedora 14, and it still boots fine).

Note, also, that if you directly replace the old image, you might have to add a flag to your command to force the overwriting. For instance, with dracut, you should add --force. Refer to your manual for the other commands. Once you are done, unmount your USB OS partition and reboot your machine with the boot sequence set to boot from USB first. Your USB OS should now boot normally.

Finally, don't forget to take note of the name you gave to the new image, if different from the name of the initial image, because you will need to update your grub.cfg file to take the new image into account.

4. Double-check the partitions on the virtual machines

If you couldn't choose yourself the partitioning when installing one of your target OSes in a virtual machine (it happened to me with Fedora 14), it is possible you end up with two ext4 partitions, with a small one containing the /boot folder and the other containing the rest of the filesystem. If this happens, clone the larger partition with the method described by jc__, then boot again your virtual machine, copy somewhere (e.g., on secondary USB stick) the /boot folder and later place it in the partition you cloned after restoring it. Don't forget to update the /etc/fstab file afterwards before re-building the initrd image (see 3), as the original file will likely still list two ext4 partitions instead of one.

5. Exiting your USB GRUB, the easy way

While jc__ recommends to use super_grub2_disk_hybrid_2.02s9.iso and memdisk to get all the options on the host machine, you can also just add this simple menu entry in your grub.cfg:

menuentry 'Continue to host machine' { exit } 

This will make GRUB on the USB stick quit and resume the boot sequence of the host machine. In my case, since my host computer is a laptop with dual boot, I get the GRUB menu of my machine right after selecting this menu item.

That's about it for the comments I wanted to make. Since I don't have a lot of experience with Linux troubleshooting, don't hesitate to comment this post to point errors or inaccurate statements.

Placing this in an answer spot because it is long and wordy. If answer corrections needed or additions will edit answer and delete this.

"grub.cfg (which I placed in the Fedora 24 partition, so /dev/sda2/"

"FC24 is /dev/sda2"

"FC8 is /dev/sda3"

Okay lets do some things to help with trouble shooting.

show the partition layout of the USB

GParted gui or sudo fdisk -l

show the UUID numbers of the USB partitions blkid

show the command used when installing grub.grub-install and grub-mkimage

On the VM used in step 5, did it boot from a hard grive (VHD) or a live OS. What OS was used.

"Did I restore my partitions wrong or can I "rescue" both OSes ?"

I don't think it is a restore issue, but a grub install issue.

Looking at the grub.cfg.

Entry: Fedora (4.5.5-300.fc24.x86_64) 24 (Workstation Edition) search ... --hint='hd0,msdos1' 913f30c2-4a22-44eb-a93d-9fad95764f5d first HD and first partition. --hint='hd0,msdos1 partition UUID 913f30c2-4a22-44eb-a93d-9fad95764f5d This tells us: /dev/sda1 and was probably the HD not USB. Expected. Entry: Fedora release 24 (Twenty Four) (on /dev/sda2) search ... --hint-bios=hd0,msdos2 d4be737a-c4ea-44cd-a22d-781086854ec8 first HD and second partition --hint-bios=hd0,msdos2 partition UUID d4be737a-c4ea-44cd-a22d-781086854ec8 This tells us: /dev/sda2 and is the second partition on the HD not USB. Not expected if this OS is on the USB. Expecting second HD (the USB) NOT the same as the booted OS. Perhaps /dev/sdb2 Entry: Fedora release 8 (Werewolf) (on /dev/sda3) search ... --hint-bios=hd0,msdos3 8bc6e37c-53b1-4fe9-b242-5b0b9c8c575e first HD and 3rd partition. --hint-bios=hd0,msdos3 partition UUID 8bc6e37c-53b1-4fe9-b242-5b0b9c8c575e Not expected if this OS is on the USB. Expectiong second HD (the USB) NOT the same as the booted OS. Perhaps /dev/sdb3 

The reason we expect any HD except the first one (or the one the host OS is on) is because the computer is booted from an OS (assume from a HD and not an ISO), the USB drive is then inserted and will be the next available HD. /dev/sdX The X is the drive letter.

At this point you are interacting with the booted OS, the host. The USB drive has been attached. The host OS may have auto mounted the USB. This is okay if read/write mount. I prefer to unmount (NOT eject) the USB and remount at my determined locations.

Verify the USB device assignment. sudo fdisk -l You know the layout and size of the USB, so get the /dev/sdX info.

OR

Unplug the USB from the host.

command dmesg

Plug the USB back in.

command dmesg

Should tell you abount the new device and the /dev assingment. sdX.

Mount the USB partitions to your predermined mount points.

example:

sudo mount /dev/sdX /media/f24

sudo mount /dev/sdx /media/f8

/media/f24 -- the root of the Fedora24 /media/f8 -- the root of the Fedora8 

Reinstall grub bootloader to the USB drive and point to the mounted partition.

sudo grub-install --recheck --boot-directory=/media/f24/boot /dev/sdX

--boot-directory=/media/f24/boot

This is where the boot loader will look for the grub.cfg file. BTW the one already there will not get us there.

/dev/sdX

This is the USB drive. Info gathered from above. No partition info here only the device.

Create a new grub.cfg file that will overwrite the file created by Fedora24 install.

sudo grub-mkconfig --output=/media/f24/boot/grub/grub.cfg

This will overwrite the grub.cfg from Fedora24 install.

Verify the new grub.cfg file.

Should have the host OS currently running info entry.

Should have the USB drive OSs. These will be on a different device than the host OS. /dev/sdX.

Also verify using command: blkid

The UUID numbers for the USB drive partitions should match the UUID numbers in the new grub.cfg.

NOTE: The /dev/sdX assignment will be incorrect when the USB is booted. Thats okay. The UUID partition number will still 'find' the correct partition and boot the correct OS. This will be adjusted in the custom grub.cfg later.

also verify the correct grub.cfg s loaded on USB boot.

After commands above change the colors in the new grub.cfg

Add the lines:

# Set menu colors set menu_color_normal=red/black set menu_color_highlight=light-gray/red 

Guessing between the lines:

### END /etc/grub.d/01_users ### and ### BEGIN /etc/grub.d/10_linux ### 

When you see a grub menu in different colors you know the correct one loaded. grub-install is correct.

Dont get stuck on this part, you could simply change the menu entries name...

If new grub.cfg passes UUID verification you know grub-mkconfig is correct.

Trouble shoot part 2.

Looking at the Ferora 24 menu entry...

GRUB is setting root 3 different ways. (NOT the Fedora kernel root. that is something else.)

1. set root='hd1,msdos2'

This is a 'hard' set of root. The 2nd. drive on the 2nd. partition.

The 'trick' here is this what drive is first can change on every boot. What ever device probes first becomes first.

2. search --no-floppy --fs-uuid --set=root --hint-bios=hd1,msdos2 --hint-efi=hd1,msdos2 --hint-baremetal=ahci1,msdos2 d4be737a-c4ea-44cd-a22d-781086854ec8

This entry inside the if is basically saying we want partition with UUID xxx, but start looking on 2nd. drive 2nd. partition.

3. search --no-floppy --fs-uuid --set=root d4be737a-c4ea-44cd-a22d-781086854ec8

This is also inside the if statement and is saying go look for partition with UUID xxx anywhere you can look, except the floppy drive.

We want to use the 3rd syntax.

NOTE: GRUB counts the drive from 0 (hd0 - hdn) and counts the partitions from 1 (msdos1 - msdosn). GRUB2 manual.

"seems to me the initrd/initiramfs might be at fault here"

Yes, I think so too, but not the files themselves but where GRUB is looking for the files.

GRUB is looking to load the linux kernel from /boot/vmlinuz-0-rescue-731a5ce6cbc840ad9cad96ddf48e5db8, or more accurately GRUBroot/boot/vmlinuz-0-rescue-731a5ce6cbc840ad9cad96ddf48e5db8

GRUBroot = partition d4be737a-c4ea-44cd-a22d-781086854ec8. If we set it correct.

initrd is same.

GRUBroot/boot/initrd-2.6.23.1-42.fc8.img

BTW: Your first grub.cfg had Fedora24 loading different kernel and initrd. You may want to revert to using those again because of production kernel and language etc...

linux /boot/vmlinuz-4.5.5-300.fc24.x86_64 root=UUID=d4be737a-c4ea-44cd-a22d-781086854ec8 ro rhgb quiet LANG=fr_BE.UTF-8

initrd /boot/initramfs-4.5.5-300.fc24.x86_64.img

Verify files still exist.

"/dev/root cannot be found"

Yes because

linux /boot/vmlinuz-0-rescue-731a5ce6cbc840ad9cad96ddf48e5db8 root=/dev/sdc2

is saying the kernel root is on the 3rd. drive 2nd. partition.

Lets create a new entry by changing grub.cfg with text edit. Delete the others after this works.

menuentry 'Fedora 24 on USB drive' { insmod part_msdos insmod ext2 ## set the GRUB root using UUID only. search --no-floppy --fs-uuid --set=root d4be737a-c4ea-44cd-a22d-781086854ec8 ## is F24 root somewhere else? ## Probably not.. remove kernel switch root=/dev/sdc2 linux /boot/vmlinuz-0-rescue-731a5ce6cbc840ad9cad96ddf48e5db8 initrd /boot/initramfs-0-rescue-731a5ce6cbc840ad9cad96ddf48e5db8.img ## I think you want these instead if the files still exist. Make adjustments by adding and removing comments ## ## commented out # linux /boot/vmlinuz-4.5.5-300.fc24.x86_64 root=UUID=d4be737a-c4ea-44cd-a22d-781086854ec8 ro rhgb quiet LANG=fr_BE.UTF-8 # initrd /boot/initramfs-4.5.5-300.fc24.x86_64.img } 

and

menuentry 'Fedora 8 on USB drive' { insmod part_msdos insmod ext2 ## set the GRUB root using UUID only. search --no-floppy --fs-uuid --set=root 8bc6e37c-53b1-4fe9-b242-5b0b9c8c575e ## is F8 root somewhere else? ## Probably not.. remove kernel switch root=LABEL=/ ## not familure with this switch, but removed. linux /boot/vmlinuz-2.6.23.1-42.fc8 ro rhgb quiet initrd /boot/initrd-2.6.23.1-42.fc8.img ## I think you want these instead if the files still exist. Make adjustments by adding and removing comments ## ## commented out # linux /boot/vmlinuz-2.6.23.1-42.fc8 ro root=LABEL=/ rhgb quiet # initrd /boot/initrd-2.6.23.1-42.fc8.img } 

No update required, just reboot to the USB drive.

Remember after you get this working you will want to modify etc/grub.d/40_custom so a Fedora kernel update will probe and generate the correct new GRUB menu entry.

Not positive this can be done. You may be forced to manually update grub.cfg every time the Fedora kernel updates.

linux /boot/vmlinuz-4.5.5-300.fc24.x86_64 root=UUID=d4be737a-c4ea-44cd-a22d-781086854ec8 ro rhgb quiet LANG=fr_BE.UTF-8

initrd /boot/initramfs-4.5.5-300.fc24.x86_64.img

The answers provided have good information and offer insight to the process, which I consider very important (particularly for the auto-didactic types, which describes a lot of us here at SE).

However, there is a free project that more or less achieves something like this for those who come here and just want to accomplish something similar, if not exactly the same.

https://www.pendrivelinux.com/yumi-multiboot-usb-creator/

The name is silly. The graphics are cheesy. The tools work, and they work pretty well. At the very least the source code they make available might help you.