3

My goal is to have a dual-boot of Qubes OS 3.1 and Fedora 23 Workstation (respin). This guy got it working on Ubuntu, but he installed Ubuntu first, while I installed Qubes first. The last resort would be to follow him and reinstall from scratch, but I want to learn something.

What have I tried:

  • installed Qubes OS 3.1 (OK)
  • updated Qubes OS 3.1 dom0 and Fedora23 template (OK)
  • installed the latest Fedora 23 Workstation respin (OK, but reported some GRUB2 bootloader failure; nevertheless Fedora booted fine)
  • updated the latest Fedora 23 Workstation (OK, there were no updates)
  • tried to boot Qubes (fail, no menuentry in GRUB2 boot menu yet)
  • created a menuentry for Qubes (OK, see bellow)
  • tried to boot Qubes (fail, see screenshot)
  • googled this workaround and installed the package
  • I get thgrough the error message now, but after a few seconds the system reboots itself and does that infinitely

How I added the menuitem:

I booted into Fedora and went to the /etc/grub.d/ folder, and edited the 40_custom file. In there i pasted everything between the xen section of the /boot/grub2/grub.cfg file on the qubes boot partition, into the end of that 40_custom file on Fedora. Then I executed the following command under root

grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

and the qubes menuitems got generated into the grub.cfg and became visible in the boot menu. I can provide the contents of grub.cfg later, in the meantime I'd like to know what might got wrong and how to fix it.

Notable information:

  • Qubes boots from GPT on Legacy BIOS (with 2MB biosboot partition)
  • Fedora boots from GPT on UEFI (with EFI partition)
  • each OS has separate /bootpartition, as well as other partitions
  • there is no extra /bootpartition

Here is the promised grub.cfg:

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/01_users ###
if [ -f ${prefix}/user.cfg ]; then
  source ${prefix}/user.cfg
  if [ -n "${GRUB2_PASSWORD}" ]; then
    set superusers="root"
    export superusers
    password_pbkdf2 root ${GRUB2_PASSWORD}
  fi
fi
### END /etc/grub.d/01_users ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Fedora (4.4.8-300.fc23.x86_64) 23 (Workstation Edition)' --class fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-4.4.8-300.fc23.x86_64-advanced-d43f46bc-7649-44ca-b02d-7599d115a8e8' {
    load_video
    set gfxpayload=keep
    insmod gzio
    insmod part_gpt
    insmod ext2
    set root='hd0,gpt6'
    if [ x$feature_platform_search_hint = xy ]; then
      search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt6 --hint-efi=hd0,gpt6 --hint-baremetal=ahci0,gpt6  440e2ced-56a2-432f-95e0-c5f1c33941a9
    else
      search --no-floppy --fs-uuid --set=root 440e2ced-56a2-432f-95e0-c5f1c33941a9
    fi
    linuxefi /vmlinuz-4.4.8-300.fc23.x86_64 root=UUID=d43f46bc-7649-44ca-b02d-7599d115a8e8 ro rootflags=subvol=root00 rhgb quiet 
    initrdefi /initramfs-4.4.8-300.fc23.x86_64.img
}
menuentry 'Fedora (0-rescue-1dccfd914c254de687a4ba1c13e159a9) 23 (Workstation Edition)' --class fedora --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-0-rescue-1dccfd914c254de687a4ba1c13e159a9-advanced-d43f46bc-7649-44ca-b02d-7599d115a8e8' {
    load_video
    insmod gzio
    insmod part_gpt
    insmod ext2
    set root='hd0,gpt6'
    if [ x$feature_platform_search_hint = xy ]; then
      search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt6 --hint-efi=hd0,gpt6 --hint-baremetal=ahci0,gpt6  440e2ced-56a2-432f-95e0-c5f1c33941a9
    else
      search --no-floppy --fs-uuid --set=root 440e2ced-56a2-432f-95e0-c5f1c33941a9
    fi
    linuxefi /vmlinuz-0-rescue-1dccfd914c254de687a4ba1c13e159a9 root=UUID=d43f46bc-7649-44ca-b02d-7599d115a8e8 ro rootflags=subvol=root00 rhgb quiet 
    initrdefi /initramfs-0-rescue-1dccfd914c254de687a4ba1c13e159a9.img
}
if [ "x$default" = 'Fedora (4.4.8-300.fc23.x86_64) 23 (Workstation Edition)' ]; then default='Advanced options for Fedora>Fedora (4.4.8-300.fc23.x86_64) 23 (Workstation Edition)'; fi;
### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###

### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###

### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Qubes, with Xen hypervisor' --class qubes --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-simple-6e934da1-b43d-434b-83b2-878d454fb6bd' {
    insmod part_gpt
    insmod ext2
    set root='hd0,gpt2'
    if [ x$feature_platform_search_hint = xy ]; then
      search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 --hint='hd0,gpt2'  9c6e43e7-856f-4840-a04f-6936735f5835
    else
      search --no-floppy --fs-uuid --set=root 9c6e43e7-856f-4840-a04f-6936735f5835
    fi
    echo    'Loading Xen 4.6.0 ...'
        if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ]; then
            xen_rm_opts=
        else
            xen_rm_opts="no-real-mode edd=off"
        fi
    multiboot   /xen-4.6.0.gz placeholder  console=none dom0_mem=min:1024M dom0_mem=max:4096M ${xen_rm_opts}
    echo    'Loading Linux 4.1.13-9.pvops.qubes.x86_64 ...'
    module  /vmlinuz-4.1.13-9.pvops.qubes.x86_64 placeholder root=UUID=6e934da1-b43d-434b-83b2-878d454fb6bd ro rootflags=subvol=root i915.preliminary_hw_support=1 vconsole.font=latarcyrheb-sun16 rhgb quiet 
    echo    'Loading initial ramdisk ...'
    module  /initramfs-4.1.13-9.pvops.qubes.x86_64.img
}
submenu 'Advanced options for Qubes (with Xen hypervisor)' $menuentry_id_option 'gnulinux-advanced-6e934da1-b43d-434b-83b2-878d454fb6bd' {
    submenu 'Xen hypervisor, version 4.6.0' $menuentry_id_option 'xen-hypervisor-4.6.0-6e934da1-b43d-434b-83b2-878d454fb6bd' {
        menuentry 'Qubes, with Xen 4.6.0 and Linux 4.1.13-9.pvops.qubes.x86_64' --class qubes --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.1.13-9.pvops.qubes.x86_64-advanced-6e934da1-b43d-434b-83b2-878d454fb6bd' {
            insmod part_gpt
            insmod ext2
            set root='hd0,gpt2'
            if [ x$feature_platform_search_hint = xy ]; then
              search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 --hint='hd0,gpt2'  9c6e43e7-856f-4840-a04f-6936735f5835
            else
              search --no-floppy --fs-uuid --set=root 9c6e43e7-856f-4840-a04f-6936735f5835
            fi
            echo    'Loading Xen 4.6.0 ...'
                if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ]; then
                    xen_rm_opts=
                else
                    xen_rm_opts="no-real-mode edd=off"
                fi
            multiboot   /xen-4.6.0.gz placeholder  console=none dom0_mem=min:1024M dom0_mem=max:4096M ${xen_rm_opts}
            echo    'Loading Linux 4.1.13-9.pvops.qubes.x86_64 ...'
            module  /vmlinuz-4.1.13-9.pvops.qubes.x86_64 placeholder root=UUID=6e934da1-b43d-434b-83b2-878d454fb6bd ro rootflags=subvol=root i915.preliminary_hw_support=1 vconsole.font=latarcyrheb-sun16 rhgb quiet 
            echo    'Loading initial ramdisk ...'
            module  /initramfs-4.1.13-9.pvops.qubes.x86_64.img
        }
    }
}
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/40_custom.backup ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom.backup ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

a copy is also on Pastebin.

Slazer
  • 123
  • 2
  • 8

2 Answers2

1

Here is an authoritative answer from Marek Marczykowski-Górecki (Invisible Things Lab), who is one of the core developers of the Qubes OS. The source is the qubes-users forum, where I originally asked the question.

This is main problem - you're trying to boot legacy system (Qubes here) from UEFI grub (Fedora). You need to choose one of them - either boot both in legacy mode, or both in UEFI mode (or change bios settings at each boot...). So, one option is to setup Fedora in legacy mode - then putting Qubes legacy boot entry in 40_custom (as you've done) should be enough.

It is also possible to boot Qubes from UEFI, but it requires different configuration. And in most cases Grub doesn't make it easier... Anyway you can try:

  1. Locate xen-4.6.0.efi - it should be installed into /boot/efi/EFI/qubes/. Place it in your EFI System Partition (if /boot/efi isn't there already).

  2. Create xen.cfg in the same directory with a content like this:

    [global] default=4.1.13-8.pvops.qubes.x86_64

    [4.1.13-8.pvops.qubes.x86_64] options=loglvl=all kernel=vmlinuz-4.1.13-8.pvops.qubes.x86_64 root=/dev/dm-2 vconsole.font=latarcyrheb-sun16 rd.lvm.lv=qubes_dom0/root ramdisk=initramfs-4.1.13-8.pvops.qubes.x86_64.img

    Adjust kernel version, root= and rd.lvm.lv= parameters.

  3. Make sure the kernel and initramfs files are in the same directory as xen.cfg (most likely you'll need to copy them from /boot). It is critical to have them on EFI System Partition (even if Grub can load them from other places).

  4. Add grub entry:

    menuentry 'Qubes UEFI' { chainloader /EFI/qubes/xen-4.6.0.efi }

Look here for related troubleshooting: https://www.qubes-os.org/doc/uefi-troubleshooting/ https://github.com/QubesOS/qubes-issues/issues/794

Slazer
  • 123
  • 2
  • 8
0

I doubt this will help, but here's what I've done to dual-boot Qubes and Mint. This is a direct crib from Micah Lee's blog, so I take no credit for it. (https://micahflee.com/2014/04/dual-booting-qubes-and-ubuntu-with-encrypted-disks/)

I booted from a Mint live DVD for the initial hdd partitioning.

  1. Create three partitions on the HDD: one for Qubes and one for Fedora, and a small boot partition. Format the partition for Fedora (ext4 or whatever) and the boot partition, and set the boot flag. Leave the Qubes partition unformatted -- Qubes' installer seems happiest finding free space for its encrypted partition. But don't install Qubes yet.

  2. Install Fedora on the ext4 partition, with the bootloader installed to the the boot partition. Don't let it install the bootloader to dev/sda.

  3. Now reboot from your Qubes installation media. The installer should find the unformatted partition and you should be able to install Qubes there.

  4. Reboot. When I did that, the only option was Qubes. Go ahead and boot to Qubes.

  5. Open a terminal in dom0, and use vim or whatever to edit /etc/grub.d/40_custom, just adding three lines at the top:

menuentry "Fedora 23" {    
set root=(hdd0,1)
chainloader +1
}

Then, as per Micah's blog, I reinstalled grub:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo grub2-install /dev/sda

I ended up with a dual-boot Qubes-and-Mint machine.

Good luck.

D In Eugene
  • 1
  • 1
  • Thank you for your advice, but I already know of Micah Lee's blog. I have even referenced it in my question under "This guy". What I want is a non-destructive way to install the bootloader without any re-installation. It would also be extremely informative to understand why does it not happen using the procedure I tried. – Slazer May 05 '16 at 21:49
  • Did the advice from Marek Marczykowski-Górecki about either legacy or UEFI, but avoid mixing them across boot systems, help the situation? – D In Eugene May 13 '16 at 01:55
  • I sense sarcasm in your comment. Marek not only pinpointed what exactly was wrong, but also given an original solution to make Legacy Qubes boot under UEFI. Your question only repeated the tutorial I referenced in the 1st line of my question, with an explanation why it is not usefull. So please, don't be offended, I could not have given you the bounty even if I wanted to, you did not have at least two upvotes to allow me to award it. – Slazer May 13 '16 at 08:52
  • No, no sarcasm was meant, Slazer. The "bounty" is irrelevant to me, and I don't come to this site often enough to worry about votes. I was (and remain) curious about the results on your hardware because I will be reconfiguring another of my machines to dual-boot Qubes (but not with Mint this time), and ... well, I would prefer to be aware of pitfalls before I start! – D In Eugene May 16 '16 at 19:31