4

I'm using Gentoo on my HomeServer. My HDDs are raid1 mirrored with mdadm.

My problem is how to boot the /boot partition correct?

/dev/md1 consists of /dev/sdc2 & /dev/sdd2 mounted on /boot

In my grub.cfg I have the uuid of /dev/md1 but at boot time the array is of course not ready.

simplest way would be to un-raid the /boot partitions, but that is not what I want to do.

so, how do i find out the UUID of the raid members? ls -l /dev/disk/by-uuid only shows me the uuid of /dev/md1. And when I unraid the disks, I cannot mount them to get their UUID mount: unknown filesystem type 'linux_raid_member'

so how can I let Grub2 boot those disks, will it be able to determine the contents before they are in raid? (they are in fact ext2).

I should maybe add that I use gpt partition tables, therefore I have a small partition /dev/sdc1 and /dev/sdd1 on both drives to host the bios_grub partition. those are not raided atm.

As pointed out in the comments below Gilles' answer my problem persists. Grub won't find the device, telling me 

error: no such device: 9f81a-(device uuid)-5580.
entering rescue mode...
grub rescue>

afterwards I added the lines 

insmod part_msdos
insmod part_gpt
insmod raid
insmod mdraid

to my grub.cfg, just above the line where root is defined root='(md0)' I still get the same error.

I did some more research and i really guess its an error related to the uuids:

# grub-probe -d --target=fs_uuid /dev/md1
9f81a35d-0813-481f-9ae0-e4fba57c5580
# blkid
/dev/sdc2: UUID="11adb545-0e80-61d1-61f6-565f18e8c3f0" UUID_SUB="88826c5e-d12b-307d-6e54-556d1ebb2458" LABEL="livecd:1" TYPE="linux_raid_member"
/dev/sdd2: UUID="11adb545-0e80-61d1-61f6-565f18e8c3f0" UUID_SUB="15150fb2-5066-edba-d39b-08b63219453c" LABEL="livecd:1" TYPE="linux_raid_member"
/dev/md1: UUID="9f81a35d-0813-481f-9ae0-e4fba57c5580" TYPE="ext2"
(I cut away the other drives here)

as you can see both partitions have the same UUID, so I edited the UUID of the drives in my grub.cfg and did grub-install /dev/sdx again, but I still get the error with the "old" /dev/md1 UUID.

Baarn
  • 882
  • 2
  • 11
  • 18
  • `grub.cfg` isn't involved at this point: `grub-install` doesn't use it, and at boot time Grub gets stuck before loading it. Apparently your core image is missing RAID support. What superblock format version does your array have? And please post a transcript of `bash -x /usr/sbin/grub-install /dev/sdc`. – Gilles 'SO- stop being evil' Jul 28 '11 at 16:18
  • http://pastebin.com/JRrWfyBM here we go. superblock on all arrays is 1.2 . – Baarn Jul 28 '11 at 16:44
  • Perhaps I am missing something, but you should be able to generate the grub config automatically with `grub-install` and `update-grub`. Writing it manually is usually not necessary. Does that not work for you? – Faheem Mitha Oct 08 '11 at 05:01
  • nope, for some setups it does not work out. changing the grub.cfg is maybe not the best way, you could also edit the files from which grub-mkconfig gets it information but in the end there is very little difference. At the moment I just boot from a USB-stick where grub and kernel are stored on, not what i initially wanted to do, but pretty nice, too. – Baarn Dec 15 '11 at 08:05

3 Answers3

6

The Gentoo Wiki has information that discusses both Grub 1 and Grub 2. Note that the situation is completely different in Grub 1 and Grub 2: Grub 1 doesn't know anything about RAID, but can fake reading from a RAID-1 device by reading one of the disks (this requires a 0.9 or 1.0 mdraid superblock format, not 1.1 or 1.2); Grub 2 supports Linux mdraid volumes natively.

Grub 2 works in three stages (see here for more details):

  1. The boot sector.
  2. A core image, generated by install-grub in /boot/grub/core.img, capable of showing the rescue prompt and loading modules but not much else.
  3. A full system with loadable modules, which normally (through the normal module) shows a boot menu described by /boot/grub/grub.cfg.

Generating the core image

Run grub-install /dev/sda to generate the core image and populate /boot/grub with module files. grub-install also installs a boot sector on the indicated device. Run both grub-install /dev/sdc and grub-install /dev/sdd if you want to have a boot sector on both drives.

If grub-install doesn't detect your drives properly, create the following script as /usr/local/sbin/grub-probe-verbose and run grub-install --grub-probe=/usr/local/sbin/grub-probe-verbose to see what's going on.

#!/bin/sh
exec /usr/sbin/grub-probe "$@" | tee /dev/stderr

From your transcript from bash -x /sbin/grub-install /dev/sdc, it appears that grub-probe isn't detecting /dev/md1 as a RAID array (/sbin/grub-probe --device-map=/boot/grub/device.map --target=partmap --device /dev/md1 and /sbin/grub-probe --device-map=/boot/grub/device.map --target=abstraction --device /dev/md1 return nothing). In case the issue is only in grub-probe and not in the boot code, try overriding its decision:

grub-install --modules='biosdisk ext2 msdos raid mdraid' /dev/sdc

Support for mdraid 1.x superblocks is still recent; it's not in Grub 1.98, you need at least Bazaar revision 2550 dated 2010-07-20 (the code was on a branch for a few months before). Debian squeeze and Ubuntu 10.10 ship with 1.x superblock support; I would expect Gentoo to have 1.99 by now.

Generating grub.cfg

Run grub-mkconfig -o /boot/grub/grub.cfg.grub-mkconfig to produce a tentative Grub configuration file. Review it, and if looks correct, rename it to /boot/grub/grub.cfg.

Device map

You may need to tweak /boot/grub/device.map to get the output of grub-mkconfig right. This file will be generated automatically by grub-mkdevicemap if it doesn't exist, but setups with many disks tend to confuse it. I would expect it to contain something like

(md0) /dev/md1
(hd0) /dev/sdc
(hd1) /dev/sdd

where md0 is the Grub volume name and /dev/md1 is the name under Linux. Grub needs to access /boot/grub for two things:

  • Grub's boot sector reads the core image at a fixed location on a disk supported by the BIOS (it doesn't have much choice). With a PC BIOS, (hd0) is the disk that the BIOS reads the boot sector from (it's the disk that contains the boot sector), and (hd1) is some other disk (you don't always get to choose). It's simpler if (hd0) is the disk (or a disk, with a RAID-1 array) that contains /boot/grub.
  • The core image loads modules and grub.cfg and the Linux kernel from a filesystem location determined by grub-install, typically /grub or /boot/grub from a device that can be a RAID array or any other volume type supported by Grub.

If you change device.map, you need to run grub-install again. You may need to run grub-mkconfig again if you're not relying on the search command for everything.

Community
  • 1
Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
  • The Link looks promising. I already modified my device.map because I had a device not found error (/dev/md1), but i in fact did not think of using grub-counting-style. – Baarn Jul 27 '11 at 18:01
  • I added the line `(md0) /dev/md1` to my device.map and `grub-mkconfig` found the corresponding UUID and wrote it to the grub.cfg. but after booting I get this `error: no such device: 9f81a...restofUUID` – Baarn Jul 27 '11 at 21:13
  • @WalterMaier-Murdnelch Do you have the necessary `insmod` directives (`insmod part_msdos`, `insmod raid`, `insmod mdraid` should do it)? I would have expected `grub-mkconfig` to generate them, but maybe it didn't. Post your `grub.cfg`. – Gilles 'SO- stop being evil' Jul 27 '11 at 21:22
  • http://pastebin.com/qWaSYw8a the insmod stuff is missing, thats exactly what I need I guess, will try it right away – Baarn Jul 27 '11 at 21:26
  • still the same error... strange. – Baarn Jul 27 '11 at 22:29
  • Grub2 prefers to use grub-probe to figure things out automagically instead of using a static device.map. It can boot just fine without a separate /boot partition also. That should also allow grub-mkconfig to automatically build the required raid modules into the core image. – psusi Jul 27 '11 at 22:42
  • the separate /boot is necessary, rest of the drive is encrypted... but lets not discuss this here, I think I will get some further problems with that as well. and yeah i read somewhere that i can also boot an encrypted partition with grub, but atm i cant even boot an unencrpyted one, so lets keep it simple ;) – Baarn Jul 27 '11 at 22:45
  • @WalterMaier-Murdnelch If you're dropped at the `rescue` prompt, it means Grub hasn't been fully loaded: your `core.img` is missing a necessary driver. Please read my updated answer, I realize I missed part of the story originally. If you still still can't generate a core image that works, post the transcript of `grub-install --grub-probe=/usr/local/sbin/grub-probe-verbose`. – Gilles 'SO- stop being evil' Jul 28 '11 at 13:12
  • i changed my device.map according to your recommendations. i still get dropped to the rescue prompt. entering commands like `insmod (md0,1)/` yields `no such disk`, `insmod (hd0,1)/` yields `no such partition` and `insmod (hd0)/` yields `unknown filesystem`. where can I specify which modules are built into the core.img? – Baarn Jul 28 '11 at 13:43
  • i just read here http://www.linuxquestions.org/questions/linux-general-1/raid-and-grub-2-on-ubuntu-karmic-koala-788540/ that grub gets its raid info from `/etc/mdadm/mdadm.conf` – Baarn Jul 28 '11 at 13:54
  • entering `mdadm --auto-detect` in the rescue console does not work, but entering `ls` yields `(hd0) (hd1) (hd2) (hd3) (fd0)` – Baarn Jul 28 '11 at 13:55
  • http://sinodun.com/2010/07/installing-grub2-on-gnulinux-software-raid/ this blog-entry looks promising as well, working through it atm. – Baarn Jul 28 '11 at 14:03
  • grub doesn't care about mdadm.conf; it reads the raid superblocks on the disk. You can't load a module from the disk when the module you are missing is one required to access the disk. Normally `grub-install` runs `grub-probe` which tells it that /boot is on a raid, so it builds raid module into the core image. This doesn't seem to be happening for you. – psusi Jul 28 '11 at 19:57
  • @WalterMaier-Murdnelch `grub-probe` doesn't detect your RAID array as such. Make sure you have a recent enough Grub, and try forcing the necessary modules into `core.img` (see my edit). – Gilles 'SO- stop being evil' Jul 28 '11 at 21:01
  • thanks so much, I added the modules to the grub-install, now i am dropped out to the "normal" grub console. bad thing is there is no error output, so I am not sure whats the error. :/ edit: my grub version is 1.99~rc1 – Baarn Jul 30 '11 at 18:29
0

To answer the question about the UUID: Use blkid

Nils
  • 18,202
  • 11
  • 46
  • 82
-3

livecd ~ # mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdc2 /dev/sdd2

livecd ~ # grub

grub> root (hd0,1) 

Filesystem type is ext2fs, partition type 0xfd

grub> setup (hd0) 

Checking if "/boot/grub/stage1" exists... yes  
Checking if "/boot/grub/stage2" exists... yes  Checking if
"/boot/grub/e2fs_stage1_5" exists... yes  Running "embed
/boot/grub/e2fs_stage1_5 (hd0)"...  16 sectors are embedded. succeeded
Running "install /boot/grub/stage1 (hd0) (hd0)1+16 p
(hd0,0)/boot/grub/stage2 /boot/ grub/menu.lst"... succeeded Done.
Killinks
  • 11
  • 1
    Please don't just post dumps of commands, it's generally not sufficient. Add explanations. And make sure what you do dump is actually relevant to the question. – Mat Feb 01 '12 at 10:30
  • if it were this easy Gilles and me would have done it, I tried it again with a completely other computer and still didn't got it working. and btw is this answer using grub legacy not 2 – Baarn Feb 01 '12 at 14:12