2

I have 6 1TB drives from several years ago. There was a problem with the Seagate drives's microcode, so I updated it. Unfortunately, this reset the UUID of two drives and the RAID6 array was broken. I was busy at the time, and just stored the disks for later. I've now come back (some years later), nothing is the same anymore, all I have are the 6 drives, and 6 images made of them. Does anyone have an idea how I might get to a mounted array to check just what data is there ?

I think this question hasn't really been covered fully before, I've tried looking for a simple step by step guide and not found one that matches my situation.

I'm currently using linuxmint19.3, at the time of the failure, it was probably xubuntu 13.04. 

mdadm --version
mdadm - v4.1-rc1 - 2018-03-22

I managed to extract this info from the original log files.

-----------------------------------------------------------
[    2.042156] scsi 0:0:0:0: Direct-Access     ATA      Hitachi HDS72105 JP2O PQ: 0 ANSI: 5
[    2.042742] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
-----------------------------------------------------------
[    2.043315] scsi 1:0:0:0: Direct-Access     ATA      ST3750640A       3.AA PQ: 0 ANSI: 5
[    2.044099] sd 1:0:0:0: [sdb] 1465149168 512-byte logical blocks: (750 GB/698 GiB)
-----------------------------------------------------------
[    2.353991] scsi 6:0:0:0: Direct-Access     ATA      MAXTOR STM310003 MX15 PQ: 0 ANSI: 5
[    2.354401] sd 6:0:0:0: [sdc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
-----------------------------------------------------------
[    2.354937] scsi 8:0:0:0: Direct-Access     ATA      ST31500541AS     CC34 PQ: 0 ANSI: 5
[    2.355790] sd 8:0:0:0: [sdd] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB)
-----------------------------------------------------------
[    2.355895] scsi 9:0:0:0: Direct-Access     ATA      MAXTOR STM310003 MX15 PQ: 0 ANSI: 5
[    2.356635] sd 9:0:0:0: [sde] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
-----------------------------------------------------------
[    2.811740] scsi 10:0:0:0: Direct-Access     ATA      ST31500341AS     CC1H PQ: 0 ANSI: 5
[    2.812103] sd 10:0:0:0: [sdf] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB)
-----------------------------------------------------------
[    3.210713] scsi 11:0:0:0: Direct-Access     ATA      ST31500541AS     CC34 PQ: 0 ANSI: 5
[    3.211137] sd 11:0:0:0: [sdg] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB)
-----------------------------------------------------------
[    3.600925] scsi 12:0:0:0: Direct-Access     ATA      WDC WD10EACS-00D 01.0 PQ: 0 ANSI: 5
[    3.601341] sd 12:0:0:0: [sdh] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
-----------------------------------------------------------
[   21.333709] md: bind<sdf1>
[   21.334160] md: bind<sdg1>
[   21.334504] md: bind<sdh1>
[   21.334817] md: bind<sdd1>
[   21.335197] md: bind<sdc1>
[   21.335474] md: bind<sde1>
[   22.902998] raid5: device sde1 operational as raid disk 0 +
[   22.903008] raid5: device sdc1 operational as raid disk 5 +
[   22.903016] raid5: device sdd1 operational as raid disk 4 *
[   22.903023] raid5: device sdh1 operational as raid disk 3 %
[   22.903031] raid5: device sdg1 operational as raid disk 2 *
[   22.903038] raid5: device sdf1 operational as raid disk 1 $
[   22.904902] raid5: allocated 6386kB for md0
[   22.905042] 0: w=1 pa=0 pr=6 m=2 a=2 r=6 op1=0 op2=0
[   22.905053] 5: w=2 pa=0 pr=6 m=2 a=2 r=6 op1=0 op2=0
[   22.905063] 4: w=3 pa=0 pr=6 m=2 a=2 r=6 op1=0 op2=0
[   22.905072] 3: w=4 pa=0 pr=6 m=2 a=2 r=6 op1=0 op2=0
[   22.905082] 2: w=5 pa=0 pr=6 m=2 a=2 r=6 op1=0 op2=0
[   22.905091] 1: w=6 pa=0 pr=6 m=2 a=2 r=6 op1=0 op2=0
[   22.905100] raid5: raid level 6 set md0 active with 6 out of 6 devices, algorithm 2
[   22.905108] RAID5 conf printout:
[   22.905113]  --- rd:6 wd:6
[   22.905122]  disk 0, o:1, dev:sde1
[   22.905130]  disk 1, o:1, dev:sdf1
[   22.905137]  disk 2, o:1, dev:sdg1
[   22.905145]  disk 3, o:1, dev:sdh1
[   22.905152]  disk 4, o:1, dev:sdd1
[   22.905159]  disk 5, o:1, dev:sdc1
[   22.905299] md0: detected capacity change from 0 to 4000808697856

Each of the raid elements have the same size of partition ,( ~1TB). This is seen by the check-sum of each of the image files.

9e43e11e04ac5d8f    1000202241024   raid_dev1.img
e1d810f9cea1cbff    1000202241024   raid_dev2.img
633e675b9b958a18    1000202241024   raid_dev3.img
8b881f07549fc7c9    1000202241024   raid_dev4.img
5727cefbc60af466    1000202241024   raid_dev5.img
1dacd8b59f896a85    1000202241024   raid_dev6.img

Each image is attached to a loop device, eg.

sudo for N in {1..6} ; do losetup -f --read-only raid_dev"${N}".img ; done

The loop devices are characterised by this from mdadm --examine;

/dev/loop12:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 00000000:00000000:00000000:00000000
  Creation Time : Fri May 18 04:32:55 2012
     Raid Level : -unknown-
   Raid Devices : 0
  Total Devices : 4
Preferred Minor : 127

    Update Time : Fri May 18 05:02:45 2012
          State : active
 Active Devices : 0
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 4
       Checksum : 82bfacf0 - correct
         Events : 1


      Number   Major   Minor   RaidDevice State
this     3       8       49        3      spare   /dev/sdd1

   0     0       8      177        0      spare
   1     1       8      145        1      spare   /dev/sdj1
   2     2       8       65        2      spare
   3     3       8       49        3      spare   /dev/sdd1
/dev/loop13:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : dc7b216a:36b68a65:c395db1e:b55a1d97
  Creation Time : Mon Nov  9 06:46:41 2009
     Raid Level : raid6
  Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
     Array Size : 3907039744 (3726.04 GiB 4000.81 GB)
   Raid Devices : 6
  Total Devices : 6
Preferred Minor : 0

    Update Time : Sun Apr 15 00:38:10 2012
          State : clean
 Active Devices : 6
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 0
       Checksum : a06dc07 - correct
         Events : 46

         Layout : left-symmetric
     Chunk Size : 4K

      Number   Major   Minor   RaidDevice State
this     1       8       33        1      active sync   /dev/sdc1

   0     0       8       17        0      active sync
   1     1       8       33        1      active sync   /dev/sdc1
   2     2       8       49        2      active sync   /dev/sdd1
   3     3       8       65        3      active sync
   4     4       8      113        4      active sync
   5     5       8       97        5      active sync   /dev/sdg1
/dev/loop14:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 00000000:00000000:00000000:00000000
  Creation Time : Fri May 18 04:32:55 2012
     Raid Level : -unknown-
   Raid Devices : 0
  Total Devices : 4
Preferred Minor : 127

    Update Time : Fri May 18 05:02:45 2012
          State : active
 Active Devices : 0
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 4
       Checksum : 82bfad4c - correct
         Events : 1


      Number   Major   Minor   RaidDevice State
this     1       8      145        1      spare   /dev/sdj1

   0     0       8      177        0      spare
   1     1       8      145        1      spare   /dev/sdj1
   2     2       8       65        2      spare
   3     3       8       49        3      spare   /dev/sdd1
/dev/loop15:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 00000000:00000000:00000000:00000000
  Creation Time : Fri May 18 04:32:55 2012
     Raid Level : -unknown-
   Raid Devices : 0
  Total Devices : 4
Preferred Minor : 127

    Update Time : Fri May 18 05:02:45 2012
          State : active
 Active Devices : 0
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 4
       Checksum : 82bfacfe - correct
         Events : 1


      Number   Major   Minor   RaidDevice State
this     2       8       65        2      spare

   0     0       8      177        0      spare
   1     1       8      145        1      spare   /dev/sdj1
   2     2       8       65        2      spare
   3     3       8       49        3      spare   /dev/sdd1
/dev/loop16:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 00000000:00000000:00000000:00000000
  Creation Time : Fri May 18 04:32:55 2012
     Raid Level : -unknown-
   Raid Devices : 0
  Total Devices : 4
Preferred Minor : 127

    Update Time : Fri May 18 05:02:45 2012
          State : active
 Active Devices : 0
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 4
       Checksum : 82bfad6a - correct
         Events : 1


      Number   Major   Minor   RaidDevice State
this     0       8      177        0      spare

   0     0       8      177        0      spare
   1     1       8      145        1      spare   /dev/sdj1
   2     2       8       65        2      spare
   3     3       8       49        3      spare   /dev/sdd1
/dev/loop17:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 00000000:00000000:00000000:00000000
  Creation Time : Wed May 16 07:23:49 2012
     Raid Level : -unknown-
   Raid Devices : 0
  Total Devices : 1
Preferred Minor : 127

    Update Time : Wed May 16 09:23:39 2012
          State : active
 Active Devices : 0
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 82baca2e - correct
         Events : 1


      Number   Major   Minor   RaidDevice State
this     0       8       17        0      spare

   0     0       8       17        0      spare

So now only one image has a UUID in its raid info.

I don't want to destroy the array, or have to re-image everything again.

To be honest, I can't really remember what I did back then, so I may be lost already. But I don't want to go trying more damaging tricks without getting a little advice first.

Can someone help me re-form the RAID6 array from these images, or is this a hopeless case?

Thanks,

JohnT
  • 21
  • 4
  • Welcome to the site. Please don't post additional information as comments; rather edit your question to include them. – AdminBee Feb 10 '20 at 13:45
  • metadata badly damaged. pretty much have to use overlays and recreate. https://unix.stackexchange.com/a/131927/30851 - either trial & error (start with disk order given in dmesg log, chunksize 4K, metadata 0.90), or derive layout from ondisk data. Difficult to write up a definitive answer as every situation is different and only you have the data to work with... – frostschutz Feb 10 '20 at 14:13
  • Hello JohnT. Do your disk image numbers `raid_dev1.img`..`raid_dev6.img` correspond directly to the original disks `sda`..`sdf`? If so, then provided you just created the RAID6 array using default values you _might possibly_ be able to rebuild the array. Otherwise you'll be looking at combinations... – roaima Feb 10 '20 at 14:13
  • But do I go with `mdadm --create` or `mdadm --assemble` or `mdadm --build` ? Will 'create' make any difference with read-only images? I did find a perl script that appears to attempt to **create** an array from every possible combination of 6 (or N) drives. I'm not sure what effect so many **create** attempts will have on the data. – JohnT Feb 10 '20 at 14:23
  • @ roaima, no I don't think that the image numbering corresponds with the device order. If I'd been more careful at the time I should have only flashed one drive at a time then recheck/rebuild the array. A Raid6 should have cope with that. – JohnT Feb 10 '20 at 14:26
  • --create with copy-on-write overlays as suggested in the linked answer (or some other snapshot-and-revert mechanism of your choice) – frostschutz Feb 10 '20 at 14:27
  • @frostschutz, Ah yes, that one. I've looked at the reference inside that post several times and always come away completely confused. I found that throwing the **parallel**ism into the mix simply obscures the useful information. I'll have a look again at building the overlays. I can then try and use the perl loop to reset the overlay after each unsuccessful cycle. This may take some time.... – JohnT Feb 10 '20 at 14:32
  • @JohnT the "Overlay manipulation functions" may be the easiest way to use it. There should be a standard tool for it... – frostschutz Feb 10 '20 at 14:34
  • As I said, it may be some time... I see that there are 720 different combinations of drive orders to try with the create. – JohnT Feb 10 '20 at 17:51

0 Answers0