3

I have observed empirically that for a raid1 mdadm array, mostly serving sequential reads from single processes, the first disk is basically always picked. This is also confirmed by this code which was linked online in this question.

To demonstrate the effect, see below iostat:

iostat -h | grep -E "sda|sdb"
     5.95         3.9M         5.9k         0.0k     **2.8T**     4.2G       0.0k sda <-- preferred
     0.38        12.1k         5.9k         0.0k       8.6G       4.2G       0.0k sdb

I would like to know if there is a non-destructive way to change that and decide which disk should be the preferred?

I tried changing the physical disk order, but the was the first disk still gets treated as preferred. During this process the RaidDevice and Number changed as expected but to no avail to what I was trying to achieve. Moreover, while the old sda changed to sdb and vice-versa, in mdadm --detail they didn't.

Following are some info of the setup:

$ mdadm -V
mdadm - v3.4 - 28th January 2016

Unbalanced reads before switch:

iostat -h | grep -E "sda|sdb"
     5.95         3.9M         5.9k         0.0k       2.8T       4.2G       0.0k sda <-- preferred
     0.38        12.1k         5.9k         0.0k       8.6G       4.2G       0.0k sdb

Unbalanced reads after switch:

$ iostat -h | grep -E "sda|sdb"
    11.60       427.9k        99.5k         0.0k     252.4M      58.7M       0.0k sda
   120.68       105.5M        99.5k         0.0k      62.2G      58.7M       0.0k sdb <-- still preferred

Details of array before switch:

cat /proc/mdstat 
md2 : active raid1 sdb3[0] sda3[2]
      13667560448 blocks super 1.2 [2/2] [UU]

mdadm --detail /dev/md2
/dev/md2:
        Version : 1.2
     Raid Level : raid1
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent
          State : clean 
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
           UUID : ddb468d7:8c866278:6f56b7eb:0a911253
    Number   Major   Minor   RaidDevice State
       0       8        3        0      active sync   /dev/sda3
       2       8       19        1      active sync   /dev/sdb3

Details of array after switch:

cat /proc/mdstat 
md2 : active raid1 sda3[0] sdb3[2]
      13667560448 blocks super 1.2 [2/2] [UU]

mdadm --detail /dev/md2
/dev/md2:
        Version : 1.2
     Raid Level : raid1
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent
          State : clean 
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
           UUID : ddb468d7:8c866278:6f56b7eb:0a911253
    Number   Major   Minor   RaidDevice State
       0       8       19        0      active sync   /dev/sdb3
       2       8        3        1      active sync   /dev/sda3 <- this partition name didn't get changed
JoeSlav
  • 128
  • 1
  • 7
  • 1
    Hmm, does it matter though? Or do you have a significant performance difference between the two? mdadm has the `--write-mostly` flag, but I think it's mostly meant for cases where the other drive is over a network, or you have an SSD+HDD. I'm not sure how useful it would be for disks that are relatively similar (but not identical) in performance. – ilkkachu Mar 27 '21 at 10:42
  • The issue is with reads and not writes. Writes are absolutely equal as that it's raid1 job (to keep the disks equal). Reads is terribly unbalanced. Imagine if you have processes (scrubs or manual) that check for bitrot or issues, they will basically only ever check 1 disk. Granted, this is implementation-specific, but the question here is more general for mdadm as I'm wondering if I can pilot this behavior. Thanks. – JoeSlav Mar 27 '21 at 12:19
  • `--write-mostly` means "do not read here". but it's not really what you want as it would cut your raid performance in half (with concurrent readers). for your request you'd have to change the raid device number / order in metadata. there is no direct/official way to do so other than re-creating or directly editing metadata... for disk checking (parity mismatches) you'd have to use mdadm's own check facilities – frostschutz Mar 27 '21 at 14:07
  • 1
    if you decide to go the re-create route see also https://unix.stackexchange.com/a/131927/30851 - if done correctly it would be non-destructive and instant (safe to --assume-clean if you umount, stop, create, mount) but the risk is getting any of the values wrong or having undetected parity mismatches – frostschutz Mar 27 '21 at 14:08

0 Answers0