How to check mdadm RAID5 integrity after power failure/random reboot

Question

So I'm currently building an mdadm RAID5 array attached to my home server. The hardware is an Odroid N2 SBC with a Mediasonic Probox 4 bay enclosure attached. The array is currently rebuilding and has been for days but moving steadily. I'm using armbian stretch with the legacy 4.9.180 kernel.

Last night, I was using the system (but not the drives) and was running a checksum on a file on a different USB drive. There is currently an unsolved bug in the USB drivers for the N2 that is exacerbated by high I/O activity. The N2 subsequently died around 11:40pm last night.

The N2 came back almost immediately and I didn't even notice until morning. However, the mdadm array rebuild was paused at 75%. I resumed the rebuild and it's progressing happily, but I want to be sure that I didn't do lasting harm to the new array.

Is there any mdadm utility that I can use to confirm there are no errors in the parity data? There is no filesystem on the array so I don't think I can use fsck in this case

Answer 1

(Once the current rebuild is finished,) you can run a check:

mdadm --wait /dev/mdX # wait for rebuild to finish
mdadm --action=check /dev/mdX
# or if mdadm is too old:
echo check > /sys/block/mdX/md/sync_action

and then watch the mismatch_cnt:

watch cat /sys/block/mdX/md/mismatch_cnt

as long as it stays 0, the parity is fine.

See also man md, SCRUBBING AND MISMATCHES.

   A  count  of  mismatches is recorded in the sysfs file md/mismatch_cnt.
   This is set to zero when a scrub starts and is incremented  whenever  a
   sector  is  found  that is a mismatch.  md normally works in units much
   larger than a single sector and when it finds a mismatch, it  does  not
   determine exactly how many actual sectors were affected but simply adds
   the number of sectors in the IO unit that was used.  So a value of  128
   could  simply  mean  that  a  single  64KB  check found an error (128 x
   512bytes = 64KB).

This process will take as long as the rebuild itself... as it's basically doing the same thing as a rebuild. For progress, refer to /proc/mdstat.

It's also possible to test a specific region only — if you only want to check around the 75% mark — but it's more complicated as (I think) there's no command option in mdadm for it. You can set md/sync_min, md/sync_max to determine a range (default range 0-max covers the entire device).

If you want parity to be fixed, instead of the purely informative check, use repair which fixes parity. However you have to be sure that data is correct, and parity incorrect. Otherwise if you can identify a single disk that has incorrect data (regardless whether that's data or parity), you have to remove the disk and add it as new disk and rebuild again.

Determining the correct course of action for mismatch handling can unfortunately be quite complicated...