2

data scrubbing is presented to me as a way to mitigate what is refered to as bit rot (on mass storage).

The rationale is that data on hard disks can "decay"/"rot" and hence get corrupted, and to prevent this hard disk producers, have the discs internally add some additional correction data for each block writen. A bit rot occurs when the data and its acompanying correction/protection data (some sort of error correction code ECC) is unable to correct the data found on a drive.

Now, my understanding/information at present is that, over time data (and its added correction info) is lost gradually. This means the longer you wait, the more bit-rot happens internally until cascadingly a point is reached in which no correction is possible (that is the internal added corrected data, does not offer enough info for repair anymore).

The data scrubbing seems to help mitigate the bit rot. I imagine it to be idealy internally a reading of all the data of the drive and a checking of the data with the additional error correction data. If while the test the drive encounteres a problem it can correct the error. Also all internally bit-rot bytes (that do not yet couse a real URE) can be set back i a better state.

This all being said, how would I trigger the device to do such a data scrubbing?

Is it smartctl, or hdparm?

With smartctl there are 3 types of test: conveyence, short, long/extended. That only one that really makes sense to me is long, as it obviously would take time to read all the data of the drive to enable a potential repair.

A question that targets this with respect to a RAID 1 found here: bit rot detection and correction with mdadm

I want to know how to do it, when there is no RAID there, just a plain SATA drive.

roaima
  • 107,089
  • 14
  • 139
  • 261
humanityANDpeace
  • 13,722
  • 13
  • 61
  • 107
  • Sorry, I misunderstood the concept. You'd need some software that does checksumming of files and is not bothered on intended changes in the files... Something like [par2](https://github.com/Parchive/par2cmdline) (see ["Using par2cmdline"](https://github.com/Parchive/par2cmdline#using-par2cmdline) to see how it works) – zagrimsan Sep 08 '15 at 11:52
  • @zagrimsan par2 seems interesting, though I look for something else. On modern harddisks data is stored in sectors of 4096 bits use data + 100bits ECC data (might be manufactorer dependet). For many reasons bits in the sectors data might flip, and up to some level the ECC will allow to tolerate this. I thought data scrubbing is "read all data, and rewrite it in a cleaner way, that is without the yet correctable bit flips that could be recovered/corrected via ECC" – humanityANDpeace Sep 08 '15 at 11:57
  • 1
    I didn't read the wikipedia article, my bad, but looking at it gives some pretty [good tips for you](https://en.wikipedia.org/wiki/Data_scrubbing#File_systems), like using brtfs or ZFS filesystem. A dumb solution would be to have a script which iterates over all files, copies each one (with verification) to a temp location and then replaces the original with the copy. The downside with any dumb solution for this, I think, is that it introduces a new source of corruption (copying stuff back-and-forth) and interferes with the normal use of the processed files... – zagrimsan Sep 08 '15 at 12:11
  • SMART tests only perform reads, so they are of no use here, but maybe `badblocks -n` could do the trick as suggested [here](http://superuser.com/a/693065/243625). I had a feeling that I've seen some regular tool mentioned for this kind of purpose but just couldn't remember it before... – zagrimsan Sep 08 '15 at 12:17
  • @zagrimsan I know to little about and can only guess about the genesis of problems of bit rots in sectors. I hoped that the hard drives could have a "read all sectors in, verify them, and in case of many bit flip, rewrite them". For this idea to work I would start thinking that the bit flips are actually gradually accumulating over time, so that regularly "integrity checking, and correction" would be the thing to do. – humanityANDpeace Sep 08 '15 at 12:18

1 Answers1

0

Reading whole plain drive:

dd if=/dev/sd(x) of=/dev/null iflag=nocache status=progress

may help determining errors.

The badblocks(8) with -n option is the closest. Unmount drive first.

AdminBee
  • 21,637
  • 21
  • 47
  • 71
fich
  • 1
  • 1