Subject: RFC: Read repair for md RAID1 after mirror read failures
"G.W. Kant - Hunenet B.V." <[email protected]>
| Newsgroups | gmane.linux.raid |
|---|---|
| Message-ID | <[email protected]> |
Hello list, Recently I encountered an interesting failure mode while migrating data from an aging backup system. The source Btrfs filesystem spanned several LVM logical volumes, each backed by an md RAID1 array. One of these logical volumes resided on a degraded RAID1 array, so I added a new logical volume on a new RAID1 array and started migrating the data using: btrfs device remove During the migration, many read requests encountered unrecoverable read errors (UNC) on the remaining member of the degraded RAID1 array. The migration continued, which is exactly what I had hoped for, since only a small fraction of the media appeared to be affected. This made me wonder whether md RAID1 has, or has ever considered, a read repair mechanism. Consider the following situation: mirror A: read -> UNC mirror B: read -> OK During a RAID1 read request, if md can satisfy the read from the alternate mirror, no data is lost. However, this also represents an opportunity to repair the degraded copy. Once a read has failed on one mirror, the array has effectively lost its ability to tolerate a second read failure for that logical block. This window persists until the affected block is rewritten by normal filesystem activity, which may never happen on cold archival data. Repairing the degraded copy immediately may restore full redundancy while a valid copy is still available. Writing the recovered block back to the the corresponding block on the failed mirror would give the drive an opportunity to recover that location, for example by successfully rewriting the sector or remapping it if the write cannot be verified. In other words, the first successfully recovered read request could automatically become a repair opportunity. The repair could even be scheduled asynchronously, so the successful read is returned immediately while the rewrite is performed in the background. Unlike a periodic resync, this repair would be driven by an actual read failure, making it targeted rather than rewriting the entire mirror. With today's 18–24 TB HDDs and backup/archive workloads, where data may remain unchanged for years, latent media degradation seems increasingly relevant. A successful read from the alternate mirror may be one of the last opportunities to refresh such a sector before it becomes permanently unreadable. One advantage of such an approach is that it does not require md to decide between two conflicting copies. One mirror has already reported an unrecoverable read error, while the other has successfully reconstructed the requested block. The proposal only applies to this specific case. Has this idea been discussed before, or is there a reason why md deliberately avoids this type of read repair? Regards, Dion Kant