Re: How to salvage a degraded mdadm RAID1 with as little data loss as possible?

Am 22.06.26 um 15:30 schrieb Andy Smith:
> Hi,
>
> The lack of an mdadm.conf should not cause you any issues. It's only
> really used to set non-default options, give a monitoring email address
> and so on.

O.k. I added an entry to mdadm.conf anyway, it shouldn't hurt at least.

> udev incrementally assembles MDADM arrays as devices appear. It does not
> need any configuration to do this. In order to end up in the situation
> OP is in, I can only imagine that they rebooted and only one of the
> LUKS devices was set up, so md0 proceeded in a degraded fashion with
> that device.
>
> What is confusing to me is how OP had an active mdadm array member with
> an event count significantly *behind* the inactive one. It makes me
> think that this may have happened more than once, with different single
> LUKS devices being activated each time.
The switch of active and inactive devices happened definitely more than once. The md array most likely switched between the LUKS devices at boot several times, therefore the different event counts. The device with newer data of course had the higher event count, as it was the one the data had been written on in the weeks before the latest switch. My best guess is also that something happened while the LUKS devices have been created which made md believe that one device is not intact or available.
> The mdadm monitor daemon runs by default and should email you about
> degraded arrays. Without any configuration that would be sending to
> root@localhost. OP should make sure that these emails will arrive
> somewhere useful, or look into other ways of checking status of mdadm
> arrays. What's happened here was likely trivial to fix at the
> time of first problem but became a complete nightmare that likely
> involved data loss (OP has backup of a device with unique data that
> cannot be integrated).

Oh well, experience is never too expensive, a German saying goes... This issue will not bug me again as much as it did. The next time I curse, it will be due a different issue, I am sure. :-)

> OP, after sorting out the monitoring I think you need to verify that
> both LUKS devices are always successfully unlocked and available at boot
> so that the RAID 1 assembles fully and properly.
mdmon is now set up, I tested that e-mails actually arrive.Latest news: The md array survived a first reboot today.
> I think it's unlikely that you have had a hardware failure of the
> underlying drives, though you should of course check your logs and
> smartctl for that. Given that LUKS is in use and is the most complicated
> thing in your storage stack, I'd be looking into whether both LUKS
> devices are being reliably created.
O.k.
> If setting this system up from scratch my preference would be to do the
> redundancy as near to the hardware as possible and the encryption as far
> away as possible. So I'd put LUKS on md0, not md0 on two LUKS devices.

Thank you very much for your advice!

Paul