Re: Re: General question

On Tue, 7 Jun 2005 20:24:23 +0200 (MET DST)
"Peter T. Breuer" <[email protected]> wrote:

> "Also sprach Stephan von Krawczynski:"

> > It does not, I checked that. Basic reason is:
> > 
> > Jun  7 18:50:15 mail-a01 kernel: md: ndb's event counter: 00000000
> > Jun  7 18:50:15 mail-a01 kernel: md: nda's event counter: 00000000
> 
> Unusual. Practically anything increments the event counter. Even
> starting raid! So 0 is not a usual number to see.

Ok, I looked for the reason and it is pretty simple: I did not use a persistent
superblock. This seems to lead to eventcounters being zero when mounting.

I changed the setup now to use mdadm (like you) and everything looks a bit
different now. The eventcounters look quite like you expected them.

Now I ran into a trivial problem that I do not understand. Following steps:

- creating a raid1 with two enbd-devices and fr1
- copy some data on it
- umount the raid and stop it
- mount _one_ enbd drive as _local_ raid1 fs with a non existent second mirror.
- copy data on it
- umount it and stop the local raid
- assemble the original network raid again

Now, what happens is that during assemble md find out that one mirror is newer
(the one that was used as local raid fs) and takes this.
But it does not rebuild the array, instead degrades it:

Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5806[23]: enbd_media_changed change nda requested
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5813[23]: enbd_media_changed REMOTE CHECK done locally, no remote ioctl!
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5806[24]: enbd_media_changed change ndb requested
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5813[24]: enbd_media_changed REMOTE CHECK done locally, no remote ioctl!
Jun  8 12:10:07 mail-a01 kernel: nda (read) [events: 0000000e]
Jun  8 12:10:07 mail-a01 kernel: md: bind<nda,1>
Jun  8 12:10:07 mail-a01 kernel: md0: notifying dev 2b00
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5568[13]: enbd_ioctl received BLKMDNTFY, am now in raid 900
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5583[13]: enbd_ioctl set show_errs on nda
Jun  8 12:10:07 mail-a01 kernel: ndb (read) [events: 00000010]
Jun  8 12:10:07 mail-a01 kernel: md: bind<ndb,2>
Jun  8 12:10:07 mail-a01 kernel: md0: notifying dev 2b10
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5568[14]: enbd_ioctl received BLKMDNTFY, am now in raid 900
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5583[14]: enbd_ioctl set show_errs on ndb
Jun  8 12:10:07 mail-a01 kernel: md: ndb's event counter: 00000010
Jun  8 12:10:07 mail-a01 kernel: md: nda's event counter: 0000000e
Jun  8 12:10:07 mail-a01 kernel: md: superblock update time inconsistency -- using the most recent one
Jun  8 12:10:07 mail-a01 kernel: md: freshest: ndb
Jun  8 12:10:07 mail-a01 kernel: md: kicking non-fresh nda from array!
Jun  8 12:10:07 mail-a01 kernel: md0: unnotifying dev 2b00
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5592[13]: enbd_ioctl received BLKMDUNTFY, now out of raid 900
Jun  8 12:10:07 mail-a01 kernel: ENBD enbd.c #5609[13]: enbd_ioctl cleared show_errs on nda
Jun  8 12:10:07 mail-a01 kernel: md: unbind<nda,1>
Jun  8 12:10:07 mail-a01 kernel: md: export_rdev(nda)
Jun  8 12:10:07 mail-a01 kernel: md: device name has changed from sda4 to ndb since last import!

Aha, it found out I mounted ndb as local sda4 raid component, so what ...

Jun  8 12:10:07 mail-a01 kernel: md0: max total readahead window set to 124k
Jun  8 12:10:07 mail-a01 kernel: md0: 1 data-disks, max readahead per data-disk: 124k
Jun  8 12:10:07 mail-a01 kernel: raid1: device ndb operational as mirror 1
Jun  8 12:10:07 mail-a01 kernel: raid1: md0, not all disks are operational -- trying to recover array
Jun  8 12:10:07 mail-a01 kernel: raid1: raid set md0 active with 1 out of 2 mirrors
Jun  8 12:10:07 mail-a01 kernel: md: updating md0 RAID superblock on device
Jun  8 12:10:07 mail-a01 kernel: md: ndb [events: 00000011]<6>(write) ndb's sb offset: 987904
Jun  8 12:10:07 mail-a01 kernel: md: recovery thread got woken up ...
Jun  8 12:10:07 mail-a01 kernel: md: recovery thread finished ...


Now, how can I rebuild this array?
Hotadd does not work:

Jun  8 12:20:12 mail-a01 kernel: nda (read) [events: 00000016]
Jun  8 12:20:12 mail-a01 kernel: md0: new disk 2b00 too old for repair (disk 22 < bitmap 0)
Jun  8 12:20:12 mail-a01 kernel: md0 (read) [events: 00000017]
Jun  8 12:20:12 mail-a01 kernel: md: bind<nda,2>
Jun  8 12:20:12 mail-a01 kernel: md0: notifying dev 2b00
Jun  8 12:20:12 mail-a01 kernel: ENBD enbd.c #5568[20]: enbd_ioctl received BLKMDNTFY, am now in raid 900
Jun  8 12:20:12 mail-a01 kernel: ENBD enbd.c #5583[20]: enbd_ioctl set show_errs on nda
Jun  8 12:20:12 mail-a01 kernel: RAID1 conf printout:
Jun  8 12:20:12 mail-a01 kernel:  --- wd:1 rd:2 nd:0
Jun  8 12:20:12 mail-a01 kernel:  disk 0, s:0, o:0, n:0 rd:0 us:1 dev:[dev 00:00]
Jun  8 12:20:12 mail-a01 kernel:  disk 1, s:0, o:1, n:1 rd:1 us:1 dev:ndb
Jun  8 12:20:12 mail-a01 kernel:  disk 2, s:1, o:0, n:2 rd:2 us:0 dev:[dev 00:00]
Jun  8 12:20:12 mail-a01 kernel:  disk 3, s:0, o:0, n:0 rd:0 us:0 dev:[dev 00:00]
Jun  8 12:20:12 mail-a01 kernel: RAID1 conf printout:
Jun  8 12:20:12 mail-a01 kernel:  --- wd:1 rd:2 nd:1
Jun  8 12:20:12 mail-a01 kernel:  disk 0, s:0, o:0, n:0 rd:0 us:1 dev:[dev 00:00]
Jun  8 12:20:12 mail-a01 kernel:  disk 1, s:0, o:1, n:1 rd:1 us:1 dev:ndb
Jun  8 12:20:12 mail-a01 kernel:  disk 2, s:1, o:0, n:2 rd:2 us:1 dev:nda
Jun  8 12:20:12 mail-a01 kernel:  disk 3, s:0, o:0, n:0 rd:0 us:0 dev:[dev 00:00]
Jun  8 12:20:12 mail-a01 kernel: md0: set repair bit to 0 on superblock
Jun  8 12:20:12 mail-a01 kernel: md: updating md0 RAID superblock on device
Jun  8 12:20:12 mail-a01 kernel: md: nda [events: 00000018]<6>(write) nda's sb offset: 987904
Jun  8 12:20:12 mail-a01 kernel: md: ndb [events: 00000018]<6>(write) ndb's sb offset: 987904
Jun  8 12:20:12 mail-a01 kernel: md: recovery thread got woken up ...
Jun  8 12:20:12 mail-a01 kernel: md0: no spare disk to reconstruct array! -- continuing in degraded mode
Jun  8 12:20:12 mail-a01 kernel: md: recovery thread finished ...

I really would have expected that an array is simply rebuilt using the latest superblock info, not?

-- 
Regards,
Stephan