Re: enbd-fr1 on 2.6.11.12 problems

Well, there is a little problem :
when an array is started with an enbd component, it can't be stopped.

I can manually fail nda1, and then stop the array,
but when there is an event causing  enbd-client to fail nda1 disk, I can't stop 
md0, mdadm says that device is busy, even if enbd-client reconnects and the nda1 
disk gets reconstructed.

I tried to force enbd module removing with echo 0>/proc/mdstat, and echo 1 too, 
but it doesn't works


enbd-client is started like this :
enbd-client ldap-croise 12345 -i A -n 4 -m /dev/nda

server like that :
enbd-server 12345 -i A

my /proc/nbdinfo :

Device a:       Open
[a] State:      verify, rw, merge requests, enabled, show_errs, last error 0, 
lives 1, bp 0
[a] Queued:     +0R/0W curr (check 0R/0W) +4R/155W max
[a] Buffersize: 262144  (sectors=512, blocks=64)
[a] Blocksize:  4096    (log=12)
[a] Size:       104856KB
[a] Blocks:     26214
[a] Sockets:    4       (*)     (+)     (+)     (+)
[a] Requested:  56.732K (13.6K) (13.4K) (14.2K) (15.3K) 299R/56.44KW    max 31
[a] Despatched: 56.727K (13.6K) (13.4K) (14.2K) (15.3K) 299R/56.43KW    md5 0W 
(0 eq, 0 ne, 0 dn)
[a] Errored:    388     (1)     (1)     (1)     (1)     4+384
[a] Pending:    0       (0)     (0)     (0)     (0)     0R/0W+0R/0W
[a] B/s now:    0       (0R+0W)
[a] B/s ave:    144K    (0R+144KW)
[a] B/s max:    145M    (1.00MR+144MW)
[a] Spectrum:   16%1    38%3    44%16
[a] Kthreads:   0       (0 waiting/0 running/1 max)
[a] Cthreads:   4       (+)     (+)     (+)     (+)
[a] Cpids:      4       (3846)  (3847)  (3848)  (3849)
Device b-p:     Closed






*Here I stop enbd-server :

md: errors occurred during superblock update, repeating
md: write_disk_sb failed for device nda1
*... 200 times...
md: errors occurred during superblock update, repeating
md: write_disk_sb failed for device nda1
md: excessive errors occurred during superblock update, exiting

*Strange ? normally md has been notified that enbd-server is down ?

raid1: Disk failure on nda1, disabling device.
         Operation continuing on 1 devices
raid1: made bitmap c5c55e00 at events 0:21c

*here server comes back :

md0: md_check_recovery 2: repair bit 0 on sb c5d14400 preserved
warning: process `enbd-client' used the obsolete bdflush system call
Fix your initscripts?
warning: process `enbd-client' used the obsolete bdflush system call
Fix your initscripts?
ENBD #3767[1]: fixup_slot failed to find slot for pid 3832 ioctl 4004ab10 arg bf
e4f598 (41)
ENBD #2535[1]: enbd_enable set VALID on nda
ENBD #74[2]: enbd_notify_md_device notifying 2344 for 2b:0 to raid devices via f
n
ENBD #81[1]: enbd_notify_md_device ioctl 2344 to raid devices returned -22
ENBD #74[3]: enbd_notify_md_device notifying 2344 for 2b:1 to raid devices via f
n
md0: repair of faulty disk 2b00001!
md0: notifying dev 2b00001 it is no longer in array
ENBD #4162[7]: enbd_ioctl received BLKMDUNTFY, now out of raid 900000
md: unbind<nda1>
md: export_rdev(nda1)
md 1243: unlock_rdev nulls bdev
md: WARNING: delaying free of exported rdev c5b51580
md 1233: lock_rdev restores bdev 2b00001 rdev(c5794dc0)->bdev(cfee0580)
nda1 (read) [events: 000000f0]
md: old uuid 8b050960 a29a0ec0 39a9072f 40e47ee4
md: new uuid 8b050960 a29a0ec0 39a9072f 40e47ee4
md0: new disk 2b00001 too old for repair (disk 240 < bitmap 543)

*really strange ! why so much events? write errors above? nothing special happens !

md: bind<nda1>
md0: notifying dev 2b00001 it is now in array
ENBD #4143[8]: enbd_ioctl received BLKMDNTFY, am now in raid 900000
ENBD #4152[1]: enbd_ioctl set show_errs on nda
md0: set repair bit to 0 on superblock c5d14400
md0: md_check_recovery 2: repair bit 0 on sb c5d14400 preserved
raid1: testing p->rdev c50dd940
raid1: testing p->rdev->bdev cfee03c0
raid1: testing p->rdev->bdev->bd_dev 800001
raid1: testing p->rdev c5b51580
raid1: testing p->rdev->bdev 00000000
raid1: late free of exported rdev c5b51580
RAID1 conf printout:
  --- wd:1 rd:2
  disk 0, wo:0, o:1, dev:sda1
  disk 1, wo:1, o:1, dev:nda1
.<6>md: syncing RAID array md0)
md: minimum _guaranteed_ reconstruction speed: 100000 KB/sec/disc.
md: using maximum available idle IO bandwith (but not more than 200000 KB/sec) f
or reconstruction.
md: using 128k window, over a total of 97152 blocks.
md0: no repair bit on sb so removed bitmap c5c55e00
raid1: synced dirty sectors 0-194303
raid1: synced dirty sectors 0-194431
md: md0: sync done.
bitmap: 3 pages in use
bitmap: 3 pages allocated
bitmap: 0 pages pre-allocated
RAID1 conf printout:
  --- wd:2 rd:2
  disk 0, wo:0, o:1, dev:sda1
  disk 1, wo:0, o:1, dev:nda1
md0: md_check_recovery 1: repair bit 0 on sb c5d14400 preserved
md0: md_check_recovery 2: repair bit 0 on sb c5d14400 preserved

*end of reconstruction
*run mdadm /dev/md0 --fail /dev/sda1

raid1: Disk failure on sda1, disabling device.
         Operation continuing on 1 devices
raid1: made bitmap c5c55e00 at events 0:225
md0: md_check_recovery 2: repair bit 0 on sb c5d14400 preserved

*run mdadm --stop /dev/md0

md: md0 still in use.



I got this on the server  :



Jun 22 13:27:50 ldap enbd-server: enbd-server  2962: <#1414> do_srv_write 
request for nonblock (4096) length at 0x2000 (8192) seqno 6680 len 512
Jun 22 13:27:50 ldap enbd-server: enbd-server  2962: <#1414> do_srv_write 
request for nonblock (4096) length at 0x2000 (8192) seqno 6681 len 512
Jun 22 13:27:50 ldap enbd-server: enbd-server  2962: <#1414> do_srv_write 
request for nonblock (4096) length at 0x2008000 (33587200) seqno 6682 len 1024