Re: block size problem ?

"Also sprach denis bonnenfant:"
> But now, another enbd problem :  sometimes, after connecting, nda device is 
> totally unuseable :  i have "buffer IO errors", and device is not accessible.

Well, I guess it's not connected then! What does nbdinfo say?

> I tried to stop client and server, restart it, but nothing changes.

I'm sure it does, but in any case, restarting the client may not be a
good idea without a picture of what is happening.

> removing module i've got an oops.

I think one frequently gets an oops on kernel 2.6 when removing ANY
module (or such is my experience) - sme kind of race condition. Please
don't do that then for the moment.

> Rebooting solves sometimes the problem

If rebooting solves the problem, then it is a kernel problem, so you
want to show me kernel messages.

> here are the logs clientside : corresponding to enbd stop and and enbd start
> stop :

What kernel is this?

> ENBD #3134[5]: enbd_disable disabled device nda

Well, this is the fifth time the device has been disabled, so it would
perhaps be better to show the first time through ...

> ENBD #2194[28]: enbd_clr_queue unqueued 0 reqs
> ENBD #3224[22]: enbd_soft_reset INVALIDATE DEVICE nda
> ENBD #3234[22]: enbd_soft_reset run enbd_request on nda
> ENBD #2194[29]: enbd_clr_queue unqueued 0 reqs
> ENBD #3224[23]: enbd_soft_reset INVALIDATE DEVICE nda
> ENBD #3234[23]: enbd_soft_reset run enbd_request on nda
> ENBD #2194[30]: enbd_clr_queue unqueued 0 reqs
> ENBD #3224[24]: enbd_soft_reset INVALIDATE DEVICE nda
> ENBD #3234[24]: enbd_soft_reset run enbd_request on nda
> ENBD #1805[22]: enbd_get_req Our slot has been vamooshed (result -19).
> ENBD #1805[23]: enbd_get_req Our slot has been vamooshed (result -19).
> ENBD #1805[24]: enbd_get_req Our slot has been vamooshed (result -19).
> ENBD #1805[25]: enbd_get_req Our slot has been vamooshed (result -19).
> ENBD #2194[31]: enbd_clr_queue unqueued 0 reqs

More or less harmless. Timing info from the syslog would be nice!

> ENBD #3224[25]: enbd_soft_reset INVALIDATE DEVICE nda
> ENBD #3234[25]: enbd_soft_reset run enbd_request on nda
> ENBD #1805[26]: enbd_get_req Our slot has been vamooshed (result -19).
> ENBD #2194[32]: enbd_clr_queue unqueued 0 reqs
> ENBD #3224[26]: enbd_soft_reset INVALIDATE DEVICE nda
> ENBD #3234[26]: enbd_soft_reset run enbd_request on nda
> ENBD #1805[27]: enbd_get_req Our slot has been vamooshed (result -19).
> ENBD #2194[33]: enbd_clr_queue unqueued 0 reqs
> ENBD #2133[2]: enbd_clr_kernel_queue removed 0 requests
> ENBD #2535[11]: enbd_enable set VALID on nda
> ENBD #3700[42]: fixup_slot failed to find slot for pid 6557 for ioctl 4004ab04 

Well, that's interesting. It says that adaemon that is not registered
tried to talk to the device. Maybe it's just the register itself. I'd
have to look up the ioctl tofind out ...

> arg 199d
> ENBD #3850[42]: enbd_ioctl CLEAR_SOCK called on full device nda arg 199d
> ENBD #4234[71]: enbd_release (3): erasing slot pid 6557
> ENBD #3700[43]: fixup_slot failed to find slot for pid 6555 for ioctl 4004ab04 
> arg 199b
> ENBD #3850[43]: enbd_ioctl CLEAR_SOCK called on full device nda arg 199b
> ENBD #4234[72]: enbd_release (1): erasing slot pid 6555
> ENBD #4234[73]: enbd_release (2): erasing slot pid 6556
> ENBD #2194[34]: enbd_clr_queue unqueued 0 reqs
> ENBD #3224[27]: enbd_soft_reset INVALIDATE DEVICE nda
> ENBD #3234[27]: enbd_soft_reset run enbd_request on nda
> ENBD #4234[74]: enbd_release (0): erasing slot pid 6554
> ENBD #3767[27]: fixup_slot failed to find slot for pid 6851 ioctl 4004ab10 arg 

Well, again, it says that an unregistered daemon is trying to talk to
the device. The nbdinfo output would be informative.


> ENBD #2535[12]: enbd_enable set VALID on nda
> ENBD #824[105]: enbd_open DISK CHECK wanted in open on device nda
> ENBD #4404[105]: enbd_media_changed MEDIA CHANGED called
> ENBD #4410[105]: enbd_media_changed REMOTE CHECK done locally, no remote ioctl!

It says that you haven't loaded the remote ioctls module?

> ENBD #4508[200]: enbd_revalidate REVALIDATE called on nda
> ENBD #4514[190]: enbd_revalidate asked to revalidate remotely dead drive nda
> ENBD #4508[201]: enbd_revalidate REVALIDATE called on nda
> ENBD #4514[191]: enbd_revalidate asked to revalidate remotely dead drive nda
>   nda:remote device invalid.

It certainly thinks the other side is not there. But then without
remote ioctls enabled it can only guess.

> Jul  7 18:05:15 se3 enbd-client: enbd-client: client (0) begins main loop
> Jul  7 18:05:15 se3 kernel: ENBD #2535[12]: enbd_enable set VALID on nda
> Jul  7 18:05:15 se3 enbd-client: enbd-client: client (1) got a signature ok from 

This looks fine.

> Jul  7 18:05:18 se3 kernel: ENBD #824[105]: enbd_open DISK CHECK wanted in open 
> on device nda
> Jul  7 18:05:18 se3 kernel: ENBD #4404[105]: enbd_media_changed MEDIA CHANGED called
> Jul  7 18:05:18 se3 kernel: ENBD #4410[105]: enbd_media_changed REMOTE CHECK 
> done locally, no remote ioctl!

Well, again, you don't have the remote ioctls module loaded, I think.

> Jul  7 18:05:18 se3 kernel: ENBD #4514[191]: enbd_revalidate asked to revalidate 
> remotely dead drive nda
> Jul  7 18:05:18 se3 kernel:  nda:remote device invalid.

It certainly thinks the other side is not healty, but without the
remote ioctls module it can only guess.

> here /proc/nbdinfo
> 
> Device a:       Open
> [a] State:      verify, rw, merge requests, enabled, remote invalid, last error 

Yes, well it says the other side is dead.

> [a] Cpids:      4       (6852)  (6853)  (6854)  (6855)

These are the pids registered - they don't match. I suppose they might
be dying off and respawning. Anyway, make sure to load the remote
ioctls module.

> and oops after module removing  :

Don't do that then.

> Jul  4 15:32:55 se3 kernel: Buffer I/O error on device nda, logical block 536870640
> Jul  4 15:32:59 se3 kernel: ENBD #5287[0]: enbd_cleanup module cleaned up.
> Jul  4 15:32:59 se3 kernel: Unable to handle kernel paging request at virtual 
> address f8922cf8

Well, memory address.

> c17e5480 f39de480 c17e1300
> Jul  4 15:32:59 se3 kernel: Call Trace:
> Jul  4 15:32:59 se3 kernel:  [exact_lock+10/28] exact_lock+0xa/0x1c
> Jul  4 15:32:59 se3 kernel:  [kobj_lookup+191/428] kobj_lookup+0xbf/0x1ac
> Jul  4 15:32:59 se3 kernel:  [get_gendisk+21/44] get_gendisk+0x15/0x2c
> Jul  4 15:32:59 se3 kernel:  [do_open+53/784] do_open+0x35/0x310
> Jul  4 15:32:59 se3 kernel:  [blkdev_open+30/96] blkdev_open+0x1e/0x60
> Jul  4 15:32:59 se3 kernel:  [dentry_open+252/500] dentry_open+0xfc/0x1f4
> Jul  4 15:32:59 se3 kernel:  [filp_open+82/92] filp_open+0x52/0x5c
> Jul  4 15:32:59 se3 kernel:  [get_unused_fd+80/180] get_unused_fd+0x50/0xb4
> Jul  4 15:32:59 se3 kernel:  [sys_open+55/112] sys_open+0x37/0x70
> Jul  4 15:32:59 se3 kernel:  [syscall_call+7/11] syscall_call+0x7/0xb

It's not in my code. Never entered it! The problem is probably that
there is a missing get in a put/get pair for the device. Thanks. That's
probably the most informative trace I've ever seen for that problem.


> Oh yes, another thing too : this is with the other device that works correctly 
> in fr1 array : syslog is blown continuously with these messages :
> 
> Jul  8 06:28:03 se3 enbd-client: enbd-netserver: seeing if (0) can discard one 
> of 1 write acks pending
> Jul  8 06:28:03 se3 enbd-client: enbd-netserver: (1) yes it can, result 0

Well, probably debugging. How come it is doing discards? Oh - you have
the async flag set on the client, and you have compiled with DEBUG set.

     // remove all pending wracks from the stream in case we're async
     while ((self->flags & ENBD_CLIENT_ASYNC)
       && (self->flags & ENBD_CLIENT_WRPLY_PEND)) {
       // PTB get (for discard) last times pending ack
       err = recv_reply(self, &rpl, buf, handle, WRITE, 0, -1, 0);
       DEBUG("seeing if (%d) can discard one of %d write acks pending\n",
             self->i, 1);
       if (err >= 0) {
         self->flags &= ~ENBD_CLIENT_WRPLY_PEND;
         DEBUG("(%d) yes it can, result %d\n", self->i, err);
       } else {
         DEBUG("(%d) no it can't, result %d\n", self->i, err);
         // PTB wait one jiffy
         microsleep(50000);
         // PTB we hope to cause a timeout eventually
         // PTB same as a goto fail?
       }
     }


Recompile without DEBUG set if you don't like those messages (or maybe
the archived code has it set?), and probably try without the async
flag on the client. 

Peter