Re: General question

Stephan von Krawczynski <[email protected]> wrote:
> > Are you sure that you are using the NEW md module? That's the part that
> > is responsible for telling the enbd module that it is in a raid array,
> > and for reacting to a network error message from enbd.
> > 
> > > :~ # lsmod
> > > Module                  Size  Used by    Not tainted
> > > fr1                    18356   1 
> > > bitmap                  6416   0  [fr1]
> > > md                     49664   2  [fr1]
> > > enbd                   71788   8 

> It wouldn't open fr1 module if it were the wrong version of md, not?

It would load and run fine either way, as far as I know. The change to
md is to allow the hotrepair ioctl/technique plus a notification
mechanism (again ioctl) for device error.

> > > Dumb question: how do I set show_errs ?
> > 
> > You don't - it's done for you if you ue the md/fr1 pair from the fr1
> > patch. But it should be shown in the output of /proc/nbdinfo (State
> > line).  And you can turn it on manually by echoing show_errs=1 to
> > /proc/nbdinfo, or by using it as a module parameter.  I think there
> > might be inputs for it in /proc/sys/dev too, but don't quote me on that!

> Ok, I set show_err=1 via proc and see the following:

But it's still not right. It should happen autmatically. I suspect the
md module is the old one, not the new one.

> If I shoot one enbd-server process manually the raid goes into degraded but
> continues to work. So this is ok.
> Unfortunately one has to restart the respective enbd-client, too, when the
> corresponding server process died (or the corresponding machine died). Obvious

No one doesn't. State across reboots is mediated by the two helper
daemons either side.

> reason is documented (the magic does not match). In fact this is bad because
> the enbd-client side has to know that it must restart the client process, which
> is not very straight forward.

It doesn't have to restart it at all! The helper daemon should send it
SIGPWR which does the trick.

> But I experienced another problem: unplugging the network.
> If you do that the raid goes in degraded mode (correct).
> But if you re-plug the network again, it does _not_ reconstruct, but loops with:

If one reconnects, the enbd restarts the device, notifies the raid and
so on. The stuff you show below says that there is an error at a low
level in the enbd device, so that never happens. I would guess that you
are not sending SIGPWR to the daemon, or some such. 

But anyway, the situation is all wrong already so it is not worth
thinking about.  You will have to make sure the right md module is
loaded.


> (client-side)
> enbd-client 26138: client (1) read passwd ok from nrhost2:60002
> enbd-client 26138: client (1) got cliserv magic ok from nrhost2:60002
> enbd-client 26138: client (1) got a signature ok from nrhost2:60002
> enbd-client 26138: client (1) begins main loop
> enbd-client 26138: <#1488> newproto get request from kernel failed (-1) Bad file descriptor
> enbd-client 26137: <#1488> newproto get request from kernel failed (-1) Bad file descriptor
> enbd-client 26138: <#1488> newproto get request from kernel failed (-1) Bad file descriptor
> enbd-client 26137: <#1488> newproto get request from kernel failed (-1) Bad file descriptor
> enbd-client 26138: <#1488> newproto get request from kernel failed (-1) Bad file descriptor
> enbd-client 26137: <#1449> newproto kernel errored 4 times when we asked for a new req: Bad file descriptor
> enbd-client 26138: <#1449> newproto kernel errored 4 times when we asked for a new req: Bad file descriptor
> enbd-client 26137: <# 156> unplug requested unplug (0) Bad file descriptor on nrhost2:60002 so clear socket
> enbd-client 26137: client (0) last error Bad file descriptor
> enbd-client 24989: sighandler relaunches child from manager 
> enbd-client 26138: <# 156> unplug requested unplug (1) Bad file descriptor on nrhost2:60002 so clear socket
> enbd-client 26138: client (1) last error Bad file descriptor
> enbd-client 24989: sighandler relaunches child from manager 
> enbd-client 24989: client (-1) reaped dead child 26138 (died of signal 0 with exit status 0)
> enbd-client 24989: client (-1) reaped dead child 26137 (died of signal 0 with exit status 0)

> and:

> Jun  6 10:44:02 mail-a01 kernel: raid1: bitmap f6fab800 already active!
> Jun  6 10:44:02 mail-a01 kernel: raid1: Disk failure on ndb, disabling device. 
> Jun  6 10:44:02 mail-a01 kernel:        Operation continuing on 1 devices
> Jun  6 10:44:02 mail-a01 kernel: device not enabled.
> Jun  6 10:44:02 mail-a01 kernel: ENBD enbd.c #3467[58]: do_nbd_request ending req f7e02580 with prejudice
> Jun  6 10:44:02 mail-a01 kernel: end_request: I/O error, dev 2b:10 (nbd), sector 8624

Peter