Re: ENBD Issues with client crash / weird errors....

Hey

Ok going to try and clean up the email here a bit...

Heavy disk activity was present until the machine locked... and yes
when that happens there is none :), but before that there was much.

> This is also showing normal working. It does not really coincide with your
> description of what is happening :(.   

- -  the only explanation I have for this is that I fiddled with the
buffer setting that run. I'm going to setup the nbd as a raid device
again and see if that causes / helped produce those error messages
some how.

Im totally lost as to how to install the enbd fr1 module. When I
download the module off
ftp://oboe.it.uc3m.es/pub/Programs/fr1-2.current.tgz , it doesn't work
because of linux 2.6. Ok... so looking in
nbd-2.4.32/kernel/linux-2.6.x  I find the kernel patches. Looked in
the make file, it looks like it should go in the root of the kernel
directory  (where I have it extracted). I backed up the directory and
tried that and its overwriting the Makefile for the kernel. I know im
doing something very wrong but I don't know what.


the nbd-2.4.32/kernel/patches directory has no patches to apply to
kernel 2.6.12 ... latest is 2.6.8.... and I don't meet the criteria,
im not sure applying one is a good idea, nor am I running debian, and
I don't know what im doing at this point :)


> as fast as it was in the first snapshot, but it is taken only a few
> seconds later.


> But to simplify the situation, you want to be running with one daemon
> only, not four.

fair enough i will make sure only one starts.


> It does not seem to me to be a snapshot of the error situation.
> 
- I tired with what I could do, will try again to get the same errors


> Are you running with "show_errors" set, by the way? It probably would be
> helpful if you were to do so (as well as appropriate for FR1, when yu
> use that).

Can you give me some more information on how to  show errors ?  I
don't see  it on the command line --help ?  (and I'm not as familiar
with things in the Linux world as you, unfortunately, but I'm trying:)
)

> >
> > ive got the nbdinfo output, what do you mean by captuing 'free' ?
> 
> "Free" is a command.

ah, ill make sure I inc. that next time then


> 
> It used to be one of the bdflush options.
> 
that was the only part in /proc/kernel/vm/* that made sense to me to
change, but I'm not knowledgeable enough to go beyond that.
> 
> > >    -y     This option  specifies  that  local  resources  are
> > >           opened in synchronous mode for writes.
> >

> 
> But you described serverside issues, no? You showed traces which said
> the server was not replying!
> 
>  

No - - server has always been fine. the machine that runs enbd-client
is the the one which locks up .


> 
> > When the box crashed now,
> 
> What do you mean by "crashed"? A really crashed kernel will leave an oops.
> 

well with a monitor and keyboard pushing enter does *nothing* ... no
alt + F1 feature , no shift pg up and down. Just *dead*.   Again I
think related to me playing in /proc/kernel/vm . I'm not going to do
that this time.



> 
> That's usually nothing much to do with the machine. The question is what
> is the kernel state at this point.  Show debug info from the machine you
> say is locked or crashed at this point .. 

How  ?  I cannot perform any commands. SSH is dead. I have a serial
console, that's DEAD, and the KB console is dead. (no response).

At least before w/o playing with /proc/kerne/vm I could type, so again
I think that was the issue.


> 
> I'm not sure what you mean here. You want to print the log messages
> that the console is showing at this point. It is not possible for a
> kernel to "lock" without a software or harware error, and a software
> error would leave an oops.

Well I put in the crontab 
* * * * * cat /proc/nbdinfo >> /nbdinfo
and
* * * * * dmesg >> /dmesg-nfo

because Ive had problems getting any error information simply because
I can't seem to access the box by any means.

> 
> You have to be at the physical computer, or using a serial or network
> console,.
> 

Right I am ... 

> If you  want to pinpoint the problem without getting any info from the
> "bad" state, you will have to remove components of the situation one by
> one until you find a stable point. Remove RAID. Use just one daemon.
> Remove the file system. Control the speed.
> 
> Peter
> 

Ok at this point there was no RAID running.   I will try again and use
only one daemon.

By remove the file system are you implying   dd if=/dev/zero of=/dev/nda   ? 

How can I get the fr1 raid module in the kernel ? Or patch mine ?
You've mentioned I should do that, now your telling me to remove raid
? I agree that starting simple is the best, so I will try

1 daemon
2 no file system
3 no raid
4 no proc changes :)
5. controlling the speed I think one daemon will take care of. 

Ill try logging everything again and hopefully I get something.  Again
this is a client problem, enbd-server machine has never had a problem.
AND as I said if i reverse the roles, with the Faster machine running
the enbd and enbd-client it works w/o any errors, but moving the
drives and everything is _a lot_ of work. So I definitely want the
client running on the 600 Mhz dual machine which is what were working
on so were good :)


thanks again for all your help!  Ill post later tonight after I have
the results.

Cheers,

Jonathan