Re: Re: Minor bug in lis_process_rput() LiS 2.18 that probably doesn't affect anyone except me.

--- Dave Grothe <[email protected]> wrote:

> Why don't you give your driver an in-memory loopback option so that
> you can 
> measure everything except the I/O part.
> 

I'm going to try that, but I think that having the ISR adds an extra
element to the mix that is necessary for a good test.  
It's kind of a lot of work to write a loopback that will excercise
the read service routine and the ISR.

Besides my driver basically is all in-memory.  The only PCI cycle
that it has to do is 1 read to see the interrupt status at the front
of the ISR.  All the rest is just writing into memory which the board
polls every 125usec to see if there is more data to send or writing
data into ram.

Part of the problem may be pressure on the memory bus.  It is only
100MHz, but that still doesn't explain why it's _so_ much worse in
SMP mode vs UP mode.

My biggest clue has been today when I changed the stream head to use
QNONE rather than QUEUE syncrhonization.  That improved my throughput
50-60% and decreased the latencies to be almost on par with UP
(although still a little worse...)

So something with the stream head queue locking was boogering me up. 
Next week I'll try and narrow it down more.  It may have been simply
the overhead of having to grab the lock but I have a feeling that the
read side queue lock was pretty contended since you have my rsrv
calling the stream head rput 45000 times per second and the thread
doing the getmsg() is also on that same queue 45000 times per second
from a different thread.

One thing that strikes me as another place to look is in the flow
control case at the stream head.  It seems like sometimes when there
is flow control at the stream head that it stays "stuck" for a
relatively long period of time which causes me to drop frames.  I am
wondering if there is a hole where either the getmsg() thread is not
woken up properly or in the backenable....   Next week....

> Also, perhaps something is leaving interrupts disabled for a long 
> time.  This can be done by passing the wrong set of flags to 
> spin_unlock_irqrestore.  I found LiS doing that in a couple of
> cases where 
> it was getting multiple locks.  I think that I fixed all of those,
> but 
> maybe I missed one, or maybe your driver has something like that in
> it.

I did have this bug last month.  It took me a while to find out what
was causing it but I could see the problem really easily.  The thing
is that our board has to do a trick with the ioctls where we have to
wait for the ISR to come back with the response from the board, so if
interrupts are disabled the ioctl just fails.  I was doing pretty
much what you described, it was being locked with
lis_spin_lock_irqsave, then unlocked with lis_spin_unlock().  ugh.

I'll take a look for things like that in my driver and in LiS.  I
thought that LiS had all of the spin locks removed in 2.18, is that
right?  I guess it still applies to the new read/write locks.

Did you have any comments about the other stuff that I raised in the
last email?

thanks-
dan