Assertion failure in do_get_write_access() with kernel 2.6.5-rc1

At Mittwoch, 17. März 2004 04:56 Yang, Chen wrote:
>  Hi, Linus:
>     Below is the patch of InterMezzo against 2.6.5-rc1.
>     I can confirm that it's effective.
> [...]


Thanks for the patch, Chen.

System is running much better than before, but still has
a problem, which is reproducable by a special high-load
situation.


Test situation:

I'm testing on one machine with two intermezzo partitions
- ext3 and synching thru the normal IP-stack via apache.
That's a situation where I think deadlocks and concurrency-
or timing-problems in the overall-system are more likely 
to surface than with 2 separate machines or over the loopback.

Tests with many mutual (but conflict-free) additions and
moves brought up the following error after a short while.
The test creates a lot of files on the master, attaches lines
to it (via echo-commands) and then moves (i.e. renames) 
the file to another directory within the same partition.
The slave is just trying to keep in synch.



Error:

k1 kernel: Assertion failure in do_get_write_access() 
at fs/jbd/transaction.c:607: "jh->b_next_transaction == ((void *)0)"

Then the slave partition could no longer be accessed. 
A try to list the partion results in a freeze of the 
user-process. The kernel continues to work normally 
with all other partitions of the system (including 
the non-blocked Intermezzo-master). The system load
is shown as very high, but that is because
there remained (naturally) a lot of intersync 
cgi_go_fetch_kml processes - but they are merely 
waiting and not eating CPU. The slave partion could 
not be unmounted - system said it was busy.
Hence I assume a a kernel lock remains after
the assertion-failure bail-out.

The error is reproducable. And when a similiar test 
creates the file in one chunk, the error is not triggered.
Hence I suspect a timing- or race-problem.

For my feeling this problem is not new, but was allready 
present in earlier kernels also. With the new kernel
the error only shows up more clearly, because the other
earlier problems seem fixed.


System used:
Linux k1 2.6.5-rc1 #1 Wed Mar 17 08:17:31 CET 2004 i686 athlon i386 GNU/Linux



bash test-procedure, which reproduces the error:
------------------------------------------------

userno=0
while [ $userno -lt 10 ] ; do
        userno=$(($userno+1))
        echo "##### $0: User $userno #####" >> /var/log/messages
        mkdir -p  $SERVER_MOUNTPOINT/user${userno}/Maildir/cur
        mkdir -p  $SERVER_MOUNTPOINT/user${userno}/Maildir/new
        mkdir -p  $SERVER_MOUNTPOINT/user${userno}/Maildir/tmp
        fileno=0
        while [ $fileno -lt 10 ] ; do
                fileno=$(($fileno+1))
                file=u${userno}f${fileno}
                lineno=0
                while [ $lineno -lt 100 ] ; do
                        lineno=$(($lineno+1))
                        echo "line $lineno user $userno file $fileno" >> $SERVER_MOUNTPOINT/user${userno}/Maildir/tmp/$file
                done
                mv $SERVER_MOUNTPOINT/user${userno}/Maildir/tmp/$file $SERVER_MOUNTPOINT/user${userno}/Maildir/new/$file
        done
done


similiar test-procedure (whithout the many open-open-close), 
which does NOT reproduce the error
------------------------------------------------------------

userno=0
while [ $userno -lt 10 ] ; do
        userno=$(($userno+1))
        echo "##### $0: User $userno #####" >> /var/log/messages
        mkdir -p  $SERVER_MOUNTPOINT/user${userno}/Maildir/cur
        mkdir -p  $SERVER_MOUNTPOINT/user${userno}/Maildir/new
        mkdir -p  $SERVER_MOUNTPOINT/user${userno}/Maildir/tmp
        fileno=0
        while [ $fileno -lt 10 ] ; do
                fileno=$(($fileno+1))
                file=u${userno}f${fileno}
                lineno=0
                while [ $lineno -lt 100 ] ; do
                        lineno=$(($lineno+1))
                        echo "line $lineno user $userno file $fileno" >> /tmp/testfile.$$.$file
                done
                mv /tmp/testfile.$$.$file $SERVER_MOUNTPOINT/user${userno}/Maildir/tmp/$file
                mv $SERVER_MOUNTPOINT/user${userno}/Maildir/tmp/$file $SERVER_MOUNTPOINT/user${userno}/Maildir/new/$file
        done
done




Greetings
Michael
-- 
Michael Zimmermann (Vegaa Safety and Security for Internet Services)
<[email protected]>   phone +49 89 6283 7632    hotline +49 160 99 123 830



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click