Re: pthread-mutex debugging (PR#44387)

matthew green wrote:

>> I could now isolate the problem on sys/kern/kern_mutex.c. It was
>> sufficient to compile kern_mutex.c with -DFULL to make the problem
>> disappear. The rest of the kernel was compiled without DIAGNOSTIC.
>>
>
> can you confirm what code is being compiled and called from where?

I had neither defined DIAGNOSTIC nor LOCKDEBUG, so I guess that a
mutex_enter() would call the function in lock_stubs.s and then, eventually,
drop into mutex_vector_enter().

A muxex_spin_enter() would directly call mutex_vector_end(), as we don't
have a stub for it.

Is it allowed to define __HAVE_MUTEX_STUBS but not __HAVE__SPIN_MUTEX_STUBS?

The -DFULL only adds a call to __cpu_simple_lock_try(), to set the mtx_lock
field in the kmutex accordingly, so it can be checked by mutex_exit(). It's
really strange that it makes such a difference.


> i'm assuming that the non debug kernel with -DFULL in kern_mutex.c
> kernel (that fails)

No. The kernel with -DFULL doesn't fail. It fails when using a non debug
kernel without -DFULL in kern_mutex.c.


> has lock_stubs.s:mutex_enter() calling into
> mutex_vector_enter().  (same with exit.)

Yes.


> you could try disabling this parts to see what happens:
>
> #define __HAVE_MUTEX_STUBS              1

There seem to be some strange dependencies. Removing this definition from
powerpc/include/mutex.h doesn't let me link the kernel anymore:

kern_mutex.o: In function `mutex_spin_enter':
kern_mutex.c:(.text+0x200): undefined reference to `_lock_cas' 
kern_mutex.c:(.text+0x244): undefined reference to `_lock_cas'
kern_mutex.o: In function `mutex_tryenter':
kern_mutex.c:(.text+0x490): undefined reference to `_lock_cas'


By looking into kern_mutex.c...

#if defined(LOCKDEBUG)
#undef  __HAVE_MUTEX_STUBS
#undef  __HAVE_SPIN_MUTEX_STUBS
#endif

...it seems that I can have the same effect when I compile a kernel with
LOCKDEBUG.

And indeed, also with LOCKDEBUG (but without DIAGNOSTIC and without -DFULL),
the problem does not appear!


> is this a spin or adaptive mutex?

I'm not even sure what is failing here. It can be observed that the
pthread-mutex test from the ATF cannot complete, because both threads are
entering "parked" or "iowait" state when the failure occurs.

Does libpthread even use kmutex? Or does it implement its own mutex?

But my guess it that the problem occurs while acquiring a spin mutex,
because otherwise -DFULL wouldn't have any effect (the code is only called
in the spin-mutex case).

-- 
Frank Wille