Re: Threadsanitizer Detection on GM

On Wed, 6 Feb 2019, Hongxu Chen wrote:

>    It is known that threadsanitizer has many false positives when
> detecting concurrent bugs with projects that depend on openmp, and GM is
> one of them. But I saw this commit (https://reviews.llvm.org/D13072), and
> it seems to solve the problem.
>    There is also a blog post about this
> https://xrunhprof.wordpress.com/2018/08/27/tsan-with-openmp.  So I followed
> the advice; and I built openmp with -DLIBOMP_TSAN_SUPPORT=TRUE and GM with
> threadsanitizer and by running "gm convert", I can see some data races
> reported. My question is:
>    1. Can I rely on this result and think there are some data races?
>    2. Or can I even build openmp with -DLIBOMP_TSAN_SUPPORT=TRUE and
> threadsanitizer
> as well?

I saw part of your large attachment, which can not possibly be sent on 
a mailing list like this.

The symbols I see warned about (e.g. __kmp_global) are not part of 
GraphicsMagick.  They appear to be part of the TSAN library 
implementation so something is wrong with the testing.  The article 
you provided a reference to shows that the reports should be about the 
code being intentionally tested.

Recent GraphicsMagick uses 'pragma omp atomic' and 'pragma omp flush' 
in pixel_iterator.c rather than using critical sections.  I am not 
sure how a data-race detector deals with OpenMP 'flush' requests 
(synchronize thread cached memory with with shared memory) since they 
are not the same as a lock and might look like a data-race.  Releases 
prior to 1.3.31 used critical sections.

Long ago I used valgrind's helgrind and drd modes to search for data 
races.  This required building a replacement libgomp based on pthreads 
rather than special Linux interfaces.

There are two things which are normally shared in GM OpenMP loops 
other than one mutex lock in pixel_cache.c.  These are the error 
status (to know when all the threads in the team should try to quit 
because one of them reported an error), and the progress callback (to 
report progress if a progress monitor is enabled).

Independent testing and analysis of GraphicsMagick is certainly 
valuable, but it is wise to identify specific code which should be 
corrected.

Bob
-- 
Bob Friesenhahn
[email protected], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
Public Key,     http://www.simplesystems.org/users/bfriesen/public-key.txt