atomic_up operations : smp_read_barrier_depends() or smb_rmb() ?

Mathieu Desnoyers <[email protected]>
Newsgroups gmane.linux.kernel.tracing
Message-ID <20061123221728.GA5551@Krystal>
Hi,

Reading the following article http://www.linuxjournal.com/article/8211 from 
Paul E. McKenney enlightened me yet a little more about the subtilities of
memory ordering across CPUs. I bring a question regarding the possible need of
smp_read_barrier_depends() in the reader code which would only affect Alpha, or
a smp_rmb().

In my case, where I use the UP primitives for writing information to per cpu
buffers (producer count, commit count), but where I need the data consumer to
see conherent values, I first added a smp_wmb() just before the atomic update
of the per-subbuffer "commit count", which is the value used to certify that
the data has been fully written to memory.

the commit count is later compared to the size of the subbuffers : if the
subbuffer is fully committed (full), it can be read.

The reader (ioctl called by the lttd daemon) can be on any CPU. So, the problem
is the following : most of the time, I don't need the data within the buffer, 
the commit count and producer count to be seen in a particular order from other
CPUs, except when a buffer switch occurs (subbuffer is full) : in this case,
the reader will have the ability to read these memory locations from another
CPU.

This is why I added a smp_wmb() just before the commit_count update (commits are
done unordered, so any commit count update may be the last one in the
subbuffer).

Now, for the reader side : it must do the following test :

if ( other cpu's subbuffer commit count % SUBBUFFER_SIZE == 0 )
  read other cpu's subbuffer

I expect the memory read of the commit count to happen _before_ the memory read
of the cpu subbuffer. The commit count is volatile (atomic_t). The other cpu's
subbuffer can be potentially accessed through memcpy, be the data source for a
disk write, network sendfile, ...

So my understanding is that there is not clear "data dependency" between the
commit count and the subbuffer data. That would make a full smb_rmb()
necessary.

In practice, we should almost never see this problem, as the reader "wakeup"
occuring when a buffer is full is asynchronous.

Comments are welcome.

Mathieu

OpenPGP public key:              http://krystal.dyndns.org:8080/key/compudj.gpg
Key fingerprint:     8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.