atomic_up operations : smp_read_barrier_depends() or smb_rmb() ?
Mathieu Desnoyers <[email protected]>
| Newsgroups | gmane.linux.kernel.tracing |
|---|---|
| Message-ID | <20061123221728.GA5551@Krystal> |
Hi, Reading the following article http://www.linuxjournal.com/article/8211 from Paul E. McKenney enlightened me yet a little more about the subtilities of memory ordering across CPUs. I bring a question regarding the possible need of smp_read_barrier_depends() in the reader code which would only affect Alpha, or a smp_rmb(). In my case, where I use the UP primitives for writing information to per cpu buffers (producer count, commit count), but where I need the data consumer to see conherent values, I first added a smp_wmb() just before the atomic update of the per-subbuffer "commit count", which is the value used to certify that the data has been fully written to memory. the commit count is later compared to the size of the subbuffers : if the subbuffer is fully committed (full), it can be read. The reader (ioctl called by the lttd daemon) can be on any CPU. So, the problem is the following : most of the time, I don't need the data within the buffer, the commit count and producer count to be seen in a particular order from other CPUs, except when a buffer switch occurs (subbuffer is full) : in this case, the reader will have the ability to read these memory locations from another CPU. This is why I added a smp_wmb() just before the commit_count update (commits are done unordered, so any commit count update may be the last one in the subbuffer). Now, for the reader side : it must do the following test : if ( other cpu's subbuffer commit count % SUBBUFFER_SIZE == 0 ) read other cpu's subbuffer I expect the memory read of the commit count to happen _before_ the memory read of the cpu subbuffer. The commit count is volatile (atomic_t). The other cpu's subbuffer can be potentially accessed through memcpy, be the data source for a disk write, network sendfile, ... So my understanding is that there is not clear "data dependency" between the commit count and the subbuffer data. That would make a full smb_rmb() necessary. In practice, we should almost never see this problem, as the reader "wakeup" occuring when a buffer is full is asynchronous. Comments are welcome. Mathieu OpenPGP public key: http://krystal.dyndns.org:8080/key/compudj.gpg Key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68