Re: LTTng on ARM - timestamps

* Richard Purdie ([email protected]) wrote:
> Hi,
> 
> I've been testing LTTng 0.9.5 on an ARM 2.6.21 kernel and have noticed a
> lot of lost events.
> 
> It turns out that there is instrumentation in do_timer() in
> kernel/timer.c which requires the xtime_lock write seqlock to be held.
> The default ARM timer implementation uses jiffies which is read using
> the same xtime_lock() and hence always fails for this event (and
> possibly others if there is more instrumentation in xtime_lock paths).
> 

Yes, using jiffies as a time source is only a best effort. But I think
we can do it better.

do_timer should be the only code that traces over the xtime write
seqlock, except NMIs and, potentially, the lockdep instrumentation
(Locking facility), which you don't have on ARM AFAIK.

> Basically this means LTTng is broken on ARM (and anything else using
> jiffies based timer implementations for their timestamps).
>
AFAIK, ARM is the only architecture where I use this. Other
architectures provide some sort of TSC or external timer to read from.

> The best solution I can see is to add a finer grained seqlock only held
> with jiffies is changed and to use this in the timestamps for systems
> without a TSC. Is this problem known about and is there any alternative?
>


Yes, I have always considered that taking a lock, even a read seqlock,
in the tracing code should be made illegal ;) I have an alternative idea
that would fix the problem:

- Use only the 32 LSB of the jiffies. No need to take a read seqlock.
- At 1000HZ, the maximum trace duration would be:
  2^32 cycles / 1000 cycles/s = 4294967.296s = 49.71 days (a little bit
  too low)

Which is not bad, but could be much better. Actually, I keep 13 bits
(max of 8192 events per timer tick) for the "logical clock" and use the
other 51 bits for the jiffies. (max trace duration of 71355.23 years)
However, it requires to have a coherent 64 bits jiffies counter trucated
to 51 bits.

In order to do that correctly, using the ltt synthetic TSC which extends
a 32 bits counter into a 64 bits counter should let you do that without
locks. See ltt/ltt-heartbeat.c. The only "different" thing would be that
you would use the 32 jiffies LSB instead of get_cycles() to feed the 32 LSBs
to ltt-heartbeat.c. See asm-mips/ltt.h for an example of how it is used.

About the "zero" test, 64 bits (even 51 bits) counters should suffice to
insure that it will never wrap to 0 in the computer's life. I have seen
that kind of test done elsewhere in the core kernel code also.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68