Re: Experiences on ARM with LTTng

* Richard Purdie ([email protected]) wrote:
> Hi,
> 
> I've been away from LTTng for a while but have recently been working
> with LTTng on a 2.6.21 based ARM kernel. Thanks for the update you sent
> in reply to Deepak's question, its useful to know where things stand.
> Things are looking good with the marker infrastructure! For what its
> worth I find the patch series format you're currently using quite
> useful. Have you any rough ideas on the timescale for completing the
> remaining invasive changes?
> 

I hope to finish that in 2 week to a month-ish before we have a LTTV
matching with the current LTTng snapshot (without stack dump nor
user-space tracing), but I see that interaction with LKML about patch
integration often drives me away from the higher-level LTTng/LTTV effort
:( Therefore, the process is slower, but at least we get things merged
in mainline.


> I ran into a couple of issues which I thought I'd mention here.
> 
> Firstly, it seems to be very easy to make LTTV segfault. Things I've
> found that make it segfault include missing eventdefs directory,

This is going away..

> missing
> trace files and extra junk files in extra directories (e.g.
> control/orig/interrupts_0).

This should be fixed.

> In the latter case I had an extra file in
> that location which was padded to the wrong size and I wouldn't have
> expected LTTng to read it yet it did. 

Well, since the tracefiles created are not "fixed", we cannot specify a
limited set of tracefiles for LTTV to read. Bogus tracefiles should not
be in a trace at all in the first place ;) This approach is useful in
many ways, including the fact that it helps dealing more easily with
crash dump - flight recorder traces merge; one just has to create a
crash/ subdirectory in its trace and put the buffers extracted from the
crash dump there, along with the buffers that may also have been
recorded while tracing was active.

> 
> Since things are probably about to change in this area would it be worth
> writing a patch to add some sanity checks or not?
> 

I guess the work would overlap a little bit. It would be better to do a
pass on the code once I finish switching over to the markers.

> Secondly, some ARM systems can quickly enter sleep modes where their TSC
> counter stops (since there is no CPU activity) and will do this when the
> system is idle. This confuses the timeline from LTTV's perspective since
> events get bunched together and traces become much shorter than they
> should be. 
> 
> We can work roughly how long the system sleeps for and I have hacks at
> the moment which increase the TSC counter accordingly. Such changes will
> break other TSC users though and won't make mainline kernels. Is there a
> way we could log this event and have LTTV account for it?
> 

I am a bit reluctant to use events to tell what happens with the TSC,
since TSC is itself used to reorder events coming from multiple traces.
I could be done, but goes against my motto about timekeeping: keep it
simple. I would rather prefer an approach that would create a wrapper
over the timestamp reads:

u64 sleep_cycles;

ltt_get_timestamp32()
  - return (u32)(get_cycles() + sleep_cycles);

ltt_get_timestamp64()
  - return get_cycles() + sleep_cycles;

 Called upon return from sleep mode:
arch_ltt_add_timestamp()
  - Updates sleep_cycles atomically

We would have to decide whether sleep_cycles should be per cpu.

What do you think of this approach ?

Mathieu


> Regards,
> 
> Richard
> 
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68