Re: MIPS linux-2.6.12 (highly patched) + LTTng-0.5.69 + ltt-control-0.10-12062006

David Wuertele <[email protected]> Tue, 16 Sep 2008 17:35:40 +0000 (UTC)
Newsgroups gmane.linux.kernel.tracing
Message-ID <[email protected]>
> The interesting question is, did you get the output files
> from the trace daemon?

Looks like I didn't:

bash-3.2# ls -lR /tmp
/tmp:
drwxr-xr-x    3 root     root           60 Jan  1 00:00 trace

/tmp/trace:
drwxr-xr-x    2 root     root          400 Jan  1 00:00 eventdefs

/tmp/trace/eventdefs:
-rw-r--r--    1 root     root         1588 Jan  1 00:00 core.xml
-rw-r--r--    1 root     root         3027 Jan  1 00:00 fs.xml
-rw-r--r--    1 root     root         1289 Jan  1 00:00 ipc.xml
-rw-r--r--    1 root     root         2478 Jan  1 00:00 kernel.xml
-rw-r--r--    1 root     root        12378 Jan  1 00:00 kernel_arch_arm.xml
-rw-r--r--    1 root     root        24535 Jan  1 00:00 kernel_arch_c2.xml
-rw-r--r--    1 root     root        12040 Jan  1 00:00 kernel_arch_i386.xml
-rw-r--r--    1 root     root        35474 Jan  1 00:00 kernel_arch_mips.xml
-rw-r--r--    1 root     root         1948 Jan  1 00:00 locking.xml
-rw-r--r--    1 root     root         1537 Jan  1 00:00 memory.xml
-rw-r--r--    1 root     root          797 Jan  1 00:00 network.xml
-rw-r--r--    1 root     root          593 Jan  1 00:00 network_ip_interface.xml
-rw-r--r--    1 root     root         5718 Jan  1 00:00 process.xml
-rw-r--r--    1 root     root         2132 Jan  1 00:00 socket.xml
-rw-r--r--    1 root     root          657 Jan  1 00:00 stack_arch_i386.xml
-rw-r--r--    1 root     root         5267 Jan  1 00:00 statedump.xml
-rw-r--r--    1 root     root         1663 Jan  1 00:00 timer.xml
-rw-r--r--    1 root     root         1194 Jan  1 00:00 user_generic.xml
bash-3.2# 

Other than whitespace, those look like the same files that were in
ltt-control-0.10-12062006/facilities/.

> If I were to guess what was happening, I would say it might
> be something causing a network delay.  At the point where you
> see the long delay, the system is trying to finalize the trace
> data to the log, and I think there's a delay specifically
> introduced to wait for stuff to flush to the daemon (and fs).

I used wireshark.  After the NFS read to load the lttctl program, there is no
network traffic from the embedded system other than replies to ARP queries.

I'm instrumenting the instrumentation with printk() now.

Regarding facilities XML:

> I recall having to do some minor customizations, but it was not critical.
> MIPS has some different signal numbers than other platforms, and I think
> that is what I had to change.

I notice that the bunch of XML files written to my /tmp/trace/eventdefs include
ones like "kernel_arch_arm.xml", "kernel_arch_i386.xml", etc., which don't seem
like they should be there.  Do I need to make sure that stuff doesn't get into
my kernel?

Thanks!
Dave