Re: [diamon-discuss] Diamon CTF Roadmap?

On 2014/11/27 3:43, Dominique Toupin wrote:
> You can see my answer below preceded by [Dominique Toupin]
> 
> -----Original Message-----
> From: Wang Nan [mailto:[email protected]] 
> Sent: November-25-14 9:08 PM
> To: Dominique Toupin; [email protected]; Jiri Olsa
> Cc: genghui 00204690; Xinwei Hu
> Subject: Re: Diamon CTF Roadmap?
> 
> On 2014/11/25 21:42, Dominique Toupin wrote:
>> Hi Nan,
>>
>> Many tools are able to handle CTF today, e.g.:
>> - GDB Tracepoint 
>> https://sourceware.org/gdb/onlinedocs/gdb/Trace-Files.html#Trace-Files
>> - LTTng kernel and LTTng user space C/C++, Java, Python, etc. 
>> https://lttng.org/download
>> - babel trace, http://lttng.org/viewers/
>> - Trace Compass 
>> http://projects.eclipse.org/projects/tools.tracecompass
>> - CTF tracer for bare metal 
>> https://github.com/efficios/barectf/blob/master/README.md
>>
>> You will also find many converter to CTF, e.g. NEXUS HW Trace, perf, 
>> And I am most likely missing a few.
>>
>> If you can provide a description of your needs it would help to give you more info related to your use case.
>> e.g. is it more related to cloud, embedded data base, real time, or do you have a specific view or analysis you want to see in the trace viewer, etc.
>>
> 
> Our use cases are diverse, both embedded and server scene have requirements on profiling, and methodologies are also different.
> 
> We have two starting points now:
> 
> 1. On ARM (32/64) platform, extracing information (instruction footprints) from CoreSight
>    using perf and use CTF as trace format, to make it possible to analysis with trace from
>    upper layer. Intel has similar hardware too, but currently only on specification.
> 
> [Dominique Toupin] I believe they are able to get perf data in CTF and if you also have 
> CTF trace from upper layer then you can use Trace Compass (python should also be an option) 
> to visualize, correlate, analyze trace data from different layers, you can also open text logs in 
> Trace Compass. In the upper layer, you can take a similar approach as the Linux kernel where 
> static tracepoints represent the wisdom of developers who are most familiar with the code and 
> the rest of the world (testers, system administrators, field engineers, developers) can use them 
> to extract a great deal of useful information without having to know the code. 
> 
> 2. On server field, providing some interactive, graphical user interfaces for some generic
>    problems. Android's SysTrace is a good example. However, our the real goal is to create
>    a tool for dealing with problems not so generic. Profiling using a language like SQL maybe,
>    like Splunk, to synthesize traces from different layers and components. So the underlying
>    traces are better to be in a uniform format.
> 
> [Dominique Toupin] SQL can be nice but it requires a lot more space, if I remember 
> correctly, the size is sometimes multiplied by 70 compared to CTF. CTF is like a specialized 
> trace data base, in Trace Compass they have efficient query to navigate in the trace, they
>  add a state system, etc. You can do all this with SQL but it require a lot more space and it is slower.
> If you see things missing in CTF compared to what you want to do with SQL, let us know we can 
> investigate how to add those features directly in CTF.
> 

Thanks for your reply. I'll try Trace Compass in our situation.

In addition, is there any description on the organization and policy of DiaMon?

> 
> We also concern the upstreaming plan about kernel stuffs in DiaMon.
> 
>> Thanks for the second post!
>>
>> Dominique
>>
>> -----Original Message-----
>> From: Wang Nan [mailto:[email protected]]
>> Sent: November-25-14 8:04 AM
>> To: [email protected]; Dominique Toupin; Jiri 
>> Olsa
>> Cc: genghui 00204690; Xinwei Hu
>> Subject: Diamon CTF Roadmap?
>>
>> Hi everyone,
>>
>> Looks like my post will be the second thread in this list. That's great.
>>
>> I'll be the technical contact person representing Huawei for Diamon.
>> I'm glad to work with this new workgroup to create "de-facto, widely used open source trace technologies".
>>
>> Beyond sharing new ideas and technologies, our main concern on Diamon now are CTF tracing format and related things for analysing traces from different sources in a unified manner. I have read Jiri Olsa's slides in TracingSummit 2014, and also begin to read his code in https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git. However, it looks like that there's only a trace generating tool is under developing. What about upper trace parsing and analysing tools? Is there any roadmap on it? How can we participate in the project?
>>
>> Thanks.
>>
> 
>