Re: [diamon-discuss] Follow up on SFrame in JIT applications

Hi Indu,

On 9/26/23 22:24, Indu Bhagat wrote:

> I would like to follow up on your question around using SFrame in JITed 
> code.  I recall you mentioning that looking at MS exception handling may 
> give us some insights.  Unfortunately, I dont recall other 
> pointers/references made in that conversation.  Can you send me a brief 
> summary and any pointers that you may have around this so I can read up?

Thanks for following up on my Tracing Summit question. I am adding the 
diamon-discuss mailing list in CC so we can keep a trace of this 
conversation, hopefully it's OK with you.

I am also adding collaborators from Microsoft to this discussion. Those 
were present in a 2019 meeting held at Microsoft Redmond offices where 
I've been told about the shortcomings of the way integration between 
JITs and the backtrace infrastructure is done in Windows. This was 
discussed in the context of improving the backtrace infrastructure on Linux.

Here is a link to your presentation abstract as reference to others:

- https://tracingsummit.org/ts/2023/sframe/ (abstract)
- https://tracingsummit.org/ts/2023/files/SFrame_TracingSummit2023.pdf 
(slides)

For context, I've been discussing with Indu how we should extend the 
SFrame section (generated by GNU binutils, currently designed to augment 
the ELF binaries with additional sections which contain relevant frame 
information permitting fast backtrace without frame pointers) to JITs. 
Or at least define an ABI between applications, libraries, and the Linux 
kernel, which allows populating frame description information efficiently.

I recall from our discussion at Microsoft that the way it was done on 
Windows ended up being a performance bottleneck for JITs. If my memory 
serves me right, every time the JIT emits a function, it pretty much 
calls into the kernel to register its existence immediately.

I would rather see a mechanism on Linux which is similar to the rseq(2), 
io_uring(2), and perf_open(2) system calls. Those are implemented with a 
memory mapping between kernel and user-space. We could do something 
similar for a JITted sframe section: Populate an empty memory mapping of 
a given size, with a header which describes how many entries are 
populated. Then as the JIT appends a functions, it increments the 
populated size with a store-release semantic. Mutual exclusion between 
multiple producers updating a mapping would be left to userspace.

I wonder if it would be acceptable for JITs to successively provide the 
functions in increasing order of memory addresses within a mapping, so 
they are already sorted.

We would also probably want to plan a tombstone bit that would be set by 
the JIT when it needs to reclaim a function.

Hopefully our collaborators at Microsoft will be able to provide more 
background information on that topic, especially references to the 
Windows side of things.

Thanks!

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com