Re: [diamon-discuss] Follow up on SFrame in JIT applications

> Hi Indu,
>=20
> On 9/26/23 22:24, Indu Bhagat wrote:
>=20
> > I would like to follow up on your question around using SFrame in
> > JITed code.  I recall you mentioning that looking at MS exception
> > handling may give us some insights.  Unfortunately, I dont recall
> > other pointers/references made in that conversation.  Can you send me
> > a brief summary and any pointers that you may have around this so I can
> read up?
>=20
> Thanks for following up on my Tracing Summit question. I am adding the
> diamon-discuss mailing list in CC so we can keep a trace of this conversa=
tion,
> hopefully it's OK with you.
>=20
> I am also adding collaborators from Microsoft to this discussion. Those w=
ere
> present in a 2019 meeting held at Microsoft Redmond offices where I've be=
en
> told about the shortcomings of the way integration between JITs and the
> backtrace infrastructure is done in Windows. This was discussed in the
> context of improving the backtrace infrastructure on Linux.
>=20
> Here is a link to your presentation abstract as reference to others:
>=20
> -
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Ftracin
> gsummit.org%2Fts%2F2023%2Fsframe%2F&data=3D05%7C01%7Cbrianrob%40
> microsoft.com%7Cd3d3f9d6f1864e61e02808dbbf382d7c%7C72f988bf86f141a
> f91ab2d7cd011db47%7C1%7C0%7C638314020881735122%7CUnknown%7CT
> WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
> XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3Dfp2ECCRPbnsCP9WvNmV4GM%
> 2Bd1pkkG8vc8dUtFqNQGEw%3D&reserved=3D0 (abstract)
> -
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Ftracin
> gsummit.org%2Fts%2F2023%2Ffiles%2FSFrame_TracingSummit2023.pdf&dat
> a=3D05%7C01%7Cbrianrob%40microsoft.com%7Cd3d3f9d6f1864e61e02808dbb
> f382d7c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6383140208
> 81743792%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3DwI
> JwUw4fktvyJ5Rcnn%2F5j38TcRFr8wSWSr5omF8OlQU%3D&reserved=3D0
> (slides)
>=20
> For context, I've been discussing with Indu how we should extend the SFra=
me
> section (generated by GNU binutils, currently designed to augment the ELF
> binaries with additional sections which contain relevant frame information
> permitting fast backtrace without frame pointers) to JITs.
> Or at least define an ABI between applications, libraries, and the Linux =
kernel,
> which allows populating frame description information efficiently.
>=20
> I recall from our discussion at Microsoft that the way it was done on Win=
dows
> ended up being a performance bottleneck for JITs. If my memory serves me
> right, every time the JIT emits a function, it pretty much calls into the=
 kernel to
> register its existence immediately.
>=20
> I would rather see a mechanism on Linux which is similar to the rseq(2),
> io_uring(2), and perf_open(2) system calls. Those are implemented with a
> memory mapping between kernel and user-space. We could do something
> similar for a JITted sframe section: Populate an empty memory mapping of a
> given size, with a header which describes how many entries are populated.
> Then as the JIT appends a functions, it increments the populated size wit=
h a
> store-release semantic. Mutual exclusion between multiple producers
> updating a mapping would be left to userspace.
>=20
> I wonder if it would be acceptable for JITs to successively provide the
> functions in increasing order of memory addresses within a mapping, so th=
ey
> are already sorted.
>=20
> We would also probably want to plan a tombstone bit that would be set by
> the JIT when it needs to reclaim a function.
>=20
> Hopefully our collaborators at Microsoft will be able to provide more
> background information on that topic, especially references to the Windows
> side of things.
>=20

Your recollection is correct that there are win32 APIs that are used to reg=
ister the existence of new jitted code in order for the OS unwinder to know=
 how to handle the code.  The functions that get used are RtlAddGrowableFun=
ctionTable [1],  RtlGrowFunctionTable [2], RtlDeleteGrowableFunctionTable [=
3].

The idea is that you create a growable function table in userspace that is =
an array of structures that point to unwind information.  When you do this,=
 you're registering the array with the operating system and a range of memo=
ry in the process that does/will contain code.  Then, when you JIT new func=
tions, you tell the OS about them by calling RtlGrowFunctionTable.  As long=
 as the new functions are located at memory addresses that are higher than =
the last entry, then you can just initialize the next structure in the arra=
y and call RtlGrowFunctionTable.  If you do an out-of-order insert, then yo=
u have to delete the old table, create a new one, sort the data, and regist=
er the new one with the OS.  This sounds somewhat similar to the proposal y=
ou've specified above.

One thing that is worth calling out is that a process can effectively call =
RtlAddGrowableFunctionTable to create as many tables as it wants.  This (or=
 something that replaces this functionality) is important when it comes to =
thinking about the lifetime of dynamically jitted code.  For jitted code th=
at represents code stored in assemblies (DLLs) on disk, the lifetime of the=
 growable function table is generally long, and the table often doesn't hav=
e to be re-created.  However, there are cases, such as dynamically generate=
d code, where the lifetime of the jitted code is quite short, and oftentime=
s, the address range gets re-used over and over by newly generated code.  I=
n this case, it is often worthwhile to create a new growable function table=
 for the method or set of methods that were dynamically generated together.=
  This allows the process to delete the table when the code destroyed witho=
ut having to re-create a new table for code that wasn't destroyed but happe=
ned to be part of the table.  All this to say, the design point of supporti=
ng multiple tables per process is an important one.

If you'd like to see how the .NET runtime does this, here's a pointer to th=
e code that adds new entries to the existing function table: https://github=
=2Ecom/dotnet/runtime/blob/7e6bd94c52b6f4d75f8e5098e814b4ffacdd5e02/src/cor=
eclr/vm/codeman.cpp#L228.

Happy to answer any questions.

--Brian

> Thanks!
>=20
> Mathieu
>=20
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww.
> efficios.com%2F&data=3D05%7C01%7Cbrianrob%40microsoft.com%7Cd3d3f9d
> 6f1864e61e02808dbbf382d7c%7C72f988bf86f141af91ab2d7cd011db47%7C1
> %7C0%7C638314020881749505%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
> 4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%
> 7C%7C%7C&sdata=3Dv6XfK7SLrOAn50bL1xt78Ul1uCzcsky0vGFt3sP6vmU%3D&r
> eserved=3D0

[1] https://learn.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtla=
ddgrowablefunctiontable
[2] https://learn.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlg=
rowfunctiontable
[3] https://learn.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtld=
eletegrowablefunctiontable