Re: [diamon-discuss] Follow up on SFrame in JIT applications

On Wed, Nov 08, 2023 at 11:00:23AM -0800, Indu Bhagat wrote:
> [Re-sending. Apologies for duplicate copy in your inbox. Previous
> message did not make it to the mailing list.]
>=20
> On 9/28/23 10:39, Brian Robbins wrote:
> >>Hi Indu,
> >>
> >>On 9/26/23 22:24, Indu Bhagat wrote:
> >>
> >>>I would like to follow up on your question around using SFrame in
> >>>JITed code.  I recall you mentioning that looking at MS exception
> >>>handling may give us some insights.  Unfortunately, I dont recall
> >>>other pointers/references made in that conversation.  Can you send me
> >>>a brief summary and any pointers that you may have around this so I can
> >>read up?
> >>
> >>Thanks for following up on my Tracing Summit question. I am adding the
> >>diamon-discuss mailing list in CC so we can keep a trace of this conver=
sation,
> >>hopefully it's OK with you.
> >>
> >>I am also adding collaborators from Microsoft to this discussion. Those=
 were
> >>present in a 2019 meeting held at Microsoft Redmond offices where I've =
been
> >>told about the shortcomings of the way integration between JITs and the
> >>backtrace infrastructure is done in Windows. This was discussed in the
> >>context of improving the backtrace infrastructure on Linux.
> >>
> >>Here is a link to your presentation abstract as reference to others:
> >>
> >>-
> >>https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.=
com/?url=3Dhttps*3A*2F*2Ftracin__;JSUl!!ACWV5N9M2RV99hQ!PXCZnoInFiL0_ATh0jN=
Tdnr7pMAM7f6iODT-lcbI07-m5gK8GIk7Fx0pRDm9aGYmF98vgt9Ef_texEZmZ-LdFurfow$
> >>gsummit.org%2Fts%2F2023%2Fsframe%2F&data=3D05%7C01%7Cbrianrob%40
> >>microsoft.com%7Cd3d3f9d6f1864e61e02808dbbf382d7c%7C72f988bf86f141a
> >>f91ab2d7cd011db47%7C1%7C0%7C638314020881735122%7CUnknown%7CT
> >>WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
> >>XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3Dfp2ECCRPbnsCP9WvNmV4GM%
> >>2Bd1pkkG8vc8dUtFqNQGEw%3D&reserved=3D0 (abstract)
> >>-
> >>https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.=
com/?url=3Dhttps*3A*2F*2Ftracin__;JSUl!!ACWV5N9M2RV99hQ!PXCZnoInFiL0_ATh0jN=
Tdnr7pMAM7f6iODT-lcbI07-m5gK8GIk7Fx0pRDm9aGYmF98vgt9Ef_texEZmZ-LdFurfow$
> >>gsummit.org%2Fts%2F2023%2Ffiles%2FSFrame_TracingSummit2023.pdf&dat
> >>a=3D05%7C01%7Cbrianrob%40microsoft.com%7Cd3d3f9d6f1864e61e02808dbb
> >>f382d7c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6383140208
> >>81743792%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> >>V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3DwI
> >>JwUw4fktvyJ5Rcnn%2F5j38TcRFr8wSWSr5omF8OlQU%3D&reserved=3D0
> >>(slides)
> >>
> >>For context, I've been discussing with Indu how we should extend the SF=
rame
> >>section (generated by GNU binutils, currently designed to augment the E=
LF
> >>binaries with additional sections which contain relevant frame informat=
ion
> >>permitting fast backtrace without frame pointers) to JITs.
> >>Or at least define an ABI between applications, libraries, and the Linu=
x kernel,
> >>which allows populating frame description information efficiently.
> >>
> >>I recall from our discussion at Microsoft that the way it was done on W=
indows
> >>ended up being a performance bottleneck for JITs. If my memory serves me
> >>right, every time the JIT emits a function, it pretty much calls into t=
he kernel to
> >>register its existence immediately.
> >>
> >>I would rather see a mechanism on Linux which is similar to the rseq(2),
> >>io_uring(2), and perf_open(2) system calls. Those are implemented with a
> >>memory mapping between kernel and user-space. We could do something
> >>similar for a JITted sframe section: Populate an empty memory mapping o=
f a
> >>given size, with a header which describes how many entries are populate=
d.
> >>Then as the JIT appends a functions, it increments the populated size w=
ith a
> >>store-release semantic. Mutual exclusion between multiple producers
> >>updating a mapping would be left to userspace.
> >>
> >>I wonder if it would be acceptable for JITs to successively provide the
> >>functions in increasing order of memory addresses within a mapping, so =
they
> >>are already sorted.
> >>
> >>We would also probably want to plan a tombstone bit that would be set by
> >>the JIT when it needs to reclaim a function.
> >>
> >>Hopefully our collaborators at Microsoft will be able to provide more
> >>background information on that topic, especially references to the Wind=
ows
> >>side of things.
> >>
> >
> >Your recollection is correct that there are win32 APIs that are used to =
register the existence of new jitted code in order for the OS unwinder to k=
now how to handle the code.  The functions that get used are RtlAddGrowable=
FunctionTable [1],  RtlGrowFunctionTable [2], RtlDeleteGrowableFunctionTabl=
e [3].
> >
> >The idea is that you create a growable function table in userspace that =
is an array of structures that point to unwind information.  When you do th=
is, you're registering the array with the operating system and a range of m=
emory in the process that does/will contain code.  Then, when you JIT new f=
unctions, you tell the OS about them by calling RtlGrowFunctionTable.  As l=
ong as the new functions are located at memory addresses that are higher th=
an the last entry, then you can just initialize the next structure in the a=
rray and call RtlGrowFunctionTable.  If you do an out-of-order insert, then=
 you have to delete the old table, create a new one, sort the data, and reg=
ister the new one with the OS.  This sounds somewhat similar to the proposa=
l you've specified above.
> >
>=20
> Its not clear to me why there is a need to tell the OS about each
> new JIT function. If a "GrowableFunctionTable" is registered with
> the OS, so long as the space and ordering constraints etc are
> satisfied, why is there a need for the OS to know about each new
> function.
>=20

=46rom a review of the code, the GrowableFunctionTable has two sizes - a cu=
rrent size and a maximum size.  The current size is representative of the c=
urrent number of valid entries in the array.  When jitting a method, we gen=
erate unwind information for that method and store it in the next available=
 slot in the array.  To publish it, we call RtlGrowFunctionTable and tell i=
t the size of the table (including the new entry).  You could imagine a des=
ign where you don't use a call into the kernel to implement this, but that'=
s the way it's implemented here.

>=20
> >One thing that is worth calling out is that a process can effectively ca=
ll RtlAddGrowableFunctionTable to create as many tables as it wants.  This =
(or something that replaces this functionality) is important when it comes =
to thinking about the lifetime of dynamically jitted code.  For jitted code=
 that represents code stored in assemblies (DLLs) on disk, the lifetime of =
the growable function table is generally long, and the table often doesn't =
have to be re-created.  However, there are cases, such as dynamically gener=
ated code, where the lifetime of the jitted code is quite short, and oftent=
imes, the address range gets re-used over and over by newly generated code.=
  In this case, it is often worthwhile to create a new growable function ta=
ble for the method or set of methods that were dynamically generated togeth=
er.  This allows the process to delete the table when the code destroyed wi=
thout having to re-create a new table for code that wasn't destroyed but ha=
ppened to be part of the table.  All this to say, the design point of suppo=
rting multiple tables per process is an important one.
> >
> >If you'd like to see how the .NET runtime does this, here's a pointer to=
 the code that adds new entries to the existing function table: https://url=
defense.com/v3/__https://github.com/dotnet/runtime/blob/7e6bd94c52b6f4d75f8=
e5098e814b4ffacdd5e02/src/coreclr/vm/codeman.cpp*L228__;Iw!!ACWV5N9M2RV99hQ=
!PXCZnoInFiL0_ATh0jNTdnr7pMAM7f6iODT-lcbI07-m5gK8GIk7Fx0pRDm9aGYmF98vgt9Ef_=
texEZmZ-Iz4eMDQw$ .
> >
>=20
> Can you share what is the format of "Unwind Information" kept per
> function ? Also, is a subset of information used for backtracing
> also kept in the frame of each function ?
>=20

Here's a pointer to the best article I can find: https://learn.microsoft.co=
m/en-us/cpp/build/exception-handling-x64?view=3Dmsvc-170.  If you have more=
 questions, I can reach out to some colleagues.

I don't believe that anything is kept in the function itself.  The stack wa=
lker will use the IP as an input to lookup the unwind information directly.

Thanks.
--Brian

> Thanks
>=20
> >Happy to answer any questions.
> >
> >--Brian
> >
> >>Thanks!
> >>
> >>Mathieu
> >>
> >>--
> >>Mathieu Desnoyers
> >>EfficiOS Inc.
> >>https://urldefense.com/v3/__https://nam06.safelinks.protection.outlook.=
com/?url=3Dhttps*3A*2F*2Fwww__;JSUl!!ACWV5N9M2RV99hQ!PXCZnoInFiL0_ATh0jNTdn=
r7pMAM7f6iODT-lcbI07-m5gK8GIk7Fx0pRDm9aGYmF98vgt9Ef_texEZmZ-I2kCGKTw$ .
> >>efficios.com%2F&data=3D05%7C01%7Cbrianrob%40microsoft.com%7Cd3d3f9d
> >>6f1864e61e02808dbbf382d7c%7C72f988bf86f141af91ab2d7cd011db47%7C1
> >>%7C0%7C638314020881749505%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
> >>4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%
> >>7C%7C%7C&sdata=3Dv6XfK7SLrOAn50bL1xt78Ul1uCzcsky0vGFt3sP6vmU%3D&r
> >>eserved=3D0
> >
> >[1] https://urldefense.com/v3/__https://learn.microsoft.com/en-us/window=
s/win32/api/winnt/nf-winnt-rtladdgrowablefunctiontable__;!!ACWV5N9M2RV99hQ!=
PXCZnoInFiL0_ATh0jNTdnr7pMAM7f6iODT-lcbI07-m5gK8GIk7Fx0pRDm9aGYmF98vgt9Ef_t=
exEZmZ-KJ7TjPBA$
> >[2] https://urldefense.com/v3/__https://learn.microsoft.com/en-us/window=
s/win32/api/winnt/nf-winnt-rtlgrowfunctiontable__;!!ACWV5N9M2RV99hQ!PXCZnoI=
nFiL0_ATh0jNTdnr7pMAM7f6iODT-lcbI07-m5gK8GIk7Fx0pRDm9aGYmF98vgt9Ef_texEZmZ-=
KpAAd9Ww$
> >[3] https://urldefense.com/v3/__https://learn.microsoft.com/en-us/window=
s/win32/api/winnt/nf-winnt-rtldeletegrowablefunctiontable__;!!ACWV5N9M2RV99=
hQ!PXCZnoInFiL0_ATh0jNTdnr7pMAM7f6iODT-lcbI07-m5gK8GIk7Fx0pRDm9aGYmF98vgt9E=
f_texEZmZ-K63L5WoQ$
>=20