Re: Deferring native compilation finalization until idle

On Sat, Jul 25, 2026 at 9:29 AM, Stéphane Marks < [email protected] > wrote:

> 
> On Sat, Jul 25, 2026 at 12:22 PM Aaron Jensen < aaronjensen@ gmail. com (
> [email protected] ) > wrote:
> 
> 
>> On Sat, Jul 25, 2026 at 9:18 AM, Stéphane Marks < shipmints@ gmail. com (
>> [email protected] ) > wrote:
>> 
>> 
>>> On Sat, Jul 25, 2026 at 12:14 PM Aaron Jensen < aaronjensen@ gmail. com (
>>> [email protected] ) > wrote:
>>> 
>>> 
>>>> On Sat, Jul 25, 2026 at 9:04 AM, Stéphane Marks < shipmints@ gmail. com (
>>>> [email protected] ) > wrote:
>>>> 
>>>> 
>>>>> FWIW, my eln-cache this morning , starting fresh from Thursday , has 146
>>>>> files with seldom used packages uncompiled, of course.  Rebuilding that
>>>>> cache was entirely asynchronous, perhaps except for the trampolines.  I
>>>>> have not tried setting native-comp-enable-subr-trampolines to nil in
>>>>> early-init.el to see if this bypasses the delays that you're seeing.  Have
>>>>> you tried that?
>>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> Setting native-comp-enable-subr-trampolines to nil does not noticeably
>>>> reduce the stutter for me.
>>>> 
>>>> 
>>>> 
>>>> Ending up with only 146 files likely means that even if you were to not
>>>> have one of the workarounds in place, you would not notice as much stutter
>>>> as one who had more.
>>>> 
>>>> 
>>>> 
>>>> My guess is that if you were to try my config, or someone else's that
>>>> notices the issue and not disable macOS's security checks, you too would
>>>> notice the stutter.
>>>> 
>>>> 
>>>> 
>>>> I work with others that use macOS. I can survey them to see how many of
>>>> them have issues if that would be convincing to anyone. However, I think
>>>> the facts are pretty well established and ultimately it's a decision as to
>>>> whether or not the maintainers want to account for the particularities of
>>>> macOS's bundle validation process by loading the bundle within the async
>>>> worker (and take on the additional maintenance burden).
>>>> 
>>>> 
>>>> 
>>>> Personally, I have my workaround, so I'm "fine". But I'm not a fan of
>>>> macOS users that have have sufficient packages having an experience that
>>>> leaves them with the opinion that Emacs has poor startup performance, or
>>>> native compilation makes things worse, etc. Yes, those are
>>>> narrowly-informed opinions, but I'm sure many users would not take the
>>>> time to compare their startup performance to other OS's and/or to find
>>>> this thread to better understand the depths of the problem. Instead,
>>>> they'll just have a bad taste and deal with it every time they install a
>>>> new version of Emacs.
>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> From first principles, what package is waiting to be built in the
>>> eln-cache that Emacs is waiting for that prevents input?  AFAIK, the byte
>>> code is first loaded and available to run and only then replaced
>>> on-the-fly when native code is produced without error to replace it in the
>>> symbol's function cell.
>>> 
>>> 
>> 
>> 
>> 
>> No, this is not what the problem is. That's the thing that I'm saying has
>> been pretty well established in this thread, though it seems not well
>> enough.
>> 
>> 
>> 
>> The problem is that macOS, upon first *loading* the bundle in the main
>> Emacs process will undergo a security check (the one disabled by the
>> entitlements call you posted). This is a synchronous, blocking operation.
>> MacOS does not allow the process to load the bundle until it has verified
>> it, so the load is blocking. During this time, Emacs cannot receive input
>> because the main thread is completely blocked by the operating system. As
>> soon as you chain together hundreds of these macOS-initiated blocking
>> operations (every single bundle being loaded) you have a very poor,
>> stuttery, slow experience.
>> 
>> 
> 
> 
> 
> Okay so it is just the macOS binary scanner then and like any virus
> scanner interferes at the OS level.  A binary has to either opt out (as
> PROBLEMS document a method to do and the jimeh build does), or has to
> otherwise be whitelisted.  Every binary on macOS undergoes the same thing
> without being opted out so this isn't an Emacs issue.  I agree that Emacs
> loads "DLLs" and those can amplify the OS-imposed delay.  But only when
> they're loaded into memory.  Someone with a jillion packages surely
> doesn't need them all at once and can benefit from a carefully-crafted
> init.el with deferred package loading.
> 
> 

Yes, this is a stance one could take, and it will likely have the impact I was describing with regard to user's experience of Emacs.

My config is carefully crafted to only load what I need. It's also, so carefully crafted that it asynchronously and unobtrusively loads all of my packages in the background. The eln validation issue makes it impossible to truly do this in the background an unobtrusively.

On Sat, Jul 25, 2026 at 9:32 AM, Eli Zaretskii < [email protected] > wrote:

> 
> 
>> 
>> 
>> 
>> 
>> 
>> Date: Sat , 25 Jul 2026 15:56:52 +0000
>> 
>> From: "Aaron Jensen" < [email protected] > Cc: Stéphane Marks < [email protected]
>> >, "Eli Zaretskii"
>> 
>> < [email protected] >, [email protected] , [email protected] , [email protected]
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> I'm afraid that the notion that this is a CPU saturation problem is
>> demonstrably false as well.
>> 
>> 
>> 
> 
> 
> 
> If the reason is macOS validation of the *.eln files being loaded, then
> CPU saturation is not relevant at all: the validation most probably runs
> in the same thread as the one Emacs uses to load the*.eln files, because
> the validation most probably intercepts the system calls that load shared
> libraries.
> 
> 
> 
>> 
>> 
>> My suggested change was to move the first load to actually be
>> asynchronous.
>> 
>> 
>> 
> 
> 
> 
> That would mean Emacs will be unable to use the native code until it is
> loaded at some future time. IOW, this comes with some performance hit,
> which the current code avoids. Therefore, I would like to introduce this
> complication only if the problem is wide enough, and even then only on
> macOS. Please be more patient.
> 
> 
> 
> 

I'm being perfectly patient, only making a case in response to emails here. Like I said, I have my workaround and I'm fine. I don't really understand your criticism about "Emacs [being] unable to use the native code until it is loaded at some future time". My suggestion wasn't "some future time" it was to load it initially in the async worker and so that by the time the async worker completed its compilation, the file was compiled and macOS verified. As far as timeline goes, there's no difference. The only difference is where the blocking happens.

Bad ascii chart depicting what I'm saying:

Master

worker: | async compile | macOS verify |

Emacs:  ...............................| Load |

Block?  ...............................XXXXXXXX

Done?   .......................................DONE

With load-in-worker workaround

worker: | async compile |

Emacs:  ................| macOS verify | Load |

Block?  ................XXXXXXXXXXXXXXXXXXXXXXX

Done?   .......................................DONE

> 
> 
>> 
>> 
>> 
>> 
>> 
>> codesign --force --sign - \
>> 
>> --entitlements macos-disable-library-validation.entitlements \ src/emacs
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Are you using that?
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> If the Jimeh build uses this, it didn't work for me. Regardless, this is
>> exactly the thing that disables the security check macOS is doing. It is
>> certainly a "workaround". And it does, in fact, prevent the stutter. If
>> you're doing this
>> 
>> (or your build is) and you're not seeing problems, that explains it
>> completely. It seems somewhat strange to say
>> 
>> "I'm already doing the workaround and I don't have the problem!", so I'm
>> not sure exactly what you're getting at here. Without disabling
>> validation, this *is a main thread problem* because the entirety of macOS
>> compilation + validation is *not* done asynchronously.
>> 
>> 
>> 
>> 
>> 
>> 
>> 
> 
> 
> 
> TBH, I don't understand why that validation is need for *.eln files. Those
> are compiled on your machine, so what validation is necessary? It's not
> like you download unknown binaries from somewhere.
> 
> 
> 
> 

Yep, I hear you. I struggle to rationalize it somewhat as well. The thing I fall back to is that disabling the security check in Emacs disables it for *everything* not just eln files you compiled. But any other binary you may load for any reason. At the point of time we are in where supply chain attacks are a real threat, I don't feel comfortable disabling the checks I have available to me, plus it could run me into legal trouble when it comes to client obligations.

Aaron

> 
> 
>> 
>> 
>> 
> 
>