bug#81223: [PATCH] Fix file-missing error during async native compilation

James Cherti <[email protected]> writes:

> On 2026-06-11 11:07, Sean Whitton wrote:
>> James Cherti [11/Jun  9:41am -04] wrote:
>>>  From a8f13a55a4838a0d2ead62513c7d1a9616c1b501 Mon Sep 17 00:00:00 2001
>>> From: James Cherti <[email protected]>
>>> Date: Thu, 11 Jun 2026 09:29:39 -0400
>>> Subject: [PATCH] Fix file-missing error during async native compilation
>>>
>>> When a package is upgraded or deleted, the async compilation queue can
>>> retain stale jobs for files that no longer exist on disk.  Attempting to
>>> resolve the target .eln filename for these missing files signals a
>>> file-missing error that disrupts the queue.
>>>
>>> * lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Verify source
>>> files exist before attempting to resolve their .eln paths in both the
>>> dispatch loop and the process sentinel.
>>
>> Shorten these remarks and add corresponding code comments, please.
>>
>> Adding Philip for review since it seems primarily a package.el issue.
>
> You can find attached v2 of the patch
> (0001-V2-Fix-file-missing-error-during-async-native-compilati.patch).
>
> Changes since the previous version:
> - Added explanatory comments regarding the file existence checks.
>
> For context, I want to clarify that this fix is not specifically tied
> to package management operations. It resolves a broader issue by
> handling any stale jobs in the async native compilation queue where
> the target files no longer exist on disk.
>
> --
> James Cherti

Hi James,

Thanks for the patch. Couple of points:

> GitHub: https://github.com/jamescherti
> Website: https://www.jamescherti.com/
>
> From 55ba77809cb1d371f79eb4e59df8db819e100456 Mon Sep 17 00:00:00 2001
> From: James Cherti <[email protected]>
> Date: Fri, 12 Jun 2026 06:07:23 -0400
> Subject: [PATCH] Fix file-missing error during async native compilation
>
> The async native compilation queue can sometimes retain stale jobs for
> files that no longer exist on disk (for example, when files are removed
> during a package upgrade or deletion). Attempting to resolve the target
> .eln filename for these missing files signals a file-missing error.
>
> * lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Verify source
> files exist before attempting to resolve their .eln paths in both the
> dispatch loop and the process sentinel.
> ---
>  lisp/emacs-lisp/comp-run.el | 23 ++++++++++++++++++-----
>  1 file changed, 18 insertions(+), 5 deletions(-)
>
> diff --git a/lisp/emacs-lisp/comp-run.el b/lisp/emacs-lisp/comp-run.el
> index 64e20327906..c70c5b0a512 100644
> --- a/lisp/emacs-lisp/comp-run.el
> +++ b/lisp/emacs-lisp/comp-run.el
> @@ -282,13 +282,20 @@ comp--run-async-workers
>           when (or native-comp-always-compile
>                    load ; Always compile when the compilation is
>                         ; commanded for late load.
>                    ;; Skip compilation if `comp-el-to-eln-filename' fails
>                    ;; to find a writable directory.
> -                  (with-demoted-errors "Async compilation :%S"
> -                    (file-newer-than-file-p
> -                     source-file (comp-el-to-eln-filename source-file))))
> +                  (and
> +                   ;; Verify that the source file still exists on disk
> +                   ;; as a regular file before calling
> +                   ;; `comp-el-to-eln-filename'. This check prevents
> +                   ;; `file-missing' errors caused by stale jobs in the
> +                   ;; async compilation queue.
> +                   (file-regular-p source-file)
> +                   (with-demoted-errors "Async compilation :%S"
> +                     (file-newer-than-file-p
> +                      source-file (comp-el-to-eln-filename source-file)))))

The file-regular-p check in the dispatch loop is currently bypassed when
either native-comp-always-compile or load is non-nil. Thus, stale jobs
in those modes still start a worker. Should the check wrap the entire or
condition?

Also, the sentinel still has a check-then-use race: the source can
disappear between file-regular-p and comp-el-to-eln-filename, causing
the sentinel to signal before restarting the queue.

I'm wondering, wouldn't maybe be safer to handle file-missing around
comp-el-to-eln-filename directly and ensure comp--run-async-workers is
always called?

Thanks

  Andrea