bug#81223: [PATCH] Fix file-missing error during async native compilation

James Cherti <[email protected]> writes:

> Hello Andrea,
>
> It makes sense. I've updated the patch:
>
> Attached: 0001-V2-Fix-file-missing-error-during-async-native-compilati.patch
>
> On 2026-07-27 11:21, Andrea Corallo wrote:
>> James Cherti <[email protected]> writes:
>> 
>>> On 2026-06-11 11:07, Sean Whitton wrote:
>>>> James Cherti [11/Jun  9:41am -04] wrote:
>>>>>   From a8f13a55a4838a0d2ead62513c7d1a9616c1b501 Mon Sep 17 00:00:00 2001
>>>>> From: James Cherti <[email protected]>
>>>>> Date: Thu, 11 Jun 2026 09:29:39 -0400
>>>>> Subject: [PATCH] Fix file-missing error during async native compilation
>>>>>
>>>>> When a package is upgraded or deleted, the async compilation queue can
>>>>> retain stale jobs for files that no longer exist on disk.  Attempting to
>>>>> resolve the target .eln filename for these missing files signals a
>>>>> file-missing error that disrupts the queue.
>>>>>
>>>>> * lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Verify source
>>>>> files exist before attempting to resolve their .eln paths in both the
>>>>> dispatch loop and the process sentinel.
>>>>
>>>> Shorten these remarks and add corresponding code comments, please.
>>>>
>>>> Adding Philip for review since it seems primarily a package.el issue.
>>>
>>> You can find attached v2 of the patch
>>> (0001-V2-Fix-file-missing-error-during-async-native-compilati.patch).
>>>
>>> Changes since the previous version:
>>> - Added explanatory comments regarding the file existence checks.
>>>
>>> For context, I want to clarify that this fix is not specifically tied
>>> to package management operations. It resolves a broader issue by
>>> handling any stale jobs in the async native compilation queue where
>>> the target files no longer exist on disk.
>>>
>>> --
>>> James Cherti
>> Hi James,
>> Thanks for the patch. Couple of points:
>> 
>>> GitHub: https://github.com/jamescherti
>>> Website: https://www.jamescherti.com/
>>>
>>>  From 55ba77809cb1d371f79eb4e59df8db819e100456 Mon Sep 17 00:00:00 2001
>>> From: James Cherti <[email protected]>
>>> Date: Fri, 12 Jun 2026 06:07:23 -0400
>>> Subject: [PATCH] Fix file-missing error during async native compilation
>>>
>>> The async native compilation queue can sometimes retain stale jobs for
>>> files that no longer exist on disk (for example, when files are removed
>>> during a package upgrade or deletion). Attempting to resolve the target
>>> .eln filename for these missing files signals a file-missing error.
>>>
>>> * lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Verify source
>>> files exist before attempting to resolve their .eln paths in both the
>>> dispatch loop and the process sentinel.
>>> ---
>>>   lisp/emacs-lisp/comp-run.el | 23 ++++++++++++++++++-----
>>>   1 file changed, 18 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/lisp/emacs-lisp/comp-run.el b/lisp/emacs-lisp/comp-run.el
>>> index 64e20327906..c70c5b0a512 100644
>>> --- a/lisp/emacs-lisp/comp-run.el
>>> +++ b/lisp/emacs-lisp/comp-run.el
>>> @@ -282,13 +282,20 @@ comp--run-async-workers
>>>            when (or native-comp-always-compile
>>>                     load ; Always compile when the compilation is
>>>                          ; commanded for late load.
>>>                     ;; Skip compilation if `comp-el-to-eln-filename' fails
>>>                     ;; to find a writable directory.
>>> -                  (with-demoted-errors "Async compilation :%S"
>>> -                    (file-newer-than-file-p
>>> -                     source-file (comp-el-to-eln-filename source-file))))
>>> +                  (and
>>> +                   ;; Verify that the source file still exists on disk
>>> +                   ;; as a regular file before calling
>>> +                   ;; `comp-el-to-eln-filename'. This check prevents
>>> +                   ;; `file-missing' errors caused by stale jobs in the
>>> +                   ;; async compilation queue.
>>> +                   (file-regular-p source-file)
>>> +                   (with-demoted-errors "Async compilation :%S"
>>> +                     (file-newer-than-file-p
>>> +                      source-file (comp-el-to-eln-filename source-file)))))
>> The file-regular-p check in the dispatch loop is currently bypassed
>> when
>> either native-comp-always-compile or load is non-nil. Thus, stale jobs
>> in those modes still start a worker. Should the check wrap the entire or
>> condition?
>>> Also, the sentinel still has a check-then-use race: the source can
>> disappear between file-regular-p and comp-el-to-eln-filename, causing
>> the sentinel to signal before restarting the queue.
>> I'm wondering, wouldn't maybe be safer to handle file-missing around
>> comp-el-to-eln-filename directly and ensure comp--run-async-workers is
>> always called? > Thanks
>>    Andrea
>> 
>
>
> --
> James Cherti
> GitHub: https://github.com/jamescherti
> Website: https://www.jamescherti.com/
>
> From fba7bda59f0dd19ae031681fafb8f64e5d8b4bcc Mon Sep 17 00:00:00 2001
> From: James Cherti <[email protected]>
> Date: Wed, 29 Jul 2026 11:13:12 -0400
> Subject: [PATCH] Fix file-missing error during async native compilation
>
> The async native compilation queue can sometimes retain stale jobs for
> files that no longer exist on disk (for example, when files are removed
> during a package upgrade or deletion). Attempting to resolve the target
> .eln filename for these missing files signals a file-missing error.
>
> * lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Verify source
> files exist before attempting to resolve their .eln paths in the
> dispatch loop. In the process sentinel, wrap the compilation block in a
> condition-case to catch file-missing errors and prevent the async worker
> queue from stalling.

Unrelated to the contents of this patch: You should mention the bug
number ("(bug#81223)")in the commit message, see the CONTRIBUTE file.

> ---
>  lisp/emacs-lisp/comp-run.el | 48 ++++++++++++++++++++++++-------------
>  1 file changed, 32 insertions(+), 16 deletions(-)
>
> diff --git a/lisp/emacs-lisp/comp-run.el b/lisp/emacs-lisp/comp-run.el
> index 64e20327906..203b8531172 100644
> --- a/lisp/emacs-lisp/comp-run.el
> +++ b/lisp/emacs-lisp/comp-run.el
> @@ -277,18 +277,25 @@ comp--run-async-workers
>           for (source-file . load) = (pop comp-files-queue)
>           while source-file
>           do (cl-assert (string-match-p comp-valid-source-re source-file) nil
>                         "`comp-files-queue' should be \".el\" files: %s"
>                         source-file)
> -         when (or native-comp-always-compile
> -                  load ; Always compile when the compilation is
> -                       ; commanded for late load.
> -                  ;; Skip compilation if `comp-el-to-eln-filename' fails
> -                  ;; to find a writable directory.
> -                  (with-demoted-errors "Async compilation :%S"
> -                    (file-newer-than-file-p
> -                     source-file (comp-el-to-eln-filename source-file))))
> +         when (and
> +               ;; Verify that the source file still exists on disk
> +               ;; as a regular file before calling
> +               ;; `comp-el-to-eln-filename'. This check prevents
> +               ;; `file-missing' errors caused by stale jobs in the
> +               ;; async compilation queue.
> +               (file-regular-p source-file)
> +               (or native-comp-always-compile
> +                   load ; Always compile when the compilation is
> +                        ; commanded for late load.
> +                   ;; Skip compilation if `comp-el-to-eln-filename' fails
> +                   ;; to find a writable directory.
> +                   (with-demoted-errors "Async compilation :%S"
> +                     (file-newer-than-file-p
> +                      source-file (comp-el-to-eln-filename source-file)))))
>           do (let* ((expr `((require 'comp)
>                             (setq comp-async-compilation t
>                                   warning-fill-column most-positive-fixnum)
>                             ,(let ((set (list 'setq)))
>                                (dolist (var '(comp-file-preloaded-p
> @@ -360,18 +367,27 @@ comp--run-async-workers
>                                 (run-hook-with-args
>                                  'native-comp-async-cu-done-functions
>                                  source-file)
>                                 (comp--accept-and-process-async-output process)
>                                 (ignore-errors (delete-file temp-file))
> -                               (let ((eln-file (comp-el-to-eln-filename
> -                                                source-file1)))
> -                                 (when (and load1
> -                                            (zerop (process-exit-status
> -                                                    process))
> -                                            (file-exists-p eln-file))
> -                                   (native-elisp-load eln-file
> -                                                      (eq load1 'late))))
> +                               ;; Catch the file-missing error that
> +                               ;; occurs if the original source file is
> +                               ;; deleted while the asynchronous worker
> +                               ;; is compiling it. Handling this error
> +                               ;; prevents the sentinel from aborting
> +                               ;; and ensures the compilation queue
> +                               ;; continues processing.
> +                               (condition-case nil
> +                                   (let ((eln-file (comp-el-to-eln-filename
> +                                                    source-file1)))
> +                                     (when (and load1
> +                                                (zerop (process-exit-status
> +                                                        process))
> +                                                (file-exists-p eln-file))
> +                                       (native-elisp-load eln-file
> +                                                          (eq load1 'late))))
> +                                 (file-missing nil))
>                                 (comp--run-async-workers))
>                               :noquery (not native-comp-async-query-on-exit))))
>                (set-process-thread process nil)
>                (puthash source-file process comp-async-compilations))
>           when (>= (comp--async-runnings) (comp--effective-async-max-jobs))