bug#81223: [PATCH] Fix file-missing error during async native compilation
James Cherti <[email protected]>
| Newsgroups | gmane.emacs.bugs |
|---|---|
| Message-ID | <[email protected]> |
Hello Andrea, It makes sense. I've updated the patch: Attached: 0001-V2-Fix-file-missing-error-during-async-native-compilati.patch On 2026-07-27 11:21, Andrea Corallo wrote: > James Cherti <[email protected]> writes: > >> On 2026-06-11 11:07, Sean Whitton wrote: >>> James Cherti [11/Jun 9:41am -04] wrote: >>>> From a8f13a55a4838a0d2ead62513c7d1a9616c1b501 Mon Sep 17 00:00:00 2001 >>>> From: James Cherti <[email protected]> >>>> Date: Thu, 11 Jun 2026 09:29:39 -0400 >>>> Subject: [PATCH] Fix file-missing error during async native compilation >>>> >>>> When a package is upgraded or deleted, the async compilation queue can >>>> retain stale jobs for files that no longer exist on disk. Attempting to >>>> resolve the target .eln filename for these missing files signals a >>>> file-missing error that disrupts the queue. >>>> >>>> * lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Verify source >>>> files exist before attempting to resolve their .eln paths in both the >>>> dispatch loop and the process sentinel. >>> >>> Shorten these remarks and add corresponding code comments, please. >>> >>> Adding Philip for review since it seems primarily a package.el issue. >> >> You can find attached v2 of the patch >> (0001-V2-Fix-file-missing-error-during-async-native-compilati.patch). >> >> Changes since the previous version: >> - Added explanatory comments regarding the file existence checks. >> >> For context, I want to clarify that this fix is not specifically tied >> to package management operations. It resolves a broader issue by >> handling any stale jobs in the async native compilation queue where >> the target files no longer exist on disk. >> >> -- >> James Cherti > > Hi James, > > Thanks for the patch. Couple of points: > >> GitHub: https://github.com/jamescherti >> Website: https://www.jamescherti.com/ >> >> From 55ba77809cb1d371f79eb4e59df8db819e100456 Mon Sep 17 00:00:00 2001 >> From: James Cherti <[email protected]> >> Date: Fri, 12 Jun 2026 06:07:23 -0400 >> Subject: [PATCH] Fix file-missing error during async native compilation >> >> The async native compilation queue can sometimes retain stale jobs for >> files that no longer exist on disk (for example, when files are removed >> during a package upgrade or deletion). Attempting to resolve the target >> .eln filename for these missing files signals a file-missing error. >> >> * lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Verify source >> files exist before attempting to resolve their .eln paths in both the >> dispatch loop and the process sentinel. >> --- >> lisp/emacs-lisp/comp-run.el | 23 ++++++++++++++++++----- >> 1 file changed, 18 insertions(+), 5 deletions(-) >> >> diff --git a/lisp/emacs-lisp/comp-run.el b/lisp/emacs-lisp/comp-run.el >> index 64e20327906..c70c5b0a512 100644 >> --- a/lisp/emacs-lisp/comp-run.el >> +++ b/lisp/emacs-lisp/comp-run.el >> @@ -282,13 +282,20 @@ comp--run-async-workers >> when (or native-comp-always-compile >> load ; Always compile when the compilation is >> ; commanded for late load. >> ;; Skip compilation if `comp-el-to-eln-filename' fails >> ;; to find a writable directory. >> - (with-demoted-errors "Async compilation :%S" >> - (file-newer-than-file-p >> - source-file (comp-el-to-eln-filename source-file)))) >> + (and >> + ;; Verify that the source file still exists on disk >> + ;; as a regular file before calling >> + ;; `comp-el-to-eln-filename'. This check prevents >> + ;; `file-missing' errors caused by stale jobs in the >> + ;; async compilation queue. >> + (file-regular-p source-file) >> + (with-demoted-errors "Async compilation :%S" >> + (file-newer-than-file-p >> + source-file (comp-el-to-eln-filename source-file))))) > > The file-regular-p check in the dispatch loop is currently bypassed when > either native-comp-always-compile or load is non-nil. Thus, stale jobs > in those modes still start a worker. Should the check wrap the entire or > condition? >> Also, the sentinel still has a check-then-use race: the source can > disappear between file-regular-p and comp-el-to-eln-filename, causing > the sentinel to signal before restarting the queue. > > I'm wondering, wouldn't maybe be safer to handle file-missing around > comp-el-to-eln-filename directly and ensure comp--run-async-workers is > always called? > Thanks > > Andrea > > > -- James Cherti GitHub: https://github.com/jamescherti Website: https://www.jamescherti.com/
0001-V2-Fix-file-missing-error-during-async-native-compilati.patch
(text/x-patch, 5.3 KB)
From fba7bda59f0dd19ae031681fafb8f64e5d8b4bcc Mon Sep 17 00:00:00 2001 From: James Cherti <[email protected]> Date: Wed, 29 Jul 2026 11:13:12 -0400 Subject: [PATCH] Fix file-missing error during async native compilation The async native compilation queue can sometimes retain stale jobs for files that no longer exist on disk (for example, when files are removed during a package upgrade or deletion). Attempting to resolve the target .eln filename for these missing files signals a file-missing error. * lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Verify source files exist before attempting to resolve their .eln paths in the dispatch loop. In the process sentinel, wrap the compilation block in a condition-case to catch file-missing errors and prevent the async worker queue from stalling. --- lisp/emacs-lisp/comp-run.el | 48 ++++++++++++++++++++++++------------- 1 file changed, 32 insertions(+), 16 deletions(-) diff --git a/lisp/emacs-lisp/comp-run.el b/lisp/emacs-lisp/comp-run.el index 64e20327906..203b8531172 100644 --- a/lisp/emacs-lisp/comp-run.el +++ b/lisp/emacs-lisp/comp-run.el @@ -277,18 +277,25 @@ comp--run-async-workers for (source-file . load) = (pop comp-files-queue) while source-file do (cl-assert (string-match-p comp-valid-source-re source-file) nil "`comp-files-queue' should be \".el\" files: %s" source-file) - when (or native-comp-always-compile - load ; Always compile when the compilation is - ; commanded for late load. - ;; Skip compilation if `comp-el-to-eln-filename' fails - ;; to find a writable directory. - (with-demoted-errors "Async compilation :%S" - (file-newer-than-file-p - source-file (comp-el-to-eln-filename source-file)))) + when (and + ;; Verify that the source file still exists on disk + ;; as a regular file before calling + ;; `comp-el-to-eln-filename'. This check prevents + ;; `file-missing' errors caused by stale jobs in the + ;; async compilation queue. + (file-regular-p source-file) + (or native-comp-always-compile + load ; Always compile when the compilation is + ; commanded for late load. + ;; Skip compilation if `comp-el-to-eln-filename' fails + ;; to find a writable directory. + (with-demoted-errors "Async compilation :%S" + (file-newer-than-file-p + source-file (comp-el-to-eln-filename source-file))))) do (let* ((expr `((require 'comp) (setq comp-async-compilation t warning-fill-column most-positive-fixnum) ,(let ((set (list 'setq))) (dolist (var '(comp-file-preloaded-p @@ -360,18 +367,27 @@ comp--run-async-workers (run-hook-with-args 'native-comp-async-cu-done-functions source-file) (comp--accept-and-process-async-output process) (ignore-errors (delete-file temp-file)) - (let ((eln-file (comp-el-to-eln-filename - source-file1))) - (when (and load1 - (zerop (process-exit-status - process)) - (file-exists-p eln-file)) - (native-elisp-load eln-file - (eq load1 'late)))) + ;; Catch the file-missing error that + ;; occurs if the original source file is + ;; deleted while the asynchronous worker + ;; is compiling it. Handling this error + ;; prevents the sentinel from aborting + ;; and ensures the compilation queue + ;; continues processing. + (condition-case nil + (let ((eln-file (comp-el-to-eln-filename + source-file1))) + (when (and load1 + (zerop (process-exit-status + process)) + (file-exists-p eln-file)) + (native-elisp-load eln-file + (eq load1 'late)))) + (file-missing nil)) (comp--run-async-workers)) :noquery (not native-comp-async-query-on-exit)))) (set-process-thread process nil) (puthash source-file process comp-async-compilations)) when (>= (comp--async-runnings) (comp--effective-async-max-jobs)) -- 2.54.0