Re: [DISCUSS] Fix for #12646 — moving project-loca l-repo out of target/
Guillaume Nodet <[email protected]>
| Newsgroups | gmane.comp.jakarta.turbine.maven.devel |
|---|---|
| Message-ID | <CAA66TpoU85GPD3F8NbJsQYJnNsNQ-o2wfer-puZkwEU7Fc+4mg@mail.gmail.com> |
Le mer. 5 août 2026 à 21:54, Romain Manni-Bucau <[email protected]> a écrit : > If we are going to break project.build.outputDirectory (.mvn/target) then > we should step back and break it keeping it sane so make it hierarchical > instead of flat (target/repo, target/build/foo.jar etc). > I think the idea was to use ${session.root}/.mvn/target/project-local-repo, which is in line with your suggestion. That said what we say if we want to leak and violate cleanable with clean > plugin rulé then it should belong to .m2 until we restructure build dir to > avoir a mess and a ton of New config for that purpose > (project.build.leakingDirectory?). > Yes good idea, i've just added a maven.project.local.repo property with a default value of ${maven.rootDirectory}/.mvn/target/project-local-repo specified in the ${MAVEN_HOME}/conf/maven-user.properties, (documents, but commented by default), so that anyone can change it. > Side question: nobody jumps on split repo solution so I assume we'll drop > the feature anytime soon in a coming release? 🤔 > Chained repo != split repo. Split repo (available from 3.9.x) has a main drawback imho., as this affects the global ~/.m2/repository, so all your maven projects need to be configured exactly the same way to use the same local repositories. For chained repo, I think this could definitely be leveraged to provide workspace isolation, it seems a really nice feature to add for 4.1, if there's anything to do in maven for it. Guillaume > Romain Manni-Bucau > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> > | Old > Blog <http://rmannibucau.wordpress.com> | Github > <https://github.com/rmannibucau> | LinkedIn > <https://www.linkedin.com/in/rmannibucau> | Book > < > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > Javaccino <https://javaccino.dev/> founder (Java/.NET service - contact > via > linkedin) > > Le mer. 5 août 2026, 21:47, Tamás Cservenák <[email protected]> a écrit > : > > > Right, will shut up now, just allow me one more question: > > what happens when a multi-project build builds AND uses a plugin from > > the reactor? > > > > T > > > > On Tue, 4 Aug 2026 at 23:36, Guillaume Nodet <[email protected]> wrote: > > > > > > I think we're talking past each other. The race condition in #12646 is > > not > > > about reactor limiting or partial builds. It happens in a plain full > mvn > > > clean install -T N — no -pl, no -f, no -r. The root module's clean > phase > > > deletes target/ while sibling modules concurrently write to > > > target/project-local-repo. That's the bug, and it needs fixing > regardless > > > of how we feel about partial builds. > > > > > > Now, on reactor limiting — you're actually making the case for > > > project-local-repo. You say mvn -f impl/maven-core shouldn't be > expected > > to > > > work because omitted modules need to be resolvable. That's exactly what > > > project-local-repo makes possible: after a full mvn verify, the > artifacts > > > are there for a subsequent partial build, without requiring install. > > That's > > > the feature MNG-7629 added. Whether we think that's a good feature is a > > > separate discussion — but it's shipped, it's in 4.0.0-rc6. > > > > > > The PR fixes a race condition in existing, released code. Can we focus > on > > > whether the fix is correct? Start a separate discussion to discuss > other > > > problems. > > > > > > > > > Le mar. 4 août 2026 à 23:25, Tamás Cservenák <[email protected]> a > > écrit : > > > > > > > "mvn verify and mvn package are supported lifecycle invocations in > > > > Maven" -- yes, they are, nobody claimed the opposite. > > > > "They work for single-module projects." -- as they per-def can have > no > > > > other dependencies than "external" ones (coming from remote > > > > repository; those are "done-done" already) > > > > "They're supposed to work for multi-module projects too" -- yes, and > > > > they do: I can just fine mvn verify on Maven master, it will work. > > > > But, I should not expect later on, that `mvn -f impl/maven-core` will > > > > work. > > > > > > > > To me this looks like "reactor limiting" does it wrong. Maven, right > > > > now, "pretend that the omitted module does not exist" (hence, it > needs > > > > to be resolvable), while the JAR, and whatever it produces, may be > > > > present in target from previous invocation... > > > > > > > > T > > > > > > > > On Tue, 4 Aug 2026 at 22:57, Guillaume Nodet <[email protected]> > > wrote: > > > > > > > > > > I don't think "isolation" is the right framing here. The question > is > > > > > simpler than that. > > > > > > > > > > mvn verify and mvn package are supported lifecycle invocations in > > Maven. > > > > > They work for single-module projects. They're supposed to work for > > > > > multi-module projects too — and they do, thanks to > > project-local-repo and > > > > > the ReactorReader. That mechanism was added (MNG-7629) precisely to > > make > > > > > the reactor resolve cross-module artifacts during builds that don't > > run > > > > > install. > > > > > > > > > > So we have a supported feature that has a race condition. The fix > is > > > > > straightforward: move the directory out of the blast radius of > clean. > > > > > That's what PR #12650 does. > > > > > > > > > > The alternative is to say: "actually, mvn package and mvn verify > are > > not > > > > > supported for multi-module builds — you must always run mvn > install." > > > > > That's a valid position, but it's a breaking change in semantics > that > > > > needs > > > > > a proper discussion, a deprecation path, and clear documentation. > We > > > > can't > > > > > just leave the race condition unfixed and tell users to install an > > > > > extension. > > > > > > > > > > If we want install to be the only goal people should use, we should > > make > > > > > that explicit. We could drop the mvn compile reactor support too > > (since > > > > > it's broken for the same reasons), make clean an option rather > than a > > > > goal, > > > > > and simplify the user experience to just mvn or mvn -c. If we were > to > > > > embed > > > > > MWM into Maven, dropping phase invocation entirely could be a path > > > > forward > > > > > — but that's a much larger conversation than fixing this race > > condition. > > > > > > > > > > MWM and chained local repositories solve a different problem — > > > > > cross-project, cross-branch isolation of installed artifacts. > That's > > > > > valuable, and it can coexist with project-local-repo. But it > doesn't > > make > > > > > the race condition go away, because it still requires install, and > > Maven > > > > > still supports builds without it. > > > > > > > > > > So: do we fix the bug in the supported workflow, or do we deprecate > > the > > > > > workflow? We've just released rc6 — deprecating a supported > workflow > > is a > > > > > Maven 5 conversation. The race condition fix is a 15-line change > > that can > > > > > land now. > > > > > > > > > > Guillaume > > > > > > > > > > Le mar. 4 août 2026 à 22:11, Tamás Cservenák <[email protected]> > a > > > > écrit : > > > > > > > > > > > Just to explain a bit more: > > > > > > Whenever I hear "I am doing this and this", I expect that it can > be > > > > > > continued with something like "because of this and this". > > > > > > > > > > > > Basically, that the actor DOING the thing knows WHY he is doing > the > > > > thing. > > > > > > > > > > > > So, I am interested in the WHYs. > > > > > > > > > > > > For example: If I am working across multiple checkouts > (projects), > > or > > > > > > I plan to limit the reactor later on (-rf etc), I want to make > > > > > > involved project(s) output build artifacts resolvable. > > > > > > This is my WHY for doing mvn install. > > > > > > > > > > > > Thanks > > > > > > T > > > > > > > > > > > > On Tue, 4 Aug 2026 at 21:59, Tamás Cservenák < > [email protected]> > > > > wrote: > > > > > > > > > > > > > > Jo, > > > > > > > > > > > > > > I get that, but this part: "many developers run mvn package or > > mvn > > > > > > verify". > > > > > > > I thought everyone runs "mvn clean install" :D Joke aside, why > > are > > > > > > > they doing that? > > > > > > > I mean, does running `mvn install` (instead of mvn verify) > incur > > some > > > > > > > huge overhead? > > > > > > > Or are they being told to do so? So what is the explanation to > > run > > > > > > > (n-1)th and not n-th phase? > > > > > > > > > > > > > > > > > > > > > T > > > > > > > > > > > > > > On Tue, 4 Aug 2026 at 21:48, Guillaume Nodet < > [email protected]> > > > > wrote: > > > > > > > > > > > > > > > > THi all, > > > > > > > > > > > > > > > > > > > > > > > > Following the discussion on whether chained local > repositories > > (à > > > > la > > > > > > MWM) > > > > > > > > could replace project-local-repo, I wanted to clarify the use > > cases > > > > > > each > > > > > > > > solves. They're complementary, not interchangeable. > > > > > > > > > > > > > > > > > > > > > > > > *What project-local-repo solves* > > > > > > > > > > > > > > > > project-local-repo was introduced in MNG-7629 to support > > > > intra-reactor > > > > > > > > partial and resumable builds without requiring mvn install. > It > > is > > > > > > populated > > > > > > > > on ProjectSucceeded regardless of the lifecycle goal — mvn > > > > package, mvn > > > > > > > > verify, anything that produces artifacts. > > > > > > > > > > > > > > > > > > > > > > > > This enables workflows like: > > > > > > > > - mvn package then mvn test -pl :child — child finds sibling > > > > artifacts > > > > > > > > - mvn verify fails at module-C, then mvn verify -rf > :module-C — > > > > > > modules A > > > > > > > > and B's artifacts (including classifiers like sources.jar, > > > > test-jar, > > > > > > > > consumer POMs) are available from the previous run > > > > > > > > - mvn package -DskipTests then mvn surefire:test -pl > > > > :single-module — > > > > > > > > resolves dependencies from siblings > > > > > > > > > > > > > > > > The key detail is that classified/attached artifacts > > (sources.jar, > > > > > > > > test-jar, javadoc.jar, consumer POMs) are only resolvable > from > > > > > > > > project-local-repo during resume. The target/classes fallback > > in > > > > > > > > ReactorReader only handles plain jars without classifiers. > > > > > > > > > > > > > > > > What chained local repositories solve > > > > > > > > > > > > > > > > Chained repos (via maven.repo.local.head / MWM) solve > > cross-project > > > > > > > > SNAPSHOT resolution with branch isolation. They allow > different > > > > > > branches to > > > > > > > > have isolated install targets, so working on multiple > branches > > (or > > > > > > multiple > > > > > > > > related projects) doesn't cause artifact conflicts in > > > > ~/.m2/repository. > > > > > > > > > > > > > > > > This enables workflows like: > > > > > > > > - Build maven-resolver SNAPSHOT on branch feature-x, then > build > > > > maven > > > > > > > > against it — without polluting ~/.m2/repository or > conflicting > > > > with the > > > > > > > > main branch > > > > > > > > - Multiple git worktrees of the same project, each with its > own > > > > install > > > > > > > > target > > > > > > > > > > > > > > > > Why one can't replace the other > > > > > > > > > > > > > > > > Chained repos require mvn install to populate the head > > repository. > > > > They > > > > > > > > cannot serve the project-local-repo use case because many > > > > developers > > > > > > run > > > > > > > > mvn package or mvn verify without install. After mvn package, > > > > > > classified > > > > > > > > artifacts are in project-local-repo (populated on > > ProjectSucceeded) > > > > > > but not > > > > > > > > in any local repository. > > > > > > > > > > > > > > > > Conversely, project-local-repo is scoped to a single reactor > — > > it > > > > > > cannot > > > > > > > > resolve artifacts across project boundaries. Chained repos > > solve > > > > this > > > > > > > > naturally. > > > > > > > > > > > > > > > > Regarding #12646 > > > > > > > > > > > > > > > > The race condition exists because project-local-repo > currently > > > > lives > > > > > > inside > > > > > > > > target/, where maven-clean-plugin deletes it during parallel > > > > builds. > > > > > > The > > > > > > > > proposed fix (PR #12650) moves it to > > > > .mvn/target/project-local-repo — > > > > > > > > outside the reach of maven-clean-plugin, while keeping the > > semantic > > > > > > split > > > > > > > > (.mvn/ = config, .mvn/target/ = build output, gitignored). > > > > > > > > > > > > > > > > This is a minimal, targeted fix. Both mechanisms can coexist > — > > > > chained > > > > > > > > repos for cross-project branch isolation, project-local-repo > > for > > > > > > > > intra-reactor partial builds without install. > > > > > > > > > > > > > > > > PR: https://github.com/apache/maven/pull/12650 > > > > > > > > Issue: https://github.com/apache/maven/issues/12646 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Le mar. 4 août 2026 à 21:26, Maarten Mulders < > > > > [email protected]> a > > > > > > > > écrit : > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > The statement that "the unsolicited install happens [...] > to > > > > make the > > > > > > > > > resume `-r` feature work" is only partially true. The > > original > > > > > > solution > > > > > > > > > to get `mvn -r` to work did /not/ involve installing all > > > > artifacts > > > > > > in a > > > > > > > > > project-local repo. It was later refactored/rewritten by > > /other/ > > > > > > people > > > > > > > > > to work the way you describe. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Maarten > > > > > > > > > > > > > > > > > > On August 4, 2026 at 15:07, Tamás Cservenák wrote: > > > > > > > > > > Howdy, > > > > > > > > > > > > > > > > > > > > I would just reflect on a funny fact (for me at least): > > > > > > > > > > > > > > > > > > > > This whole problem (being solved on this thread), stems > > from > > > > one > > > > > > > > > > single thing: the unsolicited install that Maven 4 does > (cf > > > > this to > > > > > > > > > > user invoking `mvn install` explicitly). > > > > > > > > > > > > > > > > > > > > Moreover, this unsolicited install happens, for one > thing, > > to > > > > make > > > > > > the > > > > > > > > > > resume `-r` feature work. > > > > > > > > > > > > > > > > > > > > And the true irony is, that this resume feature is backed > > and > > > > > > > > > > advertised by folks, whose mantra is "do not `mvn > install` > > but > > > > `mvn > > > > > > > > > > verify`" (as install "pollutes' your local repository", > > > > whatever > > > > > > that > > > > > > > > > > means). > > > > > > > > > > > > > > > > > > > > That mantra can be now extended with "... as Maven 4 will > > > > install > > > > > > it, > > > > > > > > > > even if you did not ask for it" :) > > > > > > > > > > > > > > > > > > > > The more I think about it, the more I find this funny. > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > T > > > > > > > > > > > > > > > > > > > > On Tue, 4 Aug 2026 at 14:56, Guillaume Nodet< > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > > >> I like the .mvn/target/project-local-repo which solves > the > > > > > > problem, > > > > > > > > > >> and also provides a good location where plugins could > move > > > > some > > > > > > > > > >> temporary data files without being disturbed by the > clean > > > > > > plugin. I'm > > > > > > > > > >> thinking about the flatten plugin, the release plugin, > and > > > > > > probably > > > > > > > > > >> more. > > > > > > > > > >> > > > > > > > > > >> Le lun. 3 août 2026 à 16:47, Slawomir Jaranowski > > > > > > > > > >> <[email protected]> a écrit : > > > > > > > > > >>> Hi, > > > > > > > > > >>> > > > > > > > > > >>> On Mon, 3 Aug 2026 at 14:09, Guillaume Nodet< > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > >>>> Hi all, > > > > > > > > > >>>> > > > > > > > > > >>>> I'd like to get your input on the fix for GH-12646 — a > > race > > > > > > condition > > > > > > > > > with > > > > > > > > > >>>> project-local-repo during parallel clean install > builds. > > > > > > > > > >>>> > > > > > > > > > >>>> The problem > > > > > > > > > >>>> > > > > > > > > > >>>> When the root pom.xml has a parent that is also part > of > > the > > > > > > reactor > > > > > > > > > (e.g. a > > > > > > > > > >>>> super-pom), MultiThreadedBuilder schedules the parent > > first, > > > > > > then > > > > > > > > > runs the > > > > > > > > > >>>> root project and child modules concurrently. Since > > clean and > > > > > > install > > > > > > > > > are > > > > > > > > > >>>> placed into the same TaskSegment, there is no barrier > > > > between > > > > > > them — > > > > > > > > > the > > > > > > > > > >>>> root project's maven-clean-plugin can delete target/ > > while > > > > > > sibling > > > > > > > > > modules > > > > > > > > > >>>> are concurrently writing artifacts into > > > > > > target/project-local-repo, > > > > > > > > > causing > > > > > > > > > >>>> the build to fail. > > > > > > > > > >>>> > > > > > > > > > >>>> Approaches considered > > > > > > > > > >>>> > > > > > > > > > >>>> 1. Lock-based synchronization (ReentrantReadWriteLock > in > > > > > > > > > ReactorReader) > > > > > > > > > >>>> > > > > > > > > > >>>> Acquire a write lock when the project owning target/ > > enters > > > > its > > > > > > clean > > > > > > > > > >>>> phase, and a read lock when installing artifacts. This > > > > prevents > > > > > > the > > > > > > > > > crash > > > > > > > > > >>>> (concurrent access) but does not guarantee ordering — > a > > > > module > > > > > > could > > > > > > > > > >>>> install artifacts before the root's clean starts, only > > to > > > > have > > > > > > them > > > > > > > > > wiped. > > > > > > > > > >>>> It also adds complexity to ReactorReader for what is > > > > > > fundamentally an > > > > > > > > > >>>> architectural problem. > > > > > > > > > >>>> > > > > > > > > > >>>> 2. Move project-local-repo to ~/.m2/ (user home) > > > > > > > > > >>>> > > > > > > > > > >>>> E.g. ~/.m2/local-repository/${projectName}_${hash}, > > similar > > > > to > > > > > > how > > > > > > > > > IntelliJ > > > > > > > > > >>>> stores project caches. This avoids both the race and > > the git > > > > > > concern, > > > > > > > > > but: > > > > > > > > > >>>> - Hard links (already used by ReactorReader) cannot > > cross > > > > > > filesystem > > > > > > > > > >>>> boundaries — if ~/.m2/ is on a different volume, every > > > > artifact > > > > > > > > > becomes a > > > > > > > > > >>>> full copy > > > > > > > > > >>>> - Stale directories accumulate when projects are > > > > deleted/moved, > > > > > > with > > > > > > > > > no > > > > > > > > > >>>> cleanup mechanism > > > > > > > > > >>>> - CI containers often share ~/.m2/ across builds, > > causing > > > > > > unwanted > > > > > > > > > >>>> accumulation > > > > > > > > > >>>> - Loses project locality (harder to inspect/debug) > > > > > > > > > >>>> > > > > > > > > > >>>> 3. Move project-local-repo to .mvn/project-local-repo > > > > (chosen) > > > > > > > > > >>>> > > > > > > > > > >>>> Move the directory from target/project-local-repo to > > > > > > > > > >>>> .mvn/project-local-repo. Since maven-clean-plugin only > > > > deletes > > > > > > > > > target/, the > > > > > > > > > >>>> race becomes structurally impossible. ReactorReader > > fully > > > > owns > > > > > > the > > > > > > > > > >>>> lifecycle — per-GAV cleanup when a project enters its > > clean > > > > > > phase, > > > > > > > > > install > > > > > > > > > >>>> on project success. The existing hard-link > optimization > > > > > > continues to > > > > > > > > > work > > > > > > > > > >>>> since both directories are on the same filesystem. > > > > > > > > > >>>> > > > > > > > > > >>>> The trade-off is that .mvn/project-local-repo needs to > > be > > > > > > gitignored. > > > > > > > > > This > > > > > > > > > >>>> follows the Gradle convention where .gradle/ at the > > project > > > > > > root is > > > > > > > > > >>>> universally in .gitignore templates. Maven could also > > > > document > > > > > > this > > > > > > > > > >>>> convention or consider auto-appending the entry. > > > > > > > > > >>>> > > > > > > > > > >>>> The fix itself is minimal — the core change is a > single > > > > line in > > > > > > > > > >>>> getProjectLocalRepo(), and the rest is removing the > lock > > > > > > > > > infrastructure > > > > > > > > > >>>> (net -40 lines). > > > > > > > > > >>>> > > > > > > > > > >>>> 4. Keep in target/ but use maven-clean-plugin fast > mode > > > > > > > > > >>>> > > > > > > > > > >>>> The fast clean option (maven.clean.fast=true, since > > plugin > > > > 3.2) > > > > > > > > > atomically > > > > > > > > > >>>> renames target/ instead of recursively deleting it. > This > > > > would > > > > > > likely > > > > > > > > > avoid > > > > > > > > > >>>> the race, but it's opt-in and not the default — users > > > > hitting > > > > > > the bug > > > > > > > > > would > > > > > > > > > >>>> need to know about it. > > > > > > > > > >>>> > > > > > > > > > >>>> FWIW, a PR has been raised on m-clean-p master branch > > (4.x) > > > > to > > > > > > > > > refactor the > > > > > > > > > >>>> fast cleaner (fixing problems) and make it the > default. > > > > > > > > > >>>> > > > > > > > > > >>>> Open questions > > > > > > > > > >>>> > > > > > > > > > >>>> - Is .mvn/ the right location, or should we consider a > > > > different > > > > > > > > > >>>> project-root directory? > > > > > > > > > >>> maybe .mvn/target/project-local-repo > > > > > > > > > >>> it should be ignored by git > > > > > > > > > >>> > > > > > > > > > >>>> - Should Maven auto-add .mvn/project-local-repo to > > > > .gitignore, > > > > > > or > > > > > > > > > document > > > > > > > > > >>>> it as a convention? > > > > > > > > > >>>> - Are there other considerations I'm missing? > > > > > > > > > >>>> > > > > > > > > > >>>> PR:https://github.com/apache/maven/pull/12650 > > > > > > > > > >>>> Issue:https://github.com/apache/maven/issues/12646 > > > > > > > > > >>>> > > > > > > > > > >>>> Looking forward to your thoughts. > > > > > > > > > >>>> > > > > > > > > > >>>> Cheers > > > > > > > > > >>>> Guillaume Nodet > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > > >>> -- > > > > > > > > > >>> Sławomir Jaranowski > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > >>> To unsubscribe, > e-mail:[email protected] > > > > > > > > > >>> For additional commands, > > e-mail:[email protected] > > > > > > > > > >>> > > > > > > > > > >> > > > > > > > > > >> -- > > > > > > > > > >> ------------------------ > > > > > > > > > >> Guillaume Nodet > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > >> To unsubscribe, e-mail:[email protected] > > > > > > > > > >> For additional commands, > e-mail:[email protected] > > > > > > > > > >> > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail:[email protected] > > > > > > > > > > For additional commands, > e-mail:[email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > ------------------------ > > > > > > > > Guillaume Nodet > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > -- > > > > > ------------------------ > > > > > Guillaume Nodet > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > -- > > > ------------------------ > > > Guillaume Nodet > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > -- ------------------------ Guillaume Nodet