Re: some things about Closure

One further AI data point below.

Cheers, Paul.

---------------

A follow-up on the custom-call question, since "does anything in the wild
rely on the old behaviour?" deserved data rather than hand-waving. I survey=
ed
the Closure subclasses in the most widely deployed Groovy-consuming project=
s
(Jenkins pipelines, Gradle, Grails, Spock, Geb, Nextflow), plus a broader
GitHub sweep for `extends Closure` in Java/Kotlin/Groovy code.

First, the exposed pattern =E2=80=94 the only shape whose behaviour 12164/1=
2165
changed =E2=80=94 is narrow: a hand-written Closure subclass declaring a ty=
ped or
multi-arg call overload, WITHOUT overriding call(Object...) itself (a varar=
gs
override bypasses the base Closure.call and its cache entirely), without a
matching doCall, invoked through the Java/GDK path with a type-matching
argument. Previously that raised MissingMethodException; now it dispatches
(consistent with what the dynamic path has done since at least 3.x).

The census =E2=80=94 ten real Closure subclasses examined, zero in the risk=
y shape:

  Jenkins CpsClosure            overrides call(), call(Object),
  (every CPS pipeline)          call(Object...) AND declares doCall
                                -> varargs override bypasses the cache
  Nextflow TaskClosure          overrides all three call forms -> same bypa=
ss
  Gradle KotlinClosure0-3       typed doCall only, per the intended contrac=
t
  (all Kotlin-DSL interop)      -> nothing for a call-keyed cache to see
  Grails TagBodyClosure         all three call forms + doCall -> bypass
  Grails TagOutput
        .ConstantClosure        three doCall overloads +
call(Object...) -> bypass
  Grails MethodInvokingClosure  call(Object[]) =3D=3D the varargs erasure -=
> bypass
  Grails MappingCapturingClosure  overrides call(Object...) -> bypass
  Geb InvocationForwarding      protected doCall(Object[]) only -> pure
                                contract shape
  Spock                         no Closure subclasses at all: its
                                Class<? extends Closure> annotation attribu=
tes
                                (@IgnoreIf etc.) take user closure LITERALS=
,
                                i.e. generated classes

Every one is either "override call(Object...)" (bypasses everything) or
"declare doCall only" =E2=80=94 and there is a selection-effect reason for =
that: the
risky shape was BROKEN on the GDK path in every prior release (the
MissingMethodException is a crash, not a feature), while the dynamic path
dispatched it fine, so anything that hit it saw an obvious inconsistency bu=
g
and got fixed into one of the two immune shapes. Code depending on the MME
itself would have to construct that broken-by-design subclass, hand it to G=
DK
iteration, and catch the MME as control flow =E2=80=94 I could not construc=
t a
plausible DSL doing that, and the sweep turned up none.

The census also produced a finding that I think strengthens the case for th=
e
doCall-keyed direction rather than against it: KotlinClosure0-3 and Geb's
InvocationForwarding declare doCall and no call overloads, so every Gradle
Kotlin-DSL Groovy-interop closure (and Geb's binding forwarders) is on the
full metaclass path for GDK calls today =E2=80=94 the current call-keyed ca=
che never
serves them. Re-keying the cache on doCall overloads (i.e. making the fast
path literally implement "call selects doCall") would extend the measured
2-6x dispatch win to them for free. One implementation note from the survey=
:
Geb's doCall is protected, so the lookup needs a getDeclaredMethods walk wi=
th
setAccessible =E2=80=94 the same access the metaclass already exercises =E2=
=80=94 rather than
getMethods().

So the options as I see them:

  (a) keep call-keying: consistent dynamic/Java behaviour, but conceptually
      wrong per the contract, and doCall-style adapters (Gradle Kotlin DSL,
      Geb) stay on the slow path;
  (b) re-key on doCall (keeping 11911's call(Object)-without-doCall compat)=
:
      contract-aligned, same performance for generated closures (their
      call(T) bridges mirror doCall(T) one-to-one), Gradle/Geb adapters gai=
n
      the fast path, and the only behavioural delta is code that previously
      crashed;
  (c) maximal caution: gate the typed/multi-arg lookup on the
      GeneratedClosure marker =E2=80=94 zero ecosystem delta, keeps the win=
s, but
      leaves the conceptual question open and the doCall-style adapters slo=
w.

Given the data I'd go with (b).

On Tue, Jul 14, 2026 at 10:50=E2=80=AFPM Paul King <[email protected]> wro=
te:
>
> Hi Jochen,
>
> AI read below. It could be wrong. I am still pondering but won't get
> time again until my tomorrow. So this is something to read in the
> meantime.
>
> Cheers, Paul.
>
> --------
>
> no disagreement with the conceptual core =E2=80=94 doCall is the semantic=
 source of
> truth, call is its polymorphic entry, and PackedClosure needs explicit MO=
P
> standing rather than the accidental one it has. But before improvements a=
re
> designed, two factual findings are worth having on the table, because the=
y
> change what "aligning with the contract" means in practice.
>
> 1) The doCall-only contract has never held on the dynamic path.
>
> I tested a Closure subclass declaring only `String call(String)` (no doCa=
ll)
> across releases:
>
>                        dynamic cl("foo")    GDK collect(cl)
>   Groovy 3.0.21-25     works                MissingMethodException
>   Groovy 4.0.32        works                MissingMethodException
>   Groovy 5.0.6         works                MissingMethodException
>   Groovy 6.0.0-alpha-1 works                MissingMethodException
>   master (12164/65)    works                works
>
> So ClosureMetaClass has dispatched declared call overloads for at least f=
our
> major versions; only the Java entry (Closure.call -> invokeMethod("doCall=
"))
> enforced doCall-only, meaning the same cl("foo") succeeded from Groovy co=
de
> and failed from each(). What GROOVY-12164/12165 changed is that the Java =
path
> now matches the dynamic path =E2=80=94 the same direction GROOVY-11911 al=
ready took
> for call(Object)-without-doCall subclasses. Your doCall expectations
> themselves hold exactly on master: with doCall(int), cl(1) works and
> cl("foo") is a MissingMethodException on both paths (the typed guards dec=
line
> to the metaclass, which coerces or fails as before).
>
> That leaves a genuine decision rather than a bug report: either custom ca=
ll
> overloads are accepted as the (now consistent) de-facto contract, or
> doCall-only is enforced on BOTH paths =E2=80=94 which is a behavioural br=
eak with
> 3.x-6.x dynamic dispatch. If the latter, the mechanical cost is small: th=
e
> CallOverride cache re-keys on doCall overloads instead of call overloads
> (keeping 11911's call(Object) compatibility case), and none of the measur=
ed
> wins move, because generated closure classes' call(T) bridges mirror thei=
r
> doCall(T) one-to-one =E2=80=94 the cache just changes lookup key. Happy t=
o do that if
> we agree doCall-only is the contract.
>
> 2) On PackedClosure, I agree with each observation, with one correction a=
nd
> one caveat:
>
> - "only one doCall analog possible": already handled by declining =E2=80=
=94 a closure
>   with default parameter values (the multiple-doCall case) is not packabl=
e
>   and keeps its class. It is extensible later (one table id per generated
>   arity) if ever wanted.
> - not a GeneratedClosure: conceded =E2=80=94 and notably the naive fix is=
 unsound.
>   ClosureMetaClass resolves the "this" type for GeneratedClosure by walki=
ng
>   getEnclosingClass(), which assumes closure classes are inner classes of=
 the
>   owner; PackedClosure is a top-level adapter, so the marker would break =
that.
>   Your sentence "strictly seen PackedClosure requires its own logic in
>   MetaClass" is exactly right, and I'd rather add that explicit handling =
than
>   overload the marker.
> - one shared MetaClass for all packed closures: conceded, and it is the o=
ne
>   semantic difference that cannot be engineered away =E2=80=94 per-litera=
l metaclass
>   is definitionally what packing removes. Class-level metaClass changes o=
n
>   PackedClosure would affect every packed closure. The delegate/
>   resolveStrategy axis is already fenced by the runtime guard; this axis
>   needs documenting as a caveat of the opt-in feature (and is one more re=
ason
>   packing must stay off for code that plays MOP games).
> - "the call path seems to be the only way to invoke the real method": tru=
e,
>   but I'd frame what that path now IS: a statically compiled, metaclass-f=
ree,
>   inlinable route (adapter -> per-arity dispatch table -> direct invocati=
on of
>   the typed hoisted body). For statically compiled callers, invokevirtual
>   call() on a packed closure reaches a typed body with no MOP involvement=
 at
>   all =E2=80=94 which is the closest thing we currently have to your poin=
t (6),
>   since the typed doCall of a closure class is hidden behind Closure<T> f=
or
>   static compilation, but a packed closure's entry is structurally fixed.
>   Recent measurements: packed closures now run 1.8-4.7x FASTER than gener=
ated
>   closure classes on capturing shapes with byte-identical behaviour, larg=
ely
>   because of this property.
> - one point in the architecture's favour under your criterion (2): dynami=
c
>   call sites invoking closures are megamorphic across per-literal closure
>   classes, but monomorphic on the single PackedClosure class =E2=80=94 th=
e per-target
>   selection moves into a switch the JIT handles well. Indy caching gets
>   easier, not harder.
>
> So: agreement on the concepts, two concrete workstreams if we also agree =
on
> the remedies =E2=80=94 (a) decide the custom-call question and, if doCall=
-only wins,
> re-key the CallOverride cache on doCall for both paths; (b) give PackedCl=
osure
> explicit ClosureMetaClass (and Selector) handling plus documentation of t=
he
> shared-metaclass caveat. I'd take both.
>
> On Tue, Jul 14, 2026 at 8:09=E2=80=AFPM Jochen Theodorou <[email protected]=
rg> wrote:
> >
> > Hi all,
> >
> >
> > when looking at https://github.com/apache/groovy/pull/2710 I was
> > thinking that something is going wrong here on a conceptual level.
> >
> > we have quite a few cases to consider here, but in general (in no order
> > really):
> > (1) we do not want to call invokeMethod, getProperty or setProperty
> > directly if we can avoid it
> > (2) for dynamic Groovy this implies caching using invokedynamic and
> > getting the information from the meta class now
> > (3) GeneratedClosure allows a simplified path since we know there is no
> > custom MOP logic for these with default MetaClass
> > (4) call is supposed to be used to call doCall only
> > (5) providing custom call methods is actually not intended
> > (6) static compilation needs to call doCall directly
> > (7) static compilation needs to can't go through getProperty or setProp=
erty
> > (8) a Closure may define multiple doCall methods.
> >
> > What I mean with 4+5.
> > If you define subclass Closure and provide a call(String) method and yo=
u
> > do a call like cl("foo") (cl is the instance of the custom Closure),
> > then this call is supposed to fail. There is no doCall method at all.
> > If you subclass Closure and provide a doCall(int) method then cl(1) is
> > supposed to work, but cl("foo") still fails since there is no doCall
> > method fitting that argument.
> >
> > Which leads to (6)
> > The big problem for static compilation is that it this doCall method
> > might be hidden if we just provide Closure<T> as type. And that means
> > static compilation may have to fall back to dynamic invocation for that
> > - or go through call, which does a kind of dynamic invocation. So numbe=
r
> > (6) (and probably 7) can currently actually not be realized.
> >
> > Now we added PackedClosure, which is very similar to GeneratedClosure,
> > except that there is no doCall method and the method is instead in the
> > host class, plus there is currently only one doCall analog possible plu=
s
> > nothing in the PackedClosure may escape the Closure. Also, PackedClosur=
e
> > replaces GeneratedClosure in some cases, without being a
> > GeneratedClosure, so the MOP behaviour is different for the two - which
> > could lead to problems. And again here the call-path seems to be the
> > only way to invoke the real method. Strictly seen PackedClosure require=
s
> > its own logic in MetaClass. And unlike GeneratedClosure, all
> > PackedClosure have only 1 MetaClass, which they share
> >
> > But it is not only static compilation and the usage of PackedClosure
> > that have trouble with this architecture it is also invocation through
> > Java, especially our GDK.
> >
> > Before making any suggestions about improvements - does somebody
> > disagree with this?
> >
> > bye Jochen