Re: Script caching

Hi Per,

that looks like a good idea & well written proposal to me G-)

Cheers,
mg

Am 01.06.2026 um 19:15 schrieb Per Nyfelt:
>
> Hi,
>
> I think it would be nice if we supported caching for Groovy scripts to 
> speed up execution time for subsequent runs. Below is a GEP style 
> proposal for that. What do you think?
>
>
>   GEP: Persistent Script Compilation Cache for the Groovy Command-Line
>   Launcher
>
>
>     Metadata
>
> *Type:* Feature
> *Status:* Draft
> *Target Groovy Version:* TBD
> *Author:* Per Nyfelt
> *Discussion:* TBD
> *Created:* 2026-06-01
>
>
>     Abstract
>
> This proposal introduces an optional persistent compilation cache for 
> Groovy scripts executed through the |groovy| command-line launcher.
>
> When a script is executed, the launcher may store the generated class 
> files in a local cache. On later executions, if the script source and 
> relevant compilation inputs are unchanged, the launcher may load the 
> cached class files instead of parsing, transforming, and compiling the 
> script again.
>
> The goal is to improve startup time for short-lived Groovy scripts and 
> command-line tools while preserving current semantics by default.
>
>
>     Motivation
>
> Groovy is frequently used for scripting, automation, build tooling, 
> data processing, and command-line utilities. In these use cases, 
> process lifetime is often short, and startup overhead matters.
>
> For simple scripts, actual execution time may be negligible compared to:
>
>  *
>
>     JVM startup
>
>  *
>
>     loading the Groovy runtime
>
>  *
>
>     parsing the script
>
>  *
>
>     applying AST transformations
>
>  *
>
>     bytecode generation
>
>  *
>
>     class loading
>
> Recent and ongoing JVM work, such as CDS and Project Leyden, can 
> reduce JVM startup and class-loading overhead. However, raw Groovy 
> script execution still pays the cost of compiling the script source on 
> every invocation.
>
> Groovy already caches compiled script classes during the lifetime of a 
> |GroovyClassLoader|, but this cache is lost when the process exits. A 
> persistent launcher-level cache would allow repeated invocations of 
> unchanged scripts to skip most compilation work.
>
> This would make Groovy more attractive for command-line scripting and 
> developer tooling, especially compared with languages and runtimes 
> that already persist bytecode or compilation artifacts between runs.
>
>
>     Goals
>
> The goals of this proposal are:
>
> 1.
>
>     Reduce repeated startup overhead for unchanged Groovy scripts.
>
> 2.
>
>     Avoid changing Groovy language semantics.
>
> 3.
>
>     Make the feature safe and conservative by default.
>
> 4.
>
>     Provide explicit ways to disable, clear, and inspect the cache.
>
> 5.
>
>     Ensure cache invalidation accounts for relevant compilation inputs.
>
> 6.
>
>     Allow future integration with JVM startup technologies such as CDS
>     or Leyden-style caches.
>
>
>     Non-Goals
>
> This proposal does not aim to:
>
> 1.
>
>     Replace |groovyc|.
>
> 2.
>
>     Change Groovy compilation semantics.
>
> 3.
>
>     Cache arbitrary runtime results.
>
> 4.
>
>     Guarantee improved performance for all scripts.
>
> 5.
>
>     Provide a distributed or shared build cache.
>
> 6.
>
>     Cache scripts run inside long-lived embedded Groovy runtimes
>     unless those runtimes explicitly opt in.
>
> 7.
>
>     Solve dependency resolution caching for |@Grab|, although it must
>     interact safely with it.
>
>
>     Proposed Behavior
>
> When running:
>
> |groovy myscript.groovy |
>
> the launcher may:
>
> 1.
>
>     Compute a cache key from the script and compilation environment.
>
> 2.
>
>     Look for previously generated class files matching that key.
>
> 3.
>
>     If found and valid, load the cached classes.
>
> 4.
>
>     Otherwise compile the script normally and store the generated
>     classes in the cache.
>
> The user-visible behavior of the script must remain the same as if the 
> script had been compiled from source during that invocation.
>
>
>     Cache Location
>
> The default cache location should follow platform conventions.
>
> Suggested defaults:
>
> |Linux/Unix: $XDG_CACHE_HOME/groovy/script-cache or 
> ~/.cache/groovy/script-cache macOS: 
> ~/Library/Caches/Groovy/script-cache Windows: 
> %LOCALAPPDATA%\Groovy\script-cache |
>
> A system property or environment variable should allow overriding the 
> location:
>
> |groovy -Dgroovy.script.cache.dir=/path/to/cache myscript.groovy |
>
> Possible environment variable:
>
> |GROOVY_SCRIPT_CACHE_DIR=/path/to/cache |
>
>
>     Enabling and Disabling
>
> The cache should initially be opt-in unless the Groovy project decides 
> the invalidation model is sufficiently conservative for default use.
>
> Possible command-line options:
>
> |groovy --script-cache myscript.groovy groovy --no-script-cache 
> myscript.groovy groovy --clear-script-cache |
>
> Possible system properties:
>
> |-Dgroovy.script.cache=true -Dgroovy.script.cache=false 
> -Dgroovy.script.cache.dir=/path/to/cache |
>
> If the feature later proves safe and reliable, it could become enabled 
> by default for normal file-based scripts.
>
>
>     Cache Key
>
> The cache key must include enough information to avoid reusing stale 
> or incompatible bytecode.
>
> At minimum, the key should include:
>
>  *
>
>     absolute or canonical script path
>
>  *
>
>     script source hash
>
>  *
>
>     Groovy version
>
>  *
>
>     Java version or class file target version
>
>  *
>
>     effective classpath
>
>  *
>
>     compiler configuration
>
>  *
>
>     invokedynamic setting
>
>  *
>
>     preview/incubating compiler flags where relevant
>
>  *
>
>     script base class
>
>  *
>
>     active AST transformations
>
>  *
>
>     relevant system properties that affect compilation
>
> For scripts using |@Grab|, the resolved dependency coordinates and 
> artifact versions should be included after dependency resolution.
>
> A conservative implementation may choose to skip caching when the 
> compilation environment cannot be reliably fingerprinted.
>
>
>     Cache Contents
>
> The cache should store:
>
>  *
>
>     generated |.class| files
>
>  *
>
>     metadata describing the compilation environment
>
>  *
>
>     cache format version
>
>  *
>
>     source hash
>
>  *
>
>     Groovy version
>
>  *
>
>     Java/classfile target
>
>  *
>
>     classpath fingerprint
>
>  *
>
>     timestamp of creation
>
>  *
>
>     optional diagnostic information
>
> The cache format should be treated as internal and may change between 
> Groovy versions.
>
>
>     Invalidation
>
> A cached script must be invalidated when any relevant compilation 
> input changes.
>
> Examples:
>
>  *
>
>     script source changed
>
>  *
>
>     Groovy version changed
>
>  *
>
>     Java target changed
>
>  *
>
>     classpath changed
>
>  *
>
>     AST transform implementation changed
>
>  *
>
>     compiler configuration changed
>
>  *
>
>     |@Grab| dependencies changed
>
>  *
>
>     cache format changed
>
> If validation fails or metadata is unreadable, the launcher should 
> silently fall back to normal compilation unless diagnostics are enabled.
>
>
>     Diagnostics
>
> The launcher should provide optional diagnostics.
>
> Examples:
>
> |groovy --script-cache-info myscript.groovy groovy 
> --script-cache-verbose myscript.groovy |
>
> Possible output:
>
> |Groovy script cache: miss Reason: source hash changed |
>
> or:
>
> |Groovy script cache: hit Cache entry: ~/.cache/groovy/script-cache/... |
>
> Diagnostics should be disabled by default to preserve normal script 
> output.
>
>
>     Security Considerations
>
> The cache stores executable bytecode. Therefore:
>
> 1.
>
>     Cache entries should be private to the current user by default.
>
> 2.
>
>     The cache directory should not be world-writable.
>
> 3.
>
>     The launcher should avoid loading cache entries with unsafe
>     permissions.
>
> 4.
>
>     Cache keys should prevent cross-user or cross-project collisions.
>
> 5.
>
>     The cache should not weaken existing script security assumptions.
>
> On systems where permissions cannot be verified reliably, the launcher 
> may disable caching or use a more conservative mode.
>
>
>     Concurrency
>
> Multiple processes may execute the same script concurrently.
>
> The implementation should use atomic writes, temporary files, and safe 
> renames to avoid corrupted cache entries.
>
> If a cache entry is being written by another process, the launcher may 
> either wait briefly, ignore the incomplete entry, or compile normally.
>
>
>     Interaction with Existing Groovy Facilities
>
>
>       GroovyClassLoader
>
> The existing in-memory class cache remains useful within a single JVM 
> process. The proposed persistent cache complements it by surviving 
> across process invocations.
>
>
>       groovyc
>
> This proposal does not replace |groovyc|. Users who want explicit 
> ahead-of-time compilation can continue using |groovyc|.
>
> The script cache is intended for the common case where users execute 
> source scripts directly with the |groovy| command.
>
>
>       @Grab
>
> Scripts using |@Grab| may be cached only if the resolved dependency 
> set can be included in the cache key.
>
> A first implementation may conservatively disable persistent script 
> caching for scripts using |@Grab|.
>
>
>       AST Transformations
>
> AST transformations affect generated bytecode and must be part of the 
> compilation fingerprint. If this cannot be done reliably, caching 
> should be disabled for affected scripts.
>
>
>     Possible Implementation Approach
>
> One possible implementation is:
>
> 1.
>
>     Extend the |groovy| launcher with a cache-aware script runner.
>
> 2.
>
>     Before compilation, compute a |ScriptCacheKey|.
>
> 3.
>
>     Look for a matching cache entry.
>
> 4.
>
>     If present, load generated classes using an appropriate class loader.
>
> 5.
>
>     If absent, compile the script as today.
>
> 6.
>
>     Capture generated bytecode.
>
> 7.
>
>     Persist generated bytecode and metadata atomically.
>
> 8.
>
>     On future runs, validate metadata before loading.
>
> The implementation should be internal and not expose cache internals 
> as stable public API in the first version.
>
>
>     Testing
>
> Tests should cover:
>
>  *
>
>     cache miss on first execution
>
>  *
>
>     cache hit on second execution
>
>  *
>
>     invalidation when script source changes
>
>  *
>
>     invalidation when classpath changes
>
>  *
>
>     invalidation when Groovy version or cache format changes
>
>  *
>
>     disabling the cache
>
>  *
>
>     clearing the cache
>
>  *
>
>     concurrent execution
>
>  *
>
>     scripts with imports
>
>  *
>
>     scripts with local classes
>
>  *
>
>     scripts using AST transformations
>
>  *
>
>     scripts using different compiler configurations
>
>  *
>
>     failure fallback to normal compilation
>
>  *
>
>     cache directory permission checks where supported
>
> Performance tests should measure:
>
>  *
>
>     trivial script
>
>  *
>
>     script with imports
>
>  *
>
>     script with AST transforms
>
>  *
>
>     script with larger source file
>
>  *
>
>     script with dependency-heavy classpath
>
>  *
>
>     script using |@CompileStatic|
>
>  *
>
>     script using dynamic Groovy features
>
>
>     Backward Compatibility
>
> This proposal should be backward compatible.
>
> If disabled, behavior is unchanged.
>
> If enabled, the observable behavior of a script should be equivalent 
> to normal source compilation. If the cache cannot guarantee this, it 
> should not be used.
>
>
>     Risks
>
> The main risks are:
>
> 1.
>
>     Incorrect cache invalidation.
>
> 2.
>
>     Security issues from loading cached bytecode.
>
> 3.
>
>     Increased launcher complexity.
>
> 4.
>
>     Hard-to-debug behavior if cached bytecode differs from source
>     compilation.
>
> 5.
>
>     Limited benefit for scripts where runtime dominates startup.
>
> These risks can be mitigated by making the feature initially opt-in, 
> using conservative invalidation, providing diagnostics, and falling 
> back to normal compilation whenever uncertainty exists.
>
>
>     Alternatives Considered
>
>
>       Use groovyc
>
> Users can already precompile scripts with |groovyc|. However, this 
> changes the workflow and removes the convenience of directly running 
> |.groovy| source files.
>
>
>       Rely Only on JVM Startup Improvements
>
> JVM-level startup improvements help Groovy, but they do not remove 
> Groovy-specific parsing, AST transformation, and bytecode generation 
> costs.
>
>
>       Keep Only In-Memory Caching
>
> Groovy already benefits from in-memory class caching in long-lived 
> processes. This does not help repeated short-lived invocations of the 
> |groovy| command.
>
>
>       External Wrapper Tool
>
> An external script runner could implement persistent caching, but 
> launcher-level support would be more discoverable, portable, and 
> consistent.
>
>
>     Future Work
>
> Future extensions could include:
>
>  *
>
>     enabling the cache by default
>
>  *
>
>     cache statistics
>
>  *
>
>     integration with CDS or Leyden-style JVM caches
>
>  *
>
>     shared cache support for trusted environments
>
>  *
>
>     Gradle/Maven integration
>
>  *
>
>     reusable public APIs for embedders
>
>  *
>
>     support for caching generated stubs where applicable
>
>  *
>
>     smarter dependency fingerprinting for |@Grab|
>
>
>     Conclusion
>
> A persistent script compilation cache would address a long-standing 
> pain point for Groovy as a scripting language: repeated startup cost 
> for short-lived scripts.
>
> By caching generated class files across invocations, the |groovy| 
> launcher could avoid unnecessary repeated parsing, AST transformation, 
> and bytecode generation when scripts are unchanged.
>
> Implemented conservatively, this feature would preserve Groovy 
> semantics while making Groovy scripts feel significantly faster in 
> day-to-day command-line use.
>
>