[GIT-PULLS] [php-src] PR #22917: Improves test suite performance by 8.5-25x [experiment!]
[email protected] (NickSdot) Wed, 29 Jul 2026 11:20:31 +0000
| Newsgroups | php.git-pulls |
|---|---|
| Message-ID | <hMMRhQ2J9BONW3A6RA9UaGiCzkFZp2WMUMlzPjToHuA@main.internal.php.net> |
Pull Request: https://github.com/php/php-src/pull/22917 Author: NickSdot While working on https://github.com/php/php-src/pull/22799 I was bothered by the test suite being so slow. CI needs ~45 minutes from push to finish; running tests locally also takes quite a while. I was wondering why that is and whether it must be like that. Hence, I decided to run an experiment to find out if we could make the developer experience notably better. > [!NOTE] > **This experiment was fully executed by Codex.** I was throwing some money on OpenAI to let GPT 5.6 Sol pursue the `/goal` to make the test suite "massively" faster. This PR is the result of Codex trying exactly that -- for 14 hours straight. Consequently, I will not pretend that I understand the implications of every proposal -- and I also did not review everything in detail yet. Though, I **did** spend plenty of time to confirm the claims and reworking things. > > The result are interesting enough to open this **clearly flagged as LLM-executed experiment**, hoping that it can serve as a discussion starter to find out whether anything can be picked up. Would love to get feedback from people with more experience than me, to pin down which of these are legit ideas and which are dead-ends right away. ## Results Values in seconds; measured on macOS. The full suite improvements are already awesome on their own. But targeted runs also show up to 25x faster results (`-j10` vs. `-j10`), many **at least** 2x faster, and very few slightly slower. ### Full Suite | flag | master | here | gain | | |--------|---------:|--------:|-------------------:|--------------------------------------| | (none) | 2,223.224 | 69.833 | -2,153.39 (31.84x) | auto-detected concurrency by default | | -j1 | 2,224.902 | 443.214 | -1,781.69 (5.02x) | optionally opting out of concurrency | | -j10 | 594.445 | 69.768 | -524.68 (8.52x) | | ### Partial Runs | target | flag | master | here | gain | |---|---|---:|---:|---:| | ext/bcmath | (none) | 10.912 | 1.033 | -9.88s (10.56x) | | | -j1 | 10.912 | 2.038 | -8.87s (5.35x) | | | -j10 | 2.477 | 0.953 | -1.52s (2.60x) | | ext/bz2 | (none) | 2.537 | 0.596 | -1.94s (4.26x) | | | -j1 | 2.537 | 0.507 | -2.03s (5.00x) | | | -j10 | 0.925 | 0.663 | -0.26s (1.40x) | | ext/calendar | (none) | 3.896 | 0.757 | -3.14s (5.15x) | | | -j1 | 3.896 | 1.220 | -2.68s (3.19x) | | | -j10 | 1.048 | 0.702 | -0.35s (1.49x) | | ext/com_dotnet | (none) | 0.119 | 0.354 | +0.23s (0.34x) | | | -j1 | 0.119 | 0.179 | +0.06s (0.66x) | | | -j10 | 0.299 | 0.353 | +0.05s (0.85x) | | ext/ctype | (none) | 3.176 | 0.625 | -2.55s (5.08x) | | | -j1 | 3.176 | 0.676 | -2.50s (4.70x) | | | -j10 | 1.031 | 0.636 | -0.39s (1.62x) | | ext/curl | (none) | 34.101 | 6.639 | -27.46s (5.14x) | | | -j1 | 34.101 | 23.905 | -10.20s (1.43x) | | | -j10 | 35.727 | 6.655 | -29.07s (5.37x) | | ext/date | (none) | 44.032 | 2.711 | -41.32s (16.24x) | | | -j1 | 44.032 | 10.187 | -33.84s (4.32x) | | | -j10 | 9.308 | 2.487 | -6.82s (3.74x) | | ext/dba | (none) | 5.617 | 0.791 | -4.83s (7.10x) | | | -j1 | 5.617 | 0.910 | -4.71s (6.17x) | | | -j10 | 3.134 | 0.853 | -2.28s (3.67x) | | ext/dom | (none) | 54.535 | 1.915 | -52.62s (28.48x) | | | -j1 | 54.535 | 6.628 | -47.91s (8.23x) | | | -j10 | 10.087 | 1.826 | -8.26s (5.52x) | | ext/enchant | (none) | 0.122 | 0.304 | +0.18s (0.40x) | | | -j1 | 0.122 | 0.127 | +0.01s (0.96x) | | | -j10 | 0.308 | 0.314 | +0.01s (0.98x) | | ext/exif | (none) | 6.351 | 0.811 | -5.54s (7.83x) | | | -j1 | 6.351 | 1.196 | -5.16s (5.31x) | | | -j10 | 1.358 | 0.916 | -0.44s (1.48x) | | ext/ffi | (none) | 6.710 | 0.850 | -5.86s (7.89x) | | | -j1 | 6.710 | 1.202 | -5.51s (5.58x) | | | -j10 | 1.576 | 1.154 | -0.42s (1.37x) | | ext/fileinfo | (none) | 5.251 | 0.876 | -4.38s (5.99x) | | | -j1 | 5.251 | 1.696 | -3.56s (3.10x) | | | -j10 | 1.551 | 0.997 | -0.55s (1.56x) | | ext/filter | (none) | 7.543 | 0.915 | -6.63s (8.24x) | | | -j1 | 7.543 | 2.415 | -5.13s (3.12x) | | | -j10 | 1.926 | 1.080 | -0.85s (1.78x) | | ext/ftp | (none) | 7.842 | 1.682 | -6.16s (4.66x) | | | -j1 | 7.842 | 3.100 | -4.74s (2.53x) | | | -j10 | 2.288 | 1.557 | -0.73s (1.47x) | | ext/gd | (none) | 25.575 | 1.503 | -24.07s (17.02x) | | | -j1 | 25.575 | 3.430 | -22.14s (7.46x) | | | -j10 | 6.162 | 1.493 | -4.67s (4.13x) | | ext/gettext | (none) | 1.783 | 0.579 | -1.20s (3.08x) | | | -j1 | 1.783 | 0.511 | -1.27s (3.49x) | | | -j10 | 0.740 | 0.636 | -0.10s (1.16x) | | ext/gmp | (none) | 8.072 | 1.131 | -6.94s (7.14x) | | | -j1 | 8.072 | 2.373 | -5.70s (3.40x) | | | -j10 | 2.075 | 1.134 | -0.94s (1.83x) | | ext/hash | (none) | 4.881 | 0.611 | -4.27s (7.99x) | | | -j1 | 4.881 | 0.918 | -3.96s (5.32x) | | | -j10 | 1.100 | 0.493 | -0.61s (2.23x) | | ext/iconv | (none) | 5.313 | 0.953 | -4.36s (5.58x) | | | -j1 | 5.313 | 2.612 | -2.70s (2.03x) | | | -j10 | 1.420 | 1.023 | -0.40s (1.39x) | | ext/intl | (none) | 32.200 | 4.522 | -27.68s (7.12x) | | | -j1 | 32.200 | 7.158 | -25.04s (4.50x) | | | -j10 | 6.659 | 1.910 | -4.75s (3.49x) | | ext/json | (none) | 6.589 | 0.609 | -5.98s (10.82x) | | | -j1 | 6.589 | 0.966 | -5.62s (6.82x) | | | -j10 | 1.404 | 0.659 | -0.74s (2.13x) | | ext/ldap | (none) | 6.641 | 2.920 | -3.72s (2.27x) | | | -j1 | 6.641 | 2.937 | -3.70s (2.26x) | | | -j10 | 7.438 | 2.999 | -4.44s (2.48x) | | ext/libxml | (none) | 2.498 | 0.660 | -1.84s (3.78x) | | | -j1 | 2.498 | 0.669 | -1.83s (3.73x) | | | -j10 | 0.941 | 0.674 | -0.27s (1.40x) | | ext/mbstring | (none) | 30.374 | 2.481 | -27.89s (12.24x) | | | -j1 | 30.374 | 10.504 | -19.87s (2.89x) | | | -j10 | 7.703 | 2.473 | -5.23s (3.11x) | | ext/mysqli | (none) | 10.021 | 3.824 | -6.20s (2.62x) | | | -j1 | 10.021 | 3.684 | -6.34s (2.72x) | | | -j10 | 9.859 | 3.834 | -6.03s (2.57x) | | ext/odbc | (none) | 0.904 | 0.452 | -0.45s (2.00x) | | | -j1 | 0.904 | 0.352 | -0.55s (2.57x) | | | -j10 | 1.050 | 0.585 | -0.47s (1.79x) | | ext/opcache | (none) | 67.487 | 8.283 | -59.20s (8.15x) | | | -j1 | 67.487 | 54.224 | -13.26s (1.24x) | | | -j10 | 13.405 | 8.552 | -4.85s (1.57x) | | ext/openssl | (none) | 45.311 | 5.035 | -40.28s (9.00x) | | | -j1 | 45.311 | 28.753 | -16.56s (1.58x) | | | -j10 | 49.155 | 5.106 | -44.05s (9.63x) | | ext/pcntl | (none) | 6.540 | 2.506 | -4.03s (2.61x) | | | -j1 | 6.540 | 2.888 | -3.65s (2.26x) | | | -j10 | 3.305 | 2.554 | -0.75s (1.29x) | | ext/pcre | (none) | 12.600 | 1.716 | -10.88s (7.34x) | | | -j1 | 12.600 | 3.450 | -9.15s (3.65x) | | | -j10 | 4.478 | 1.811 | -2.67s (2.47x) | | ext/pdo | (none) | 1.933 | 0.628 | -1.31s (3.08x) | | | -j1 | 1.933 | 0.549 | -1.38s (3.52x) | | | -j10 | 0.971 | 0.679 | -0.29s (1.43x) | | ext/pdo_dblib | (none) | 237.447 | 8.270 | -229.18s (28.71x) | | | -j1 | 237.447 | 11.324 | -226.12s (20.97x) | | | -j10 | 200.863 | 8.234 | -192.63s (24.39x) | | ext/pdo_firebird | (none) | 0.124 | 0.249 | +0.12s (0.50x) | | | -j1 | 0.124 | 0.131 | +0.01s (0.95x) | | | -j10 | 0.289 | 0.269 | -0.02s (1.07x) | | ext/pdo_mysql | (none) | 3.330 | 2.324 | -1.01s (1.43x) | | | -j1 | 3.330 | 2.451 | -0.88s (1.36x) | | | -j10 | 3.807 | 2.287 | -1.52s (1.66x) | | ext/pdo_odbc | (none) | 2.126 | 2.233 | +0.11s (0.95x) | | | -j1 | 2.126 | 2.120 | -0.01s (1.00x) | | | -j10 | 2.467 | 2.160 | -0.31s (1.14x) | | ext/pdo_pgsql | (none) | 3.288 | 2.311 | -0.98s (1.42x) | | | -j1 | 3.288 | 2.564 | -0.72s (1.28x) | | | -j10 | 2.582 | 2.275 | -0.31s (1.13x) | | ext/pdo_sqlite | (none) | 20.236 | 14.875 | -5.36s (1.36x) | | | -j1 | 20.236 | 15.297 | -4.94s (1.32x) | | | -j10 | 16.838 | 14.561 | -2.28s (1.16x) | | ext/pgsql | (none) | 1.654 | 0.751 | -0.90s (2.20x) | | | -j1 | 1.654 | 0.626 | -1.03s (2.64x) | | | -j10 | 2.032 | 2.158 | +0.13s (0.94x) | | ext/phar | (none) | 56.768 | 5.441 | -51.33s (10.43x) | | | -j1 | 56.768 | 24.778 | -31.99s (2.29x) | | | -j10 | 11.601 | 7.420 | -4.18s (1.56x) | | ext/posix | (none) | 4.892 | 0.658 | -4.23s (7.43x) | | | -j1 | 4.892 | 0.796 | -4.10s (6.15x) | | | -j10 | 1.686 | 0.822 | -0.86s (2.05x) | | ext/random | (none) | 4.283 | 0.492 | -3.79s (8.71x) | | | -j1 | 4.283 | 0.774 | -3.51s (5.53x) | | | -j10 | 0.964 | 0.445 | -0.52s (2.17x) | | ext/readline | (none) | 2.292 | 0.676 | -1.62s (3.39x) | | | -j1 | 2.292 | 1.228 | -1.06s (1.87x) | | | -j10 | 1.065 | 0.640 | -0.42s (1.66x) | | ext/reflection | (none) | 31.491 | 1.242 | -30.25s (25.36x) | | | -j1 | 31.491 | 4.245 | -27.25s (7.42x) | | | -j10 | 6.614 | 1.262 | -5.35s (5.24x) | | ext/session | (none) | 17.882 | 2.428 | -15.45s (7.36x) | | | -j1 | 17.882 | 10.166 | -7.72s (1.76x) | | | -j10 | 3.771 | 2.731 | -1.04s (1.38x) | | ext/shmop | (none) | 0.474 | 0.331 | -0.14s (1.43x) | | | -j1 | 0.474 | 0.307 | -0.17s (1.54x) | | | -j10 | 0.336 | 0.341 | +0.01s (0.99x) | | ext/simplexml | (none) | 9.555 | 0.720 | -8.83s (13.27x) | | | -j1 | 9.555 | 1.109 | -8.45s (8.62x) | | | -j10 | 2.105 | 0.795 | -1.31s (2.65x) | | ext/skeleton | (none) | 0.117 | 0.196 | +0.08s (0.60x) | | | -j1 | 0.117 | 0.126 | +0.01s (0.93x) | | | -j10 | 0.194 | 0.193 | -0.00s (1.01x) | | ext/snmp | (none) | 43.267 | 6.695 | -36.57s (6.46x) | | | -j1 | 43.267 | 6.455 | -36.81s (6.70x) | | | -j10 | 43.186 | 6.596 | -36.59s (6.55x) | | ext/soap | (none) | 40.262 | 2.767 | -37.49s (14.55x) | | | -j1 | 40.262 | 10.148 | -30.11s (3.97x) | | | -j10 | 7.781 | 2.916 | -4.87s (2.67x) | | ext/sockets | (none) | 11.027 | 2.600 | -8.43s (4.24x) | | | -j1 | 11.027 | 3.256 | -7.77s (3.39x) | | | -j10 | 3.956 | 2.757 | -1.20s (1.43x) | | ext/sodium | (none) | 12.679 | 1.117 | -11.56s (11.35x) | | | -j1 | 12.679 | 2.257 | -10.42s (5.62x) | | | -j10 | 10.710 | 1.115 | -9.60s (9.61x) | | ext/spl | (none) | 51.527 | 2.119 | -49.41s (24.32x) | | | -j1 | 51.527 | 9.217 | -42.31s (5.59x) | | | -j10 | 9.515 | 2.597 | -6.92s (3.66x) | | ext/sqlite3 | (none) | 6.761 | 0.737 | -6.02s (9.17x) | | | -j1 | 6.761 | 1.203 | -5.56s (5.62x) | | | -j10 | 1.494 | 0.707 | -0.79s (2.11x) | | ext/standard | (none) | 322.044 | 14.536 | -307.51s (22.15x) | | | -j1 | 322.044 | 73.297 | -248.75s (4.39x) | | | -j10 | 56.019 | 14.828 | -41.19s (3.78x) | | ext/sysvmsg | (none) | 0.541 | 0.378 | -0.16s (1.43x) | | | -j1 | 0.541 | 0.211 | -0.33s (2.56x) | | | -j10 | 0.374 | 0.375 | +0.00s (1.00x) | | ext/sysvsem | (none) | 0.300 | 0.304 | +0.00s (0.99x) | | | -j1 | 0.300 | 0.242 | -0.06s (1.24x) | | | -j10 | 0.370 | 0.310 | -0.06s (1.19x) | | ext/sysvshm | (none) | 0.997 | 0.471 | -0.53s (2.12x) | | | -j1 | 0.997 | 0.380 | -0.62s (2.62x) | | | -j10 | 0.638 | 0.582 | -0.06s (1.10x) | | ext/tidy | (none) | 3.212 | 0.586 | -2.63s (5.48x) | | | -j1 | 3.212 | 0.706 | -2.51s (4.55x) | | | -j10 | 1.058 | 0.783 | -0.28s (1.35x) | | ext/tokenizer | (none) | 3.469 | 0.565 | -2.90s (6.14x) | | | -j1 | 3.469 | 0.693 | -2.78s (5.01x) | | | -j10 | 0.999 | 0.552 | -0.45s (1.81x) | | ext/uri | (none) | 22.419 | 0.779 | -21.64s (28.78x) | | | -j1 | 22.419 | 2.107 | -20.31s (10.64x) | | | -j10 | 5.517 | 0.730 | -4.79s (7.56x) | | ext/xml | (none) | 4.333 | 0.764 | -3.57s (5.67x) | | | -j1 | 4.333 | 0.809 | -3.52s (5.36x) | | | -j10 | 1.224 | 0.779 | -0.44s (1.57x) | | ext/xmlreader | (none) | 4.449 | 0.618 | -3.83s (7.20x) | | | -j1 | 4.449 | 0.697 | -3.75s (6.38x) | | | -j10 | 1.379 | 0.827 | -0.55s (1.67x) | | ext/xmlwriter | (none) | 3.234 | 0.618 | -2.62s (5.23x) | | | -j1 | 3.234 | 0.592 | -2.64s (5.46x) | | | -j10 | 0.925 | 0.553 | -0.37s (1.67x) | | ext/xsl | (none) | 0.129 | 0.416 | +0.29s (0.31x) | | | -j1 | 0.129 | 0.135 | +0.01s (0.96x) | | | -j10 | 0.340 | 0.342 | +0.00s (0.99x) | | ext/zend_test | (none) | 10.192 | 1.549 | -8.64s (6.58x) | | | -j1 | 10.192 | 5.229 | -4.96s (1.95x) | | | -j10 | 2.176 | 1.376 | -0.80s (1.58x) | | ext/zip | (none) | 11.918 | 1.040 | -10.88s (11.46x) | | | -j1 | 11.918 | 1.468 | -10.45s (8.12x) | | | -j10 | 2.609 | 0.904 | -1.71s (2.89x) | | ext/zlib | (none) | 11.232 | 1.199 | -10.03s (9.37x) | | | -j1 | 11.232 | 2.992 | -8.24s (3.75x) | | | -j10 | 2.196 | 1.025 | -1.17s (2.14x) | | sapi/cgi | (none) | 5.833 | 2.246 | -3.59s (2.60x) | | | -j1 | 5.833 | 5.047 | -0.79s (1.16x) | | | -j10 | 2.242 | 2.142 | -0.10s (1.05x) | | sapi/cli | (none) | 24.166 | 2.849 | -21.32s (8.48x) | | | -j1 | 24.166 | 16.504 | -7.66s (1.46x) | | | -j10 | 5.508 | 2.902 | -2.61s (1.90x) | | sapi/fpm | (none) | 1.595 | 0.421 | -1.17s (3.79x) | | | -j1 | 1.595 | 0.417 | -1.18s (3.82x) | | | -j10 | 1.491 | 0.410 | -1.08s (3.64x) | | sapi/phpdbg | (none) | n/a | n/a | n/a | | | -j1 | 246.564 | 6.091 | -240.47s (40.48x) | | | -j10 | n/a | n/a | n/a | | tests | (none) | 64.478 | 4.645 | -59.83s (13.88x) | | | -j1 | 64.478 | 20.204 | -44.27s (3.19x) | | | -j10 | 13.876 | 4.398 | -9.48s (3.16x) | | Zend | (none) | 373.757 | 15.621 | -358.14s (23.93x) | | | -j1 | 373.757 | 49.758 | -324.00s (7.51x) | | | -j10 | 75.593 | 13.010 | -62.58s (5.81x) | _The -j1 row compares the master default with the explicit sequential opt-out._ ## Commits | commit | change | type | before | after | saved | speedup | comment | | --- | --- | --- | ---: | ---: | ---: | ---: | --- | | 9b36b9145a0 | Run tests in parallel by default | (D) | 387.450 | 373.760 | 13.690 | 1.04× | Automatically selects up to 10 workers. Explicit `-j1` remains the serial opt-out. Includes a regression proving the cap. | | 01dbbb753be | Select Windows test workers automatically | (W) | — | — | — | — | Non-ASAN Windows CI uses automatic detection; ASAN remains `-j2`. Requires Windows CI measurement. | | 90c943c22ed | Parallelize `new_oom` subprocesses | (D) | 363.220 | 283.265 | 79.955 | 1.28× | Uses bounded subprocess concurrency, ignores non-instantiable classes, and rejects subprocesses killed by signals. | | 6e0047855c0 | Reduce parallel batch size | (D) | 283.265 | 157.091 | 126.174 | 1.80× | Reduces the long tail caused by a few slow batches. | | f0f42a01620 | Reuse CLI startup with an isolated fork server | (F) | 0.679 | 0.251 | 0.428 | 2.71× | Readonly class/property subset, `-j10`, median of three runs, disabled versus enabled. Includes signal-safe child tracking, environment-aware caching and timeout/failure fallback coverage. | | dda593f40d4 | Cache and bound failed PDO connection probes | (C) | 200.863 | 8.234 | 192.629 | 24.39× | `ext/pdo_dblib`, `-j10`, final branch versus master. A one-second timeout applies only to the default availability probe. Configured connection attributes are preserved. | | 7d7ddc611b4 | Handle redirected tests in parallel runs | (V) | fatal / `3/2` | pass / `3/3` | — | — | Redirected child tests stay serial inside their already-parallel wrapper, while the parent receives the expanded test count. | | 2546b0748ce | Add bounded directory concurrency | (F) | 17.804 | 3.211 | 14.593 | 5.54× | Adds independent `MAX_CONCURRENCY` and `SCOPED_CONFLICTS`. Ordinary `CONFLICTS` remains strict. Includes regressions for both semantics. | | 1d5cdeb84f7 | Cache failed SNMP-agent probes | (C) | 43.186 | 6.596 | 36.590 | 6.55× | `ext/snmp`, `-j10`, final branch versus master. Only failed probes are cached. | | 7c3349b8a51 | Avoid external DNS in addrinfo test | (F) | 0.560 | 0.560 | 0.000 | 1.00× | No median gain. Retained as an offline/reliability improvement because the hostname is incidental to the error-code assertions. | | bba23c51046 | Run scandir overflow test on Windows only | (F) | 9.620 | 0.450 | 9.170 | 21.38× | Avoids creating 70,000 files on platforms unaffected by the Win32 `DIR_W32.offset` regression. | | c95d9c38a60 | Remove redundant filesystem-test delays | (F) | 15.540 | 3.410 | 12.130 | 4.56× | Replaces timing assumptions with explicit timestamps or removes waits after the required state is already established. | | 318646f54b0 | Reduce password-test work factors | (F) | 14.990 | 1.790 | 13.200 | 8.37× | Uses valid lower-cost bcrypt, Argon2, sodium, and extended-DES parameters while retaining the tested behavior. | | d3ec35d181f | Optimize Phar regression fixtures | (F) | 3.040 | 1.112 | 1.928 | 2.73× | Preserves bug #13727’s source-stream-close regression and uses a controlled iterator fixture. Median of three runs. | | 112f3ca117e | Avoid opcache revalidation delays | (F) | 4.650 | 0.630 | 4.020 | 7.38× | Advances the fixture timestamp explicitly instead of waiting for wall-clock time. | | 8779644b4b4 | Synchronize `proc_open` tests without fixed delays | (F) | 9.300 | 1.210 | 8.090 | 7.69× | Replaces sleeps with polling, socket handshakes, and observable process state. | | af1d402109d | Use a bounded passthru binary fixture | (F) | 0.880 | 0.700 | 0.180 | 1.26× | Preserves exact binary round-trip coverage with a deterministic 256 KiB all-byte pattern. | | 2bf072b1b00 | Parallelize OpenSSL tests safely | (C) | 49.155 | 5.106 | 44.049 | 9.63× | `ext/openssl`, `-j10`, final branch versus master. Uses isolated fixtures, dynamic ports, scoped `server` conflicts and a cap of 10. | | cf72013641f | Bypass the shell for test subprocesses | (F) | 12.120 | 10.690 | 1.430 | 1.13× | Structured `proc_open()` microbenchmark. Preserves environment isolation and falls back where shell semantics are required. | | b37c4620c4e | Run Curl tests with bounded concurrency | (C) | 35.727 | 6.655 | 29.072 | 5.37× | `ext/curl`, `-j10`, final branch versus master. Isolates duplicate fixtures and caps the directory at four workers. | | 559f5290780 | Isolate `open_basedir` fixtures | (F) | 1.372 | 0.667 | 0.705 | 2.06× | Uses independent fixture paths, scoped `open_basedir` conflicts, a cap of 10 and symlink-safe cleanup. | | 7246f618223 | Skip unavailable Unix datagram socket tests | (E) | 121.060 | 0.620 | 120.440 | 195.26× | Restricted environment only. Recognized unsupported/permission errors skip quickly; unexpected bind failures BORK. Capable hosts execute the original test body. | Legend: `(D)` development-step measurement; `(C)` cumulative module measurement; `(F)` focused local measurement; `(E)` environment-specific measurement; `(V)` correctness verification; `(W)` Windows CI pending. ## Misc - At first, I found it weird that it would propose adding test-only code to the SAPI. But then I learned that the already existing, internal `--repeat` flag does exactly that. - I am pretty sure that there are reasons for Windows in CI only running on `-j2`, and parallel not being the default. I, however, could not find out what these reasons are; hence, I kept changes Codex proposed. - not sure whether we fancy adding new phpt sections like `MAX_CONCURRENCY`, but it shows gains. - My repeated benchmark runs exposed existing bugs, fixed here in 9905f197bbe118843b1651cb42ab84bd5b7587b2 and 6be6a43ab1c93ff5c29b26eb07128e159ee3678f (also PR'ed separately in #22911), and f68eeaae2d32b847f61235f0626eacf18388a381 and a029596a5ca4251136ecc32d9777108a269edc3d (also PR'ed separately in #22912). - buggy concurrent execution CLI output (eg: TEST **22386**/22094 [2/10 concurrent test workers running]) was fixed. - Codex did *not* consider this done yet; so if there is general interest I will let it finish the job. - I know, another huge PR by me. Keen to split that up as we go.