[GIT-PULLS] [php-src] PR #23461: Fix bug #80056: SPL directory iterators lose entries on 9p filesystems

[email protected] (denkfabrik-li)
Newsgroups php.git-pulls
Message-ID <[email protected]>
Pull Request: https://github.com/php/php-src/pull/23461
Author: denkfabrik-li

### Summary

`FilesystemIterator`, `RecursiveDirectoryIterator` and `DirectoryIterator` silently
lose directory entries on filesystems that cannot rewind a directory handle after a
partial read — most notably 9p mounts, which is what WSL2 and Docker Desktop on
Windows use to expose the Windows drive to Linux. An entire libc `getdents` buffer
vanishes from the iteration: ~21 entries with musl (2 KiB buffer), ~343 with glibc
(32 KiB buffer). `scandir()`, `glob()` and a plain `readdir()` loop over the same
directory return the complete listing.

The cause is PHP-specific behaviour, not just the kernel bug: the iterator
constructors eagerly read the first directory entry, and the implicit `rewind()` at
the start of the first `foreach` then calls `rewinddir()` on a stream whose position
is already past that first entry. On a healthy filesystem this seek is redundant (it
re-reads the entry the constructor already had); on 9p the seek is silently ignored
server-side while the C library discards its read buffer — everything that was in
that buffer is skipped.

This PR makes the rewind a no-op while the stream is still positioned at its first
entry, so the standard construct-then-`foreach` pattern never seeks at all.

### History

- https://bugs.php.net/bug.php?id=80056 (Sep 2020, RecursiveDirectoryIterator misses
  files under Docker/WSL2) — closed "Not a bug": *"this is unlikely to be a PHP issue"*.
- https://bugs.php.net/bug.php?id=80227 (Oct 2020, same on Alpine/Docker/WSL2) —
  closed "Not a bug": *"it's a WSL bug"*, pointing to microsoft/WSL#5074.
- https://github.com/microsoft/WSL/issues/5074 — closed by the bot without a fix;
  still reproducible today (measurements below, Windows 11, current WSL2/Docker
  Desktop).

Both reports were closed on the grounds that the underlying seek failure lives below
PHP — which is true. But no code change was ever proposed back then, and the seek
itself is avoidable: it is issued by PHP at a moment where it has nothing to do.
Every other PHP directory API (`scandir()`, `glob()`, `opendir()`/`readdir()`)
works on these mounts precisely because none of them seeks a partially-read handle.

The failure mode is nasty in practice because it is silent, size-dependent (only
directories with more entries than one libc buffer are affected) and
environment-dependent (works on ext4, breaks on the very same code under Docker
Desktop on Windows). Real-world sightings include Composer autoload scans, Laravel
migration discovery and PHPStan/Larastan file collection:
laravel/framework#61336, larastan/larastan#2538, projectsend/projectsend#1680.

### The mechanism

```text
new FilesystemIterator($dir)          # constructor opens the dir
  -> readdir()                        #   ...and eagerly reads entry #1
                                      #   libc fills its getdents buffer
foreach ($it as $f)                   # foreach starts with rewind()
  -> rewinddir()                      #   seek to offset 0
                                      #   9p: seek silently ignored,
                                      #   libc buffer discarded anyway
  -> readdir(), readdir(), ...        #   continues AFTER the discarded buffer
```

With musl's 2 KiB buffer the first ~21 entries disappear; with glibc's 32 KiB
buffer the first ~343. Directories at or below one buffer appear complete, which is
why small test cases pass and production directories fail.

### The fix

Track in `spl_filesystem_object.u.dir` whether the stream is still positioned at its
first (non-skipped) entry (`at_initial_entry`). All four rewind paths
(`DirectoryIterator::rewind()`, `FilesystemIterator::rewind()`, and the two internal
`zend_object_iterator` rewind handlers) are consolidated into one helper,
`spl_filesystem_dir_rewind()`, which skips the `rewinddir()` + re-read when nothing
has been consumed yet. Any actual read (`next()`, internal `move_forward`, the
clone catch-up loop, `seek()`) clears the flag, so a rewind after real consumption
still performs a real `rewinddir()` exactly as before.

### Behaviour / BC analysis

Unchanged:
- The constructor still opens the directory eagerly — an invalid path still throws
  `UnexpectedValueException` from the constructor.
- The constructor still pre-reads the first entry: `$it->current()`,
  `getFilename()`, `key()`, `hasChildren()` etc. right after construction behave
  exactly as before (and now also work on 9p, since the pre-read is sequential).
- `rewind()` after any entry has been consumed performs a real `rewinddir()` +
  re-read, as before (re-iterating an iterator, `seek()` backwards, clone).
- Result sets, ordering, keys, flags handling (`SKIP_DOTS`, …) on healthy
  filesystems: byte-for-byte identical output; the full test suite passes.

Observable differences (deliberate):
- While the stream is still on its first entry, `rewind()` no longer issues
  `rewinddir()` + `readdir()`. For userland stream wrappers this means
  `dir_rewinddir()` is no longer invoked at the start of the first iteration (one
  callback less; a wrapper that relied on being rewound before the first read was
  already broken, since the constructor pre-read has always happened before any
  rewind). One redundant syscall pair per iteration start is saved everywhere.
- POSIX allows `rewinddir()` to pick up directory modifications made after
  `opendir()`. A file created between construction and the first `foreach` was
  previously *sometimes* visible (filesystem-dependent, POSIX explicitly leaves it
  unspecified); with the no-op rewind the iteration keeps the view the constructor
  started with. Code relying on that was already unreliable across filesystems.

Known limitation (unchanged, inherent to broken seeks): re-iterating the *same*
iterator object (second `foreach`, explicit `rewind()` after `next()`) still
requires a real `rewinddir()` and therefore still misbehaves on 9p — same as a
userland `rewinddir()` call. This patch fixes the overwhelmingly common
construct-then-iterate-once pattern, which is what Composer/Laravel/PHPStan & co.
use.

### Tests

- `ext/spl/tests/bug80056.phpt` — simulates the broken filesystem with a userland
  stream wrapper whose `dir_rewinddir()` pretends to succeed without resetting the
  position (exactly the observable 9p behaviour). Fails on current master (each
  iterator loses its first entry), passes with the fix. Covers DirectoryIterator,
  FilesystemIterator, RecursiveDirectoryIterator + RecursiveIteratorIterator, and
  the "entry accessed before iteration" pattern.
- `ext/spl/tests/spl_dir_iterator_rewind_noop.phpt` — pins the new seek semantics
  with a counting wrapper: no `dir_rewinddir()` on first iteration, exactly one on
  re-iteration, exactly one for `rewind()` after `next()`, none for repeated
  `rewind()` without reads.
- Existing SPL suite: green (see verification below).

### Verification

Two-stage, with unpatched and patched CLI binaries built from the *same* master
checkout (`ext/spl/spl_directory.*` diff applied incrementally in the same
container build):

**(a) Test suite (ext4 inside the build containers):**

Full `ext/spl` suite with the patched CLI — identical results on
Debian bookworm (glibc) and Alpine 3.22 (musl):

```text
Number of tests :   810               799
Tests skipped   :    11 (  1.4%)
Tests failed    :     0 (  0.0%)
Expected fail   :     1 (  0.1%)   (pre-existing XFAIL, unrelated)
Tests passed    :   798 ( 98.5%)
```

A second build with `--enable-phar --enable-zend-test` additionally runs the
SPL-over-phar and stack-limit tests (gh14687, gh17225, gh15911, gh15672 — all
pass; 802 SPL tests passed total) plus the full `ext/phar` suite: 382 runnable
tests, 0 failures. Relevant because `Phar` extends `RecursiveDirectoryIterator`
and embeds `spl_filesystem_object`; its in-memory directory stream keeps a
working seek, so behaviour is unchanged there.

Both new tests FAIL on unpatched master, each iterator losing its first entry
(`array(4)` instead of `array(5)`, `a.txt` missing) — i.e. the regression tests
demonstrably catch the bug.

**(b) Live 9p mount (Docker Desktop on Windows 11, WSL2 backend; a Windows
directory with 500 `.php` files — plus 1 in a subdirectory for the recursive
case — mounted into the container; `stat -f -c %T` reports `v9fs`):**

| API                          | musl unpatched | musl patched | glibc unpatched | glibc patched |
|------------------------------|---------------:|-------------:|----------------:|--------------:|
| `scandir()`                  | 500            | 500          | 500             | 500           |
| `glob()`                     | 500            | 500          | 500             | 500           |
| `readdir()` loop             | 500            | 500          | 500             | 500           |
| `FilesystemIterator`         | **479**        | 500          | **157**         | 500           |
| `RecursiveDirectoryIterator` | **480**        | 501          | **158**         | 501           |
| `DirectoryIterator`          | **479**        | 500          | **157**         | 500           |

The losses match the libc buffer sizes exactly: musl drops its first 2 KiB
`getdents` buffer (21 entries), glibc its first 32 KiB buffer (343 entries).

One-liner reproducer for anyone with Docker Desktop on Windows (stock image,
directory with a few hundred files):

```text
> docker run --rm -v C:\some\big\dir:/mnt/d php:8.4-cli php -r \
    "echo count(scandir('/mnt/d'))-2, ' vs ', iterator_count(new FilesystemIterator('/mnt/d'));"
501 vs 158        # PHP 8.4.24, verified 2026-08-25
```
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.