[GIT-PULLS] [php-src] PR #23210: ext/pdo_pgsql: Add `Pdo\Pgsql::ATTR_CHUNK_SIZE` for chunked result fetching

[email protected] (KentarouTakeda)
Newsgroups php.git-pulls
Message-ID <QYKKfZzyDTWDoS3C01roFVKeBtGu2OjbPvHdMvuM4wc@main.internal.php.net>
Pull Request: https://github.com/php/php-src/pull/23210
Author: KentarouTakeda

Note: this PR currently also contains the bug fix commits from #23065. The feature is independent of them, but the existing lazy fetch defects break this feature's tests *too*. Once #23065 is merged, rebasing will leave only the last commit here.

## Summary

Adds chunked fetching through `Pdo\Pgsql::ATTR_CHUNK_SIZE`. It generalizes the lazy fetch from #15287, and the chunk size lets you choose the balance between memory and speed. ext/pgsql got this as `pg_set_chunked_rows_size` in #14571, but there was no way to use it from ext/pdo_pgsql.

## Benchmark

<details>

<summary>Benchmark code</summary>

```php
<?php
$pdo = new Pdo\Pgsql('pgsql:');

$pdo->exec(<<<SQL
    drop table if exists bench;

    create temp table bench as select
        g as id,
        md5(g::text) c1,
        'user_'||g as c2,
        (g%997)::int as c3,
        (g*1.5)::numeric(12,2) as c4,
        timestamp '2020-01-01'+(g||' seconds')::interval as c5
    from
        generate_series(1,500000) g;

    analyze bench;
SQL);

$out = fopen('/dev/null', 'w');

$modes = [
    'buffered' => [],
    'ATTR_PREFETCH => 0' => [PDO::ATTR_PREFETCH => 0],
    'ATTR_CHUNK_SIZE => 1' => [Pdo\Pgsql::ATTR_CHUNK_SIZE => 1],
    'ATTR_CHUNK_SIZE => 10' => [Pdo\Pgsql::ATTR_CHUNK_SIZE => 10],
    'ATTR_CHUNK_SIZE => 100' => [Pdo\Pgsql::ATTR_CHUNK_SIZE => 100],
    'ATTR_CHUNK_SIZE => 1000' => [Pdo\Pgsql::ATTR_CHUNK_SIZE => 1000],
];

$held = $rows = $ms = [];

for ($round = -1; $round < 9; $round++) {
    foreach ($modes as $name => $options) {
        $t0 = hrtime(true);

        $statement = $pdo->prepare('select * from bench', $options);
        $statement->execute();

        $held[$name] = $statement->getAttribute(Pdo\Pgsql::ATTR_RESULT_MEMORY_SIZE);
        $rows[$name] = $statement->rowCount();

        while ($row = $statement->fetch(PDO::FETCH_ASSOC)) {
            fputcsv($out, $row, escape: '');
        }

        unset($statement);

        // round -1 warms up
        if ($round >= 0) {
            $ms[$name][] = (hrtime(true) - $t0) / 1e6;
        }
    }
}

fputcsv(STDOUT, ['mode', 'median_ms', 'bytes_held', 'rows_per_fetch'], escape: '');

foreach ($ms as $name => $times) {
    sort($times);
    fputcsv(STDOUT, [$name, round($times[4]), $held[$name], $rows[$name]], escape: '');
}
```

</details>

Run on a build without `--enable-debug` (PostgreSQL 18.4 / libpq 18.4):

| mode | median_ms | bytes_held | rows_per_fetch |
| --- | ---: | ---: | ---: |
| `buffered` | 527 | 97,231,064 | 500,000 |
| `ATTR_PREFETCH => 0` | 522 | 3,288 | 1 |
| `ATTR_CHUNK_SIZE => 1` | 518 | 3,288 | 1 |
| `ATTR_CHUNK_SIZE => 10` | 484 | 3,288 | 10 |
| `ATTR_CHUNK_SIZE => 100` | 442 | 19,672 | 100 |
| `ATTR_CHUNK_SIZE => 1000` | 429 | 188,632 | 1,000 |

## Usage

Set on the connection:

```php
$pdo->setAttribute(Pdo\Pgsql::ATTR_CHUNK_SIZE, 1000);
```

Set on the statement, overriding the connection level:

```php
$pdo->prepare(
    'select * from bench',
    [Pdo\Pgsql::ATTR_CHUNK_SIZE => 1000]
);
```

## Design note

The part in bold is where I am not confident that the behavior is right. I would appreciate feedback.

- `ATTR_CHUNK_SIZE` overrides `ATTR_PREFETCH`.
  - When `ATTR_PREFETCH` is left unset and only `ATTR_CHUNK_SIZE` is given, this is treated as a request for chunked fetching.
  - **It also overrides an explicit `ATTR_PREFETCH => 0`.** The behavior changes from lazy to chunk. `ATTR_CHUNK_SIZE => 1` behaves the same as lazy, and once a chunk size is given, I think honoring it is the natural behavior.
- `ATTR_CHUNK_SIZE` and `ATTR_CURSOR => CURSOR_SCROLL` are mutually exclusive.
  - A statement-level option takes precedence over the connection level, so a statement-level `CURSOR_SCROLL` disables a connection-level `ATTR_CHUNK_SIZE`.
  - When both are given at the statement level, there is no basis for deciding precedence, so a `ValueError` is thrown.
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.