[php-src] Issue #21936: pdo_sqlite on macOS: SELECT … ORDER BY raises SQLSTATE HY000 error 25 "column index out of range" (Apple ENABLE_API_ARMOR + sqlite3_data_count() guard bug in pdo_sqlite_stmt_get_col)

[email protected] (belisoful)
Newsgroups php.bugs
Message-ID <[email protected]>
Issue: https://github.com/php/php-src/issues/21936
Author: belisoful

### Description


## Description

On macOS, any `SELECT … ORDER BY` query executed through `pdo_sqlite` against Apple's system SQLite library (`/usr/lib/libsqlite3.dylib`) raises a `PDOException` with SQLSTATE `HY000`, error code 25, message `"column index out of range"`. The query is syntactically and semantically valid; the result set is never returned.

The bug has two cooperating causes, both in `ext/pdo_sqlite/sqlite_statement.c`, and is fully reproducible and root-caused.

---

## PHP Version

Reproducible on all PHP versions that link `pdo_sqlite` against Apple's system SQLite. On macOS 12 (Monterey) and later, Apple no longer ships `/usr/bin/php`, so the affected PHP is typically one explicitly compiled against `/usr/lib/libsqlite3.dylib`. Homebrew PHP links against its own libsqlite3 and is **not** affected.

To confirm which SQLite library the active PHP links against:

```bash
php -r "echo SQLite3::version()['versionString'], PHP_EOL;"
otool -L "$(php -r 'echo ini_get("extension_dir");')/pdo_sqlite.so" | grep sqlite
```

The bug appears when the output shows `/usr/lib/libsqlite3.dylib`. Apple's build carries the source-ID suffix `apl` (e.g. `...114dcaapl`) and is compiled with `SQLITE_ENABLE_API_ARMOR`.

---

## Operating System

macOS (any version where PHP links `pdo_sqlite` against `/usr/lib/libsqlite3.dylib`). Not reproducible on Linux or on macOS with Homebrew PHP.

---

## Steps to Reproduce

```php
<?php
$pdo = new PDO('sqlite::memory:');
$pdo->exec('CREATE TABLE t (id INTEGER PRIMARY KEY, val TEXT)');
$pdo->exec("INSERT INTO t VALUES (1, 'b'), (2, 'a')");

// Any ORDER BY triggers the exception:
$stmt = $pdo->prepare('SELECT id, val FROM t ORDER BY val ASC');
$stmt->execute();           // may raise here
$rows = $stmt->fetchAll();  // or here
var_dump($rows);
```

A single-column minimal case also reproduces: `SELECT id FROM t ORDER BY id`.

---

## Expected Results

```
array(2) {
  [0]=> array(4) { ["id"]=> int(2) ["val"]=> string(1) "a" ... }
  [1]=> array(4) { ["id"]=> int(1) ["val"]=> string(1) "b" ... }
}
```

---

## Actual Results

```
PHP Fatal error: Uncaught PDOException: SQLSTATE[HY000]: General error: 25 column index out of range
```

---

## Root Cause

### Background — Apple's `SQLITE_ENABLE_API_ARMOR`

Apple's `/usr/lib/libsqlite3.dylib` is compiled with `SQLITE_ENABLE_API_ARMOR`. This causes `sqlite3_column_type()` (and `sqlite3_column_text()`, `sqlite3_column_int()`, etc.) to call `sqlite3_log(SQLITE_RANGE, "column index out of range")` and set `db->errCode = 25` when called with an index `>= column_count`. On standard SQLite builds these calls return `SQLITE_NULL` / 0 silently. `sqlite3_column_name()` and `sqlite3_column_decltype()` do **not** trigger this behaviour for out-of-bounds indices.

### Cause 1 — Apple's internal ORDER BY processing calls `sqlite3_column_type()` with sort-key indices

When `sqlite3_step()` processes an `ORDER BY` query, Apple's SQLite accesses sort-key columns via the public `sqlite3_column_type()` API using internal result indices that are `>= result_column_count`. For example, for `SELECT id FROM t ORDER BY val ASC`, the sort key `val` is at internal index 1 while `result_column_count = 1`. `SQLITE_ENABLE_API_ARMOR` fires and sets `db->errCode = SQLITE_RANGE (25)`.

### Cause 2 — `pdo_sqlite_stmt_get_col` uses `sqlite3_data_count()` as a bounds guard

In `ext/pdo_sqlite/sqlite_statement.c`, `pdo_sqlite_stmt_get_col` begins:

```c
if(colno >= sqlite3_data_count(S->stmt)) {
    pdo_sqlite_error_stmt(stmt);
    return 0;
}
switch (sqlite3_column_type(S->stmt, colno)) {
```

`sqlite3_data_count()` returns the number of columns in the **current row** — it returns `0` after `sqlite3_reset()`. After `pdo_sqlite_stmt_execute` sees `SQLITE_DONE` and calls `sqlite3_reset(S->stmt)`, `sqlite3_data_count()` drops to 0. Any subsequent call to `pdo_sqlite_stmt_get_col` with any `colno` satisfies `colno >= 0 == TRUE`, causing `pdo_sqlite_error_stmt(stmt)` to be called unconditionally.

### The full chain

1. `sqlite3_step()` processes ORDER BY sort keys; Apple's ENABLE_API_ARMOR sets `db->errCode = SQLITE_RANGE (25)`.
2. `pdo_sqlite_stmt_execute` SQLITE_DONE branch calls `sqlite3_reset(S->stmt)` — `data_count` drops to 0; `db->errCode` remains 25.
3. `pdo_sqlite_stmt_get_col` guard: `colno >= sqlite3_data_count() == colno >= 0 == TRUE` → `pdo_sqlite_error_stmt(stmt)` called.
4. `pdo_sqlite_error_stmt` reads `sqlite3_errcode(S->H->db) == 25 (SQLITE_RANGE)` → raises `PDOException` SQLSTATE HY000 / 25 / "column index out of range".

> **Note on `pdo_sqlite_stmt_describe`:** this function was audited including via git history going back 6+ years. It calls only `sqlite3_column_name()` and uses `sqlite3_column_count()` for bounds. It never calls `sqlite3_column_type()` and is **not** the bug site.

---

## C-Level Proof

A standalone C probe tests `/usr/lib/libsqlite3.dylib` directly via the SQLite C API without PHP involvement. Building with `-DSQLITE_ENABLE_API_ARMOR` (which exposes the header declarations; the runtime armor code is always present in Apple's dylib) and calling `sqlite3_column_type()` with an out-of-bounds index reproducibly sets `sqlite3_extended_errcode()` to 25. `sqlite3_column_name()` and `sqlite3_column_decltype()` called with the same index do not.

Probe FINDINGS SUMMARY output on Apple SQLite:

```
FINDINGS SUMMARY
============================================================
  Part 1  Hidden sort-key columns / NULL column names   CLEAN
  Part 2  ENABLE_COLUMN_METADATA functions              CLEAN
  Part 3  ENABLE_API_ARMOR out-of-bounds detection      CONFIRMED
```

The Part 3 output shows, per query shape and per accessor:

```
sqlite3_column_name     [index=1, before step]  ec_before=0   ec_after=0   (no change)
sqlite3_column_decltype [index=1, before step]  ec_before=0   ec_after=0   (no change)
sqlite3_column_type     [index=1, before step]  ec_before=0   ec_after=25  <--- SQLITE_RANGE SET — API_ARMOR CONFIRMED
```

---

## Proposed Fix

### Fix 1 — Correct the bounds guard in `pdo_sqlite_stmt_get_col`

**File:** `ext/pdo_sqlite/sqlite_statement.c`

Change:
```c
if(colno >= sqlite3_data_count(S->stmt)) {
```
To:
```c
if(colno >= sqlite3_column_count(S->stmt)) {
```

`sqlite3_column_count()` returns the prepared statement's column count regardless of execution state and is the correct guard for column metadata access. `sqlite3_data_count()` is intended to reflect whether a row is currently available, not to bound column index checks.

This fix alone is sufficient to prevent the PDOException from being raised.

### Fix 2 — Clear stale error state after `sqlite3_reset()` in `pdo_sqlite_stmt_execute`

**File:** `ext/pdo_sqlite/sqlite_statement.c`, SQLITE_DONE branch

```c
case SQLITE_DONE:
    php_pdo_stmt_set_column_count(stmt, sqlite3_column_count(S->stmt));
    stmt->row_count = sqlite3_changes(S->H->db);
    sqlite3_reset(S->stmt);
    /* Clear any error state Apple's armored SQLite may have set during
       internal ORDER BY sort-key processing (SQLITE_ENABLE_API_ARMOR). */
    sqlite3_exec(S->H->db, "SELECT 1", NULL, NULL, NULL);
    S->done = 1;
    return 1;
```

This prevents the stale `db->errCode = 25` from persisting to subsequent queries on the same connection.

---

## Notes

- The bug is specific to Apple's `/usr/lib/libsqlite3.dylib` with `SQLITE_ENABLE_API_ARMOR`. Standard SQLite builds (Linux, Homebrew) are not affected because out-of-bounds `sqlite3_column_type()` calls return `SQLITE_NULL` silently.
- Queries without `ORDER BY` are not affected because no sort-key column access occurs inside `sqlite3_step()`.
- The `sqlite3_data_count()` guard bug is latent on unarmored builds: for successful non-DONE executions `data_count == column_count`, so the wrong guard never misfires. It only misfires when `data_count` drops to 0 after reset **and** a stale non-zero `db->errCode` is present.

### PHP Version

```plain
PHP 8.1.34 (cli) (built: Dec 16 2025 18:33:34) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.1.34, Copyright (c) Zend Technologies
    with Xdebug v3.2.1, Copyright (c) 2002-2023, by Derick Rethans
    with Zend OPcache v8.1.34, Copyright (c), by Zend Technologies
```

### Operating System

macOS 26.3.1, most relevant!
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.