Re: Discussing a Standardized Asynchronous Interface for DBI

--43b2887c950363f197869a30500048aa1beee9ba
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

/me raises hand

On Fri, Jul 10, 2026, at 12:46, C.J. Collier wrote:
> On Fri, Jul 10, 2026 at 12:32=E2=80=AFAM David Nicol <davidnicol@gmail=
.com> wrote:
>=20
>>=20
>> On Fri, Jul 10, 2026 at 1:06=E2=80=AFAM C.J. Collier <cjac@colliertec=
h.org> wrote:
>>>=20
>>> It would give modern cloud-native drivers a standard interface to ta=
rget, rather than everyone hacking their own custom event loops into ind=
ividual DBDs.
>>>=20
>>> Thoughts?
>>=20
>>=20
>> Nice! The problem with event loops is that a system should have only =
one. Is there a thing like "async-check-and-do" that will do stuff or no=
t block, that could get polled as part of an existing event loop?=20
>>=20
>> A quick search yields https://metacpan.org/dist/POE-Component-EasyDBI=
 for discussion; POE wants to be the standard event loop services provid=
er.
>>=20
>> I'm old-school and still write loops around select(2), which allows a=
sync comms with separate processes using a little language over a pipe, =
robustly, but that's no way to define a standard interface unless the fa=
cility gets wrapped somehow. The tradeoff of only polling after the sele=
ct times out would be acceptable, I think, although it wouldn't be as re=
sponsive as including the protobuf system's file descriptors in the sele=
ct set.
>>=20
>> I don't know if there is a standard for mapping a FD number to a [on-=
readable, on-writable, on-error] coderef triple, aside from just doing t=
hat with an AoA, but were I wanting to integrate an async system into an=
 existing select loop, I could work with that. Especially if the system =
offered to take the same shape of data as an input for its own select lo=
op.POE might have a standard for such things, I don't know.
>>=20
>> cheers
>>=20
>> dln
>>=20
>>=20
>> --
>> "The profit motive is often in conflict with the aims of art." -- Urs=
ula K. Le Guin
>=20
> David, thank you for reinforcing the importance of the single event lo=
op. That=E2=80=99s a fundamental constraint any enhancement to DBI must =
honor.
>=20
> Your insights, combined with the trend towards streaming protocols and=
 managed services in cloud databases (e.g., gRPC for BigQuery/Spanner, a=
nd the native protocol for Cloud SQL for PostgreSQL <https://cloud.googl=
e.com/sql/docs/postgres>), highlight the value of standardized non-block=
ing operations within the DBI ecosystem.
>=20
> I=E2=80=99m mindful of the potential complexity and maintenance burden=
 of introducing new concepts into the DBI core. However, the complexity =
seems to be arising anyway, but in driver-specific ways (e.g., `DBD::Pg`=
=E2=80=99s async features). Without a common interface, we risk greater =
ecosystem fragmentation. Centralizing the async interaction model within=
 DBI, while keeping it optional, could *tame* this complexity.
>=20
> Furthermore, DBI has a history of adapting beyond traditional SQL data=
bases, as seen with drivers like `DBD::CSV`. A well-designed, event-loop=
-agnostic async interface could not only benefit SQL databases but also =
pave the way for more performant and idiomatic drivers for other modern =
data systems, including NoSQL stores, message queues, and real-time plat=
forms, which often rely on persistent connections and server-push.
>=20
> To what extent would the community be interested in formalizing an exp=
loration of this? I propose we establish a small working group to design=
 an optional, additive interface to the DBI specification. The goal woul=
d be to offer a clear, event-loop-agnostic contract for drivers to integ=
rate with any external event loop.
>=20
> This would not alter behavior for existing synchronous code but would =
aim to provide a solid foundation for the next generation of Perl databa=
se and data source drivers.
>=20
> Is this a direction others would be keen to contribute to?
>=20
> Regards,
>=20
>=20
>=20
> C.J.=20

--43b2887c950363f197869a30500048aa1beee9ba
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><title></title></head><body><div>/me raises h=
and</div><div><br></div><div>On Fri, Jul 10, 2026, at 12:46, C.J. Collie=
r wrote:</div><blockquote type=3D"cite" id=3D"qt" style=3D""><div dir=3D=
"ltr"><div dir=3D"ltr"><p><span style=3D"background-color:transparent;">=
On Fri, Jul 10, 2026 at 12:32=E2=80=AFAM David Nicol &lt;<a href=3D"mail=
to:[email protected]">[email protected]</a>&gt; wrote:</span></p><=
/div><div class=3D"qt-gmail_quote qt-gmail_quote_container"><blockquote =
class=3D"qt-gmail_quote" style=3D"margin-top:0px;margin-right:0px;margin=
-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-style:so=
lid;border-left-color:rgb(204, 204, 204);padding-left:1ex;"><div dir=3D"=
ltr"><div dir=3D"ltr"><br></div><div class=3D"qt-gmail_quote"><div dir=3D=
"ltr" class=3D"qt-gmail_attr">On Fri, Jul 10, 2026 at 1:06=E2=80=AFAM C.=
J. Collier &lt;<a href=3D"mailto:[email protected]" target=3D"_blank"=
>[email protected]</a>&gt; wrote:</div><blockquote class=3D"qt-gmail_=
quote" style=3D"margin-top:0px;margin-right:0px;margin-bottom:0px;margin=
-left:0.8ex;border-left-width:1px;border-left-style:solid;border-left-co=
lor:rgb(204, 204, 204);padding-left:1ex;"><div dir=3D"auto"><div dir=3D"=
auto"><div dir=3D"auto"><br></div><div dir=3D"auto">It would give modern=
 cloud-native drivers a standard interface to target, rather than everyo=
ne hacking their own custom event loops into individual DBDs.</div><div =
dir=3D"auto"><br></div><div dir=3D"auto">Thoughts?</div></div></div></bl=
ockquote></div><div><br></div><div><br></div><div>Nice! The problem with=
 event loops is that a system should have only one. Is there a thing lik=
e "async-check-and-do" that will do stuff or not block, that could get p=
olled as part of an existing event loop?&nbsp;</div><div><br></div><div>=
A quick search yields&nbsp;<a href=3D"https://metacpan.org/dist/POE-Comp=
onent-EasyDBI" target=3D"_blank">https://metacpan.org/dist/POE-Component=
-EasyDBI</a> for discussion; POE wants to be the standard event loop ser=
vices provider.</div><div><br></div><div>I'm old-school and still write =
loops around select(2), which allows async comms with separate processes=
 using a little language over a pipe, robustly, but that's no way to def=
ine a standard interface unless the facility gets wrapped somehow. The t=
radeoff of only polling after the select times out would be acceptable, =
I think, although it wouldn't be as responsive as including the protobuf=
 system's file descriptors in the select set.</div><div><br></div><div>I=
 don't know if there is a standard for mapping a FD number to a [on-read=
able, on-writable, on-error] coderef triple, aside from just doing that =
with an AoA, but were I wanting to integrate an async system into an exi=
sting select loop, I could work with that. Especially if the system offe=
red to take the same shape of data as an input for its own select loop.P=
OE might have a standard for such things, I don't know.</div><div><br></=
div><div>cheers</div><div><br></div><div>dln</div><div><br></div><div><b=
r></div><div><span class=3D"qt-gmail_signature_prefix">--</span></div><d=
iv dir=3D"ltr" class=3D"qt-gmail_signature"><div dir=3D"ltr"><span class=
=3D"font" style=3D"font-family:&quot;times new roman&quot;, serif;">"The=
 profit motive is often in conflict with the aims of art." -- Ursula K. =
Le Guin</span></div></div></div></blockquote><div><br></div><p>David, th=
ank you for reinforcing the importance of the single event loop. That=E2=
=80=99s a fundamental constraint any enhancement to DBI must honor.</p><=
p>Your insights, combined with the trend towards streaming protocols and=
 managed services in cloud databases (e.g., gRPC for BigQuery/Spanner, a=
nd the native protocol for&nbsp;<a href=3D"https://cloud.google.com/sql/=
docs/postgres">Cloud SQL for PostgreSQL</a>), highlight the value of sta=
ndardized non-blocking operations within the DBI ecosystem.</p><p>I=E2=80=
=99m mindful of the potential complexity and maintenance burden of intro=
ducing new concepts into the DBI core. However, the complexity seems to =
be arising anyway, but in driver-specific ways (e.g.,&nbsp;<code>DBD::Pg=
</code>=E2=80=99s async features). Without a common interface, we risk g=
reater ecosystem fragmentation. Centralizing the async interaction model=
 within DBI, while keeping it optional, could&nbsp;<i>tame</i>&nbsp;this=
 complexity.</p><p>Furthermore, DBI has a history of adapting beyond tra=
ditional SQL databases, as seen with drivers like&nbsp;<code>DBD::CSV</c=
ode>. A well-designed, event-loop-agnostic async interface could not onl=
y benefit SQL databases but also pave the way for more performant and id=
iomatic drivers for other modern data systems, including NoSQL stores, m=
essage queues, and real-time platforms, which often rely on persistent c=
onnections and server-push.</p><p>To what extent would the community be =
interested in formalizing an exploration of this? I propose we establish=
 a small working group to design an optional, additive interface to the =
DBI specification. The goal would be to offer a clear, event-loop-agnost=
ic contract for drivers to integrate with any external event loop.</p><p=
>This would not alter behavior for existing synchronous code but would a=
im to provide a solid foundation for the next generation of Perl databas=
e and data source drivers.</p><p>Is this a direction others would be kee=
n to contribute to?</p><p>Regards,</p><p><br></p><div><span style=3D"bac=
kground-color:transparent;">C.J.</span>&nbsp;</div></div></div></blockqu=
ote><div><br></div></body></html>
--43b2887c950363f197869a30500048aa1beee9ba--