Re: Dealing with LLMs in IETF discussions draft

Stephen, Chong

Thanks for this - fascinating document.

I spend a lot of time, mainly personal, using AI and from that =
experience it seems to me that the most crucial aspect of the use of AI =
is being obscured by the focus on "producing precise, idiomatic =
technical prose in a second language" and in particular the statement =
"AI removes that cost without changing who is responsible for the =
ideas".=20

To be clear, we are not talking about straight translation here, but, as =
explicitly explained, full AI generation of text based on instructions =
from a human after an iterative interaction of back-and-forth =
discussions. =20

The best way I can currently describe this iterative interaction with AI =
is that it augments my thinking and accelerates my ability to write =
something as a result of that, whether that is text or code.  The =
augmentation is because it can provide an expert review of my thinking, =
highlighting issues, suggesting new lines of reasoning and including =
concepts and ideas that I am unfamiliar with.

However, and this is the big "however", every time I use it I am faced =
with a tradeoff between how much acceleration I want and how much I =
understand of what is being done - the more I want to understand, the =
more I have to slow it down.  To give an example the Tools Team have =
been discussing, if the developers read every line an AI produces then =
the acceleration is low (e.g. 1.5x, 2x) , but if they don=E2=80=99t read =
every line then the acceleration can be high (e.g. 5x, 10x).  The =
temptation to lean into the acceleration is intense as it can be so =
productive.

The third key observation I would make about AI, which is touched on in =
the document, is that they very rarely disagree with the human they are =
interacting with.  They are programmed to be people-pleasers and take =
you at your word.  This regularly leads me down the wrong path and it =
can be weeks later before I realise that and explain to the AI why I =
think that, only for it to then agree with my new path as confidently as =
it agreed with the original wrong path. Admittedly the use of =
adversarial AIs can go some way to addressing that problem but I can=E2=80=
=99t yet judge the efficacy.

The claim I would make is that, in most cases, where this process has =
been followed and AI generated text supplied then three key things =
change from when a human thinks alone, which I can best explain with =
three questions:

1. Did all of the ideas originate with that person?  My answer is "no", =
the AI suggested things that the human agreed with and incorporated.

2. Does the person fully understand what the AI has written?  My answer =
is also "no", the human chose acceleration over understanding.

3. Has the person fully thought through the proposal to understand where =
it fails?  My answer is also "no" because AIs just don=E2=80=99t think =
that way (though perhaps with adversarial AI use this might improve).


So the real crisis then is, what value are contributions where the =
answer is "no" to those three questions?

Jay


> On 31 Jul 2026, at 13:20, Stephen Farrell =
<[email protected]> wrote:
>=20
>=20
> Hi all,
>=20
> Myself and Chong Feng have posted an I-D [1] that (from two
> pretty different perspectives) aims to progress discussion
> of how we might better handle LLM text in IETF discussions.
>=20
> I think this'd be a fine topic for broad discussion. I do
> not know what would be the correct mailing list for such a
> discussion, but also think it doesn't warrant waiting for
> another round of dispatchery before being discussed. (Nor
> would any dispatch outcome above 'new mailing list' suit.)
>=20
> So, I'd be v. interested in comments on this and/or on it
> being moved to some other mailing list for discussion. If
> no existing list suits, I'd suggest [email protected]
> (which could be contracted to [email protected]:-) as a name
> for a new list, but whatever works is fine.
>=20
> Meanwhile, we'd be v. interested in any comments here or
> on the github repo [2].
>=20
> And of course, this is a v. rough thing, not intended to
> be prescriptive or authoritative, so have at it:-)
>=20
> Cheers,
> S.
>=20
> [1] https://datatracker.ietf.org/doc/draft-fengfar-led/
> [2] https://github.com/sftcd/led/
>=20

--=20
Jay Daley
[email protected]
www.ietf.org