Re: Dealing with LLMs in IETF discussions draft

Hi Chong,

I'm not convinced of the "human's originate" argument, at least not for the recent generations of frontier models.  I think that it is easy to assume that because LLMs are just token prediction engjnes that means they cannot originate new ideas or concepts, but only regurgitate what is already there, but I don't think that is true.

E.g., this link (https://www.forbes.com/sites/jonmarkman/2026/08/03/openais-astra-solved-10-decades-old-math-problems-for-just-2000/) is about how LLMs were used to cheaply solve maths problems that no human mathematicians have been able to solve.  There must surely be some element of an LLM creating new knowledge as part of forming these proofs even if the individual steps of the proofs happen to build on other known work.

As a complete tangent to the discussion, there is also this book: https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_Dies, where the authors claim that if anyone builds AIs (using current techniques and little actual control of their behaviour) that have greater than human intelligence then it is a 100% guaranteed certainty to wipe out humanity.  Since I believe that their proposed mitigation steps are not likely I hope they are wrong ;-)

Kind regards,
Rob


From: chong feng <[email protected]>
Date: Friday, 7 August 2026 at 07:42
To: Nathanael Ritz <[email protected]>
Cc: Stephen Farrell <[email protected]>; IETF-Discussion <[email protected]>
Subject: Re: Dealing with LLMs in IETF discussions draft

Hi Nathanael,

Thanks for the thoughtful comments. I think you raised an important
distinction that is easy to overlook: the difference between knowledge
that is new to an individual and knowledge that is new to humanity.

I agree that LLMs can introduce concepts, connections, or perspectives
that are new to the human user. However, I am not sure this is
fundamentally different from other sources of intellectual input ―
reading a paper, a book, an encyclopedia, or discussing with an
expert.

Let me give a concrete example from my own work. AIN started from a
question I did not know the answer to: how can AI agents collaborate
with each other at scale? I asked an LLM and learned about existing
approaches. That information was useful, but I identified a
fundamental bottleneck: manually configured relationships between
agents would not scale. This led me to ask a different question: why
can't agents discover each other through something analogous to
routing, not based on network addresses but on semantic intent?

The LLM helped me explore the existing landscape and validate the
direction. But it did not identify the bottleneck, make the
cross-domain analogy with IP routing, or create the abstraction of
semantic addressing. Those steps required human understanding of the
problem space and the ability to connect concepts across domains.

This is what I mean by "humans originate." The origin is not about
where information first entered my mind. It is about who performs the
abstraction, integration, and judgement required to turn information
into a new conceptual framework.

This distinction matters for IETF. The final contributor needs to
understand, evaluate, and stand behind the technical claims being
made. If someone submits a draft they cannot explain under questioning
― whether it was produced with AI, a ghostwriter, or copied from
existing material ― the community will discover that through review.
The important question is not tracing the provenance of every idea,
but whether the author truly understands and takes responsibility for
the work.

Best,
Chong

Nathanael Ritz <[email protected]> 于2026年8月7日周五 11:09写道：
>
> Comments inline
>
> On Thu, 6 Aug 2026 at 20:24, Jay Daley <[email protected]> wrote:
>>
>> Hi Stephen, Chong
>>
>> > We just pushed a -01 version of the draft [1] that lists
>> > (most of) the arguments raised on the list as that may be
>> > a more useful starting point for a discussion on a new
>> > list.
>>
>> Thanks for this.
>>
>> The one big thing the document does not tackle is the assertion that where the AI introduces something the human did not know about, then the human will inevitably trade off their own understanding of that for the acceleration of output that AIs enable.
>>
>>
>> In addition, the hypothesis of section 4 of the document relies on two pillars that I think obscure the real issues:
>>
>> 1. The claim that only humans can create new knowledge, but not AIs, and that is what is being introduced by the human into the discourse:
>>
>> > Genuine innovation --- the creation of new conceptual territory rather than more efficient mapping of existing terrain --- remains a human capacity.
>>
>>
>> > The symmetry is clean: humans originate, AI executes. Humans open new territory; AI operates efficiently within it.
>>
>>
>> I doubt very much if every message sent with the use of an AI is creating new knowledge rather than applying existing knowledge and would be surprised if it happens at all.  Engineering is, by definition, the application of existing knowledge to new problems.  It is more likely that authors who think this is happening are mistaking knowledge that is new to them as knowledge that is new to humanity.
>>
>> We could always test it by asking an AI "is this message introducing any genuine innovation or is it just applying existing knowledge?"
>>
>> 2.  The claim that the human being the decision maker somehow shapes what is said:
>>
>> > the AI is functioning as a thinking partner within a bounded space, not as an originator. The human remains the decision-maker about what to accept and what to discard. What falls outside this paradigm is delegating the thinking itself: asking AI what position to take, what arguments to make, or what conclusions to draw
>>
>> The key point that is being obscured there is that the AI will inevitably be introducing that human to concepts that are new to that human and if they are incorporated in any way, either directly or by linkage/comparison, then the AI is responsible for that innovation not the human.  This is exactly the same as if I were to sit down with an expert in a particular field and discuss my proposed email with them and then ask them to draft if for me.
>>
>
> Ultimately, I don't think many of us can claim to know much beyond what others have imparted to us anyway. Even the self-taught autodidacts surely started by reading through something of personal interest that someone else first discussed, and then experimenting from there.
>
> In this case, I honestly don't see the situation as dramatically different from an informal web search. We have not and do not typically cite every signal piece of knowledge we share that's new to us despite having access to web search, Wikipedia and/or say TV Tropes for figurative eons.
>
> And much like I personally often saw Wikipedia as a useful jumping off point, I'm sure we know plenty of people who treat a two-minute read of a Wikipedia article as having 'done their research' enough to speak confidently on a subject they don't really have the means to understand yet.
>
> What's different, perhaps, is an LLM's conversational nature, which allows for much faster traversal of a topic. This, combined with AI's unguarded propensity to make stuff up while speaking as confidently as a genuine expert (who might otherwise know when to say "I don't know") creates some genuine risks when neither party in the exchange has enough information to make informed decisions about what was just reviewed.
>
> But where someone learns something genuinely new to them and then composes it with another existing system to introduce some productive novel effect?
>
>>
>> This conversation has made it clear to me that this is about so much more than someone putting their name to the output and taking responsibility for it, because it seems as if the boundary between what the AI introduced and what the human introduced is hard for the human to self-identify.
>
>
> In that case, while deeply fascinating to me personally, I am not sure why it would matter so much if it was AI, Wikipedia, a book or a magazine that first made them wise to the subject matter.
>
> Cheers,
> Nathanael
>
>>
>>
>> Jay
>>
>>
>> --
>> Jay Daley
>> [email protected]
>> www.ietf.org<http://www.ietf.org>
>>
>>
>>
>>