Re: AGENTS.md

--000000000000b85e66065826a0c1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Aug 3, 2026 at 11:56=E2=80=AFAM Sean Whitton <[email protected]=
ame>
wrote:

> Eli Zaretskii [03/Aug  3:46pm +03] wrote:
> > That would help, but I was actually asking about some evidence that
> > someone already provided.  Perhaps some investigation into the effects
> > of this is available somewhere?
>
> Well, I just ran a test of my own.  I asked it "Implement an Emacs
> function to show the message "hello" whenever someone enters VC-Dir."
>
> It refused, it said it "couldn't" do it because of AGENTS.md.
> It then briefly commented on what a solution would look like.
>

Which "it" did you use and do you think that represents the behavior of all
the other ones?  Our basic testing suggested many/most/sometimes all
negated instructions get ignored by many models/harnesses.

--000000000000b85e66065826a0c1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon=
t-family:monospace"><span style=3D"font-family:Arial,Helvetica,sans-serif;b=
ackground-color:transparent">On Mon, Aug 3, 2026 at 11:56=E2=80=AFAM Sean W=
hitton &lt;<a href=3D"mailto:[email protected]">spwhitton@spwhitton.=
name</a>&gt; wrote:</span></div></div><div class=3D"gmail_quote gmail_quote=
_container"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0=
.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Eli Zaretskii=
 [03/Aug=C2=A0 3:46pm +03] wrote:<br>
&gt; That would help, but I was actually asking about some evidence that<br=
>
&gt; someone already provided.=C2=A0 Perhaps some investigation into the ef=
fects<br>
&gt; of this is available somewhere?<br>
<br>
Well, I just ran a test of my own.=C2=A0 I asked it &quot;Implement an Emac=
s<br>
function to show the message &quot;hello&quot; whenever someone enters VC-D=
ir.&quot;<br>
<br>
It refused, it said it &quot;couldn&#39;t&quot; do it because of AGENTS.md.=
<br>
It then briefly commented on what a solution would look like.<br></blockquo=
te><div><br></div><div class=3D"gmail_default" style=3D"font-family:monospa=
ce">Which &quot;it&quot; did you use and do you think that represents the b=
ehavior of all the other ones?=C2=A0 Our basic testing suggested many/most/=
sometimes all negated instructions get ignored by many models/harnesses.</d=
iv></div></div>

--000000000000b85e66065826a0c1--