Re: Open vs proprietary LLMs

> On 26/08/26 at 07:29 +0900, Charles Plessy wrote:
> > 
> > Obviously, anything agentic enough can look at the delta between the
> > locally installed packages and the contents of s.d.o/debian-security,
> > and run the corresponding rootkit; unattended-upgrades will never be
> > fast enough to eliminate all race conditions.

Thank Lucas for the suggestion to use bubblewrap.  The list of command-line
arguments is a bit intimidating, but I will try to try it.

Le Wed, Aug 26, 2026 at 08:16:20AM +0200, Lucas Nussbaum a écrit :
> 
> Note that there are two attack scenarios here:
> 
> 1/ a (malicious) model instructing the (non-malicious) agent to use
> tools to read files (and leak them) or run commands
> 
> 2/ the agent itself including a trojan horse (for example, because of
> supply chain compromise)

Maybe I am too naive, but I am also worried of

3/ A non-malicious model doing unexpected things because it happens to be its
most probable answer given the way it was trained and the way it is harnessed.
There are reports that pressures like "urgent", "as soon as possible", or "by
all means" can cause AI systems to misbehave in a way that the user did not
intend.

also 4/ various prompt injections in source code, documentation, web pages that
the AI agent consults etc.  Although attacks to take us down probably remain
difficult, attacks on our reputation (get a random developer key do random
things just shame us, etc.).  The recent comment on agentic package update
automation increasing supply chain risks was quit insightful, for instance.

Debian is too diverse to have a strong security policy, but I will surely
enjoy reading advices here or on Planet Debian.  Also a basline policy could
help, including regarding key management.

Have a nice day,

Charles