Re: Linux 7.2

Rich <[email protected]> wrote:
> The pixie girl is there to *try* to halt the onslaught of the AI 
> companies that do abjectly stupid things like download every version of 
> linux on kernel.org from 0.0.1 through to 7.2 278 times a day, every 
> day, times 75 AI companies to feed to their AI models.
> 
> I setup a really simple markov chain text generator as an AI honey pot 
> on a server I admin and linked it to one, hidden (display: none; in the 
> css) link on the page footer, with a robots.txt saying "don't access 
> this link".

You don't need to try hard to trap one of these crawlers like that.
Just enable PHPSESSID on a PHP website and every time it gets a new
session variable it will think it's found a whole new website,
while remembering to still go back and re-crawl the site with all
the old PHPSESSIDs it's ever received. "AmazonBot" is still trying
old PHPSESSID URLs on one site many months after I disabled
generating them.

It's more than a little ironic that the "AI revolution" seems to
have born a whole new level of idiocy in web crawling. We'll just
have to keep waiting until their money runs out I suppose...

> That's why the pixie girl is there on kernel.org.  You can like or 
> dislike it, but the "burn" from the AI companies irresponsible accesses 
> is the underlying cause.

But how much "burn" for real users of such websites is acceptable?
They could password-protect the whole site and send out passwords
to just the people who they think matter, telling everyone else to
bugger off. Somewhere between that and trying to optimise/upgrade
their website to handle the new AI load without adding any new
access restrictions is the "pixie girl" solution which just tells
people like me using obscure web browsers to bugger off. I still
won't be OK with effectively being told to bugger off, especially
by an organisation with as much money as The Linux Foundation
(though browsing their Git repos is still working for me in Dillo).

-- 
__          __
#_ < |\| |< _#