Re: self host providers

On 2026-08-13, Theo wrote:

> Computer Nerd Kev <[email protected]> wrote:
>> Theo <[email protected]> wrote:
>> > Do consider some kind of AI bot blocking, as this consumes a big chunk of
>> > traffic and CPU time nowadays.
>> 
>> Oh great, and make another site that won't let me in unless I'm
>> using a mainstream Web browser.
>
> I didn't say that.  There are other ways, including blocking known AI
> User-Agents and blocking IPs/ranges which are seen to hammer your server. 
> It's a depressing game of cat and mouse.  The Cloudflare / Anubis
> browser-checkers are one tool you can use, but there are others.

Please do not use Cloudflare unless you configure it to somehow allow
other browsers than the ones they support. It was already frustrating
that they'd require JS, but ever since the Browser Integrity Check was
reimplemented on top of Turnstile they've stopped caring about
fallbacks.

At least Anubis in the default config allows some workarounds, and from
what I've seen some deployments even work without JS.

>> CPU load is also low (most content there is static). That said,
>> their free VPSs are over-subscribed so you don't get the sort of
>> CPU performance you might see with a paid service using an
>> equivalent-spec CPU.
>
> It's a particular problem if you are hosting dynamic content - eg I hosted
> webSVN and the bots tied up the server in knots as every page is full of links
> to other dynamic pages.  The CPU couldn't serve them fast enough to keep on
> top of the bots requesting them all.
>
> Theo

-- 
Nuno Silva