Re: Effect of blocking AI crawlers in forums and wikis
Marco van de Voort via fpc-pascal <[email protected]> Sat, 6 Sep 2025 17:41:05 +0200
| Newsgroups | gmane.comp.compilers.free-pascal.general |
|---|---|
| Message-ID | <[email protected]> |
Op 5-9-2025 om 23:05 schreef Wayne Sherman via fpc-pascal: > Is it possible to make the forum and wiki data available on a mirror > for the use of AI crawlers? > (are search engine crawlers also blocked?) To my best knowledge, only misbehaving, high frequency robots are banned. These mostly operate out of Asia with very high numbers of distinct IPs and very high fetch frequencies. It appears there are Chinese ISPs that offer packages to "accelerate" gathering content for AIs. It is unlikely to affect good citizens like ChatGPT. _______________________________________________ fpc-pascal maillist - [email protected] https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal