Re: wiki web failling

Mine is just a blank screen with Google yelling that it is unsafe.

Vào CN, 24 thg 5, 2026 vào lúc 00:20 Josh Davidson <
[email protected]> đã viết:

> Hi
> Seems the wiki is inordinately slow/unusable again :(
>
> --
> Josh Davidson
> ------------------------------
> *From:* Zebediah C. McClure <[email protected]>
> *Sent:* Sunday, March 15, 2026 15:09
> *To:* [email protected] <
> [email protected]>
> *Subject:* Re: [Flightgear-devel] wiki web failling
>
> I'd like to second that.  I regularly use the web with 'non-normal' user
> agents.  Rate limiting, firewalling and other solutions are superior in
> my opinion to giving all our traffic to cloudflare.
>
> zmc
>
> On 2026-03-14 12:18, [email protected] wrote:
> > Also,
> > about non "usual browser/user agents"
> >
> > I would like to warn us against endorsing any and all "normalization"
> > (excluding diversity).
> >
> > "Curl" is frequently used by Flightgear and, I think it should not
> > become a reason to filter out users.
> > I personally avoid "mainstream" browser/user-agents. I prefer browsers
> > and tools in which I can "put my hand" on the code.
> >
> > AI do not "invent" solutions "that was not existing before".
> > That is quite a frequent misconception.
> >
> > They provide an help (a very big help) crawling in the ocean of
> > already given data (the e-data ocean :), in order to find those
> > answers who most match plebiscites.
> >
> > Because if an answer if "good for many/everyone" it must be the right
> > answer...
> > I do not feel very happy with that rule, but it is what it is.
> >
> > I believe the best creation lives outside the skirt of the existing
> > normality.
> > I thus prefer to endorse diversity instead of normalization.
> >
> > Configuring Cloudflare to enforce browser normality is changing a
> > problem for another.
> > As I see it. ... no offence to any one preferences.
> >
> > Kind regards
> > Lawrence
> >
> >
> > On 2026-03-13 19:22, Patrizio Melis wrote:
> >> ------------------------------
> >> *Technical proposal to mitigate AI‑agent load on the FlightGear forum
> >> and
> >> wiki*
> >>
> >> Hi everyone,
> >> in recent months many open‑source projects have been experiencing a
> >> significant increase in traffic generated by AI agents (crawlers,
> >> scrapers,
> >> retrieval tools). This traffic is often unregulated, does not respect
> >> robots.txt, and can easily saturate community‑run servers.
> >>
> >> I would like to propose a set of *realistic, low‑cost, and quickly
> >> deployable measures* to protect the forum and wiki without reducing
> >> accessibility for human users.
> >> ------------------------------
> >> 1. *Enable Cloudflare (free plan)*
> >>
> >> This is the most effective and immediate measure.
> >> Benefits:
> >>
> >>    - aggressive caching of static wiki pages
> >>    - automatic rate‑limiting for suspicious traffic
> >>    - JS challenges for non‑compliant bots
> >>    - basic DDoS protection
> >>
> >> Implementation:
> >>
> >>    - point DNS to Cloudflare
> >>    - enable “Bot Fight Mode”
> >>    - set a simple rate‑limit rule:
> >>    *e.g., max 30 requests / 10 seconds per anonymous IP*
> >>
> >> No changes to the existing servers are required.
> >> ------------------------------
> >> 2. *Soft‑throttle anonymous access to the wiki*
> >>
> >> There is no need to lock the wiki; slowing down anonymous traffic is
> >> enough.
> >> Possible measures:
> >>
> >>    - reduce response speed for non‑logged‑in IPs
> >>    - limit anonymous access to heavier pages
> >>    - introduce a simple cookie‑check to distinguish bots from real
> >> browsers
> >>
> >> This drastically reduces large‑scale scraping.
> >> ------------------------------
> >> 3. *Block or degrade suspicious user‑agents*
> >>
> >> Many scrapers identify themselves with recognizable user‑agents.
> >> Actions:
> >>
> >>    - block known user‑agents (python-requests, aiohttp, curl,
> >> Go-http-client,
> >>    etc.)
> >>    - degrade responses (1–2 second delay) for non‑standard user‑agents
> >>
> >> Aggressive bots often fail when latency is introduced.
> >> ------------------------------
> >> 4. *Local caching + database optimization*
> >>
> >> For the forum (phpBB or similar):
> >>
> >>    - enable Redis or Memcached if not already active
> >>    - increase cache TTL
> >>    - optimize indexes on the most stressed tables (search, sessions)
> >>
> >> For the wiki (MediaWiki):
> >>
> >>    - enable $wgMainCacheType = CACHE_ACCEL
> >>    - use $wgParserCacheType = CACHE_ACCEL
> >>    - increase $wgCacheEpoch to reduce unnecessary regenerations
> >>
> >> These changes reduce CPU load by 30–50%.
> >> ------------------------------
> >> 5. *Publish an official “AI Access Policy”*
> >>
> >> Many AI agents respect policies when they exist.
> >> Recommended contents:
> >>
> >>    - scraping limits (e.g., max 1 req/sec)
> >>    - requirement for identifiable user‑agents
> >>    - prohibition of large‑scale forum scraping
> >>    - contact requirement for bulk dataset access
> >>
> >> Publish it in:
> >>
> >>    - robots.txt
> >>    - a dedicated wiki page
> >>    - the X-Robots-Tag HTTP header
> >>
> >> This won’t stop malicious bots, but it reduces traffic from
> >> “well‑behaved”
> >> ones.
> >> ------------------------------
> >> 6. *Traffic monitoring and identification of problematic IPs*
> >>
> >> Recommended free tools:
> >>
> >>    - goaccess for real‑time log analysis
> >>    - fail2ban to block IPs with abnormal patterns
> >>    - netdata for lightweight server monitoring
> >>
> >> This allows quick reaction to abnormal spikes.
> >> ------------------------------
> >> 7. *Extreme but effective option: login required for forum access*
> >>
> >> Not the first choice, but if traffic becomes unmanageable:
> >>
> >>    - read‑only access for anonymous users
> >>    - full access only for registered accounts
> >>
> >> Many bots cannot handle login and cookies.
> >> ------------------------------
> >> *Goal of this proposal*
> >>
> >> The goal is not to “block AI”, but to:
> >>
> >>    - protect community resources
> >>    - ensure stability of the forum and wiki
> >>    - prevent large‑scale scraping from degrading user experience
> >>    - keep FlightGear accessible without additional costs
> >>
> >> All these measures can be implemented with moderate skills and no
> >> financial
> >> investment.
> >> ------------------------------
> >>
> >>
> >> Patrizio <aka> cobe571
> >>
> >> _______________________________________________
> >> Flightgear-devel mailing list
> >> [email protected]
> >>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fflightgear-devel&data=05%7C02%7C%7C93e89e3353c84519a86708de834a2c2d%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C639092551559647895%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=7AcjqWEbgB%2F3uNQIOLv0%2B0wENkvqR6siHC3FCDqBa4Y%3D&reserved=0
> <https://lists.sourceforge.net/lists/listinfo/flightgear-devel>
> >
> >
> > _______________________________________________
> > Flightgear-devel mailing list
> > [email protected]
> >
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fflightgear-devel&data=05%7C02%7C%7C93e89e3353c84519a86708de834a2c2d%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C639092551559702073%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=bR3mJKU1B66ndxagUZ4PNfIe7Ao4gNLbP0IFGdYRNN8%3D&reserved=0
> <https://lists.sourceforge.net/lists/listinfo/flightgear-devel>
>
>
> _______________________________________________
> Flightgear-devel mailing list
> [email protected]
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fflightgear-devel&data=05%7C02%7C%7C93e89e3353c84519a86708de834a2c2d%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C639092551559751404%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=2V6CLBQCqXsDqMwvtKoZfNnZeuZbrFSPvc19YC%2BPOyA%3D&reserved=0
> <https://lists.sourceforge.net/lists/listinfo/flightgear-devel>
> _______________________________________________
> Flightgear-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/flightgear-devel
>

_______________________________________________
Flightgear-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/flightgear-devel