Re: New global API rate limits deployed for anonymous requests

Great idea, I think Referer is completely reasonable as a rate limiting
signal, because it is no worse than User-Agent from a spoofing/abuse
perspective.

-Travis

On Mon, Mar 16, 2026 at 3:45 AM Addshore via Wikitech-l <
[email protected]> wrote:

> > Maybe one solution would be to move unauthenticated requests to the
> > backend (the PHP API layer) as well because then it's clear that
> > they're coming from toolforge.org? But that would unfortunately
> > require a lot of refactoring of this app, and many older ones which i
> > doubt still have maintainers.
>
> >  Also, you would basically be running an open proxy now, that could be
> abused by scrapers to access the API. If the kind of access is
> restrictive enough this could be a viable work-around, but you'd have to
> keep an eye on it to spot abuse.
>
> So I tried this (in a way) for wikicrowd at the hackathon, with
> authentication still for the user to this (open proxy) to both 1 perform
> the requests fro WMCS but then also more easily authenticate them.
> However due to the number of requests tihs essentially just overwhelms my
> tool infrastructure.
> I have since undone this.
>
> I was then also directed toward `Api-User-Agent` and the hcakathon, and
> implemented this for requests, but still ran into 429 issues.
>
> I'll be trying to take a fesh look at this at some point to try and
> reoslve some issues.
>
> wikicrowd essentially makes requests to:
>  - Thumbnails such as this at 960px resolution
> https://upload.wikimedia.org/wikipedia/commons/thumb/6/6c/Lake_LA-4-200_Buccaneer_AN1895272.jpg/960px-Lake_LA-4-200_Buccaneer_AN1895272.jpg
>  - Wikidata rest API for label lookup, such as
> https://www.wikidata.org/w/rest.php/wikibase/v1/entities/items/Q7428391/labels_with_language_fallback/en
>  - Entity retrieval such as
> https://commons.wikimedia.org/w/api.php?action=wbgetentities&format=json&ids=M75693953&props=claims&origin=*
>
> As with depictor, for some parts of the tool these are all client side
> unauthenticated requests currently, made by the users browser.
> I need to read up on the docs for doig user authentication for these GET
> requests.
>
> I can likely resolve some of my issues by changing some of these requests
> for additional batching (Vs the current optimization which is for caching).
>
> I also wonder if things like
> origin https://wikicrowd.toolforge.org
> referer https://wikicrowd.toolforge.org/
> could be considered as part of this rate limiting approach?
>
> On Sun, 15 Mar 2026 at 10:08, Daniel Kinzler via Wikitech-l <
> [email protected]> wrote:
>
>>
>> Am 15.03.26 um 09:03 schrieb Siddharth VP:
>>
>> So, it would be great if we could treat requests to
>> rest.php/oauth2/authorize as trusted even if they don't have a user agent.
>>
>> This should become a non-issue when that endpoint is exempted from rate
>> limits.
>>
>> --
>> Daniel Kinzler
>> Principal Software Engineer
>> MediaWiki Engineering Group
>> Wikimedia Foundation
>>
>>
>> _______________________________________________
>> Wikitech-l mailing list -- [email protected]
>> To unsubscribe send an email to [email protected]
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> _______________________________________________
> Wikitech-l mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

_______________________________________________
Wikitech-l mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/