Re: How to limit access rate by header?

Eero Volotinen <[email protected]>
Newsgroups gmane.comp.apache.mod-security.user
Message-ID <CABzZrXeJefgE-QxAYNRq-acTD--8Ru1m8ykQZT3SoD7H=DM2nw@mail.gmail.com> 
are you using nginx or apache?

Luciano Guillermo Fantuzzi <[email protected]> kirjoitti pe 7.
jouluk. 2018 klo 14.19:

> Already tried that, but still the same message in debug log. Anyway, I'm
> behind Cloudflare so I need to access that var (that contains the real IP)
> from header. Moreover, I tried with global collection and same luck. Am I
> missing some initialization step in modsec?
>
> Thanks.
>
> El vie., 7 dic. 2018 00:56, Scheblein, Adam <[email protected]>
> escribió:
>
>> I had a similar problem.  You need to initialize the collection with
>> something like this:
>>
>>
>>
>>                 SecAction
>> id:'2000000',phase:1,nolog,pass,initcol:IP=%{REMOTE_ADDR}
>>
>>
>>
>> *From: *Luciano Guillermo Fantuzzi <[email protected]>
>> *Reply-To: *"[email protected]" <
>> [email protected]>
>> *Date: *Thursday, December 6, 2018 at 8:51 PM
>> *To: *"[email protected]" <
>> [email protected]>
>> *Subject: *Re: [mod-security-users] How to limit access rate by header?
>>
>>
>>
>> I've very frustrated... I can't make it work, even for IP control. What
>> am I doing wrong here? It always returns:
>>
>> Could not set variable "IP.access_count" as the collection does not exist.
>>
>>
>>
>> 105 <LocationMatch "^/.*">
>>
>> 109   SecRule REQUEST_HEADERS:CF-Connecting-IP "@unconditionalMatch"
>> "phase:2,initcol:IP=%{MATCHED_VAR},pass,nolog,id:35003"
>>
>> 112   SecRule IP:ACCESS_COUNT "@gt 1"
>> "phase:2,pause:300,deny,status:503,setenv:RATELIMITED,skip:1,nolog,id:35004"
>>
>> 116   SecAction "phase:2,setvar:IP.access_count=+1,pass,nolog,id:35005"
>>
>>
>>
>> 119   SecAction
>> "phase:5,deprecatevar:IP.access_count=1/10,pass,nolog,id:35006"
>>
>> 122   Header always set Retry-After "10" env=RATELIMITED
>>
>> 123 </LocationMatch>
>>
>> 124
>>
>> 125 ErrorDocument 503 "Service Unavailable"
>>
>>
>>
>>
>>
>> El jue., 6 de dic. de 2018 a la(s) 20:38, Luciano Guillermo Fantuzzi (
>> [email protected]) escribió:
>>
>> Thank you for your answer, Christian. Do you think it's possible for you
>> to just build the first part of the rule (in Modsec)? I'm trying but I'm
>> not understanding how variables work with the global scope. I was be able
>> to build some basic rules like:
>>
>>
>>
>> # Banned Bots and Crawlers
>>
>> SecRule REQUEST_HEADERS:User-Agent "@pmFromFile blacklist-bots.data" \
>>
>>     "id:350001,phase:1,t:none,deny,log,msg:'BANNED BOT'"
>>
>>
>>
>> # Specific IPs
>>
>> SecRule REMOTE_ADDR "@pmFromFile blacklist-ip.data" \
>>
>>     "id:350002,phase:1,t:none,deny,log,msg:'BANNED IP'"
>>
>>
>>
>> I'm trying to understand examples from stackoverflow and different
>> places, but they are all intended to limit by IP and for specific resources
>> (the scope of the rule). Eg.:
>>
>> https://gist.github.com/josnidhin/91d1ea9cd71fde386c27a9228476834e
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_josnidhin_91d1ea9cd71fde386c27a9228476834e&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=L7iKlxwUA3exA-ByaKl7gyvQkoOevQwuEjv4ZKC6hOY&e=>
>>
>>
>>
>> I'm not asking for the entire rule, just an example of how var counters
>> work in the global scope (directly in /etc/modsecurity/modsecurity.conf)
>> and how can I connect them to sum by header instead of IP.
>>
>>
>>
>> Thank you!
>>
>>
>>
>> El jue., 6 de dic. de 2018 a la(s) 10:30, Christian Folini (
>> [email protected]) escribió:
>>
>> Hello Luciano,
>>
>> You have a peculiar use case, but I see your thinking.
>>
>> There are examples in the ModSecurity books that are really close to your
>> plan. They should be easy to adopt.
>>
>> Other than that, you may want to look into mod_qos. It has functionality
>> that might be useful in your case.
>>
>> Best,
>>
>> Christian
>>
>>
>> On Wed, Dec 05, 2018 at 06:26:03PM -0300, Luciano Guillermo Fantuzzi
>> wrote:
>> > Thank you for your answer, but maybe I'm not asking it the right way or
>> > this is not the right place to ask(?).
>> >
>> > I need a Modsecurity rule (I'm using it through Apache) to be able to
>> > control hits from clients with a specific header, like
>> > "facebookexternalhit/1.1".
>> > Ie. to stop some agressive bots hitting too often my webservers and
>> taking
>> > them down eventually. I don't want to block them at all because I need
>> some
>> > of them (like Facebook bot to parse shared content), but I need a way to
>> > tell them "stop, retry in some seconds".
>> >
>> > Thanks.
>> >
>> > El mié., 5 de dic. de 2018 a la(s) 16:16, Reindl Harald (
>> > [email protected]) escribió:
>> >
>> > >
>> > >
>> > > Am 05.12.18 um 16:57 schrieb Luciano Guillermo Fantuzzi:
>> > > > First of all, I'm new here so I'm not sure this is the right place
>> for
>> > > > asking for help (free modsec version). If it's not, I'll really
>> > > > appreciate it if you can tell me where should I go.
>> > > >
>> > > > I'm trying to limit hit rate by:
>> > > >
>> > > > 1. Request's header (like "facebookexternalhit").
>> > > > 2. (All hits to non static resources)
>> > > >
>> > > > And then return a friendly "429 Too Many Requests" and
>> "Retry-After: 3"
>> > > > (seconds).
>> > > > I know I can read a file of headers like:
>> > > >
>> > > > SecRule REQUEST_HEADERS:User-Agent "@pmFromFile ratelimit-bots.txt"
>> > > >
>> > > > But I'm getting trouble building the entire rule.
>> > > >
>> > > > Any help would be really appreciated. Thank you!
>> > >
>> > > this a non-iusse
>> > >
>> > > normally you have rate-limits per IP in place and they should not be
>> > > within the application layer at all and in the best case not even on
>> the
>> > > same machine
>> > >
>> > > that below is from a firewall-vm on a complete /24 network before any
>> > > packet reaches a server at all, and for the individual servers are
>> > > simimlar rules with lower values per 2 seconds in place
>> > >
>> > > when the request reachs the webserver damage is long done and if no
>> > > damage is done you are wasting expensive ressources with the rules
>> > >
>> > > Chain INBOUND (2 references)
>> > >  pkts bytes target     prot opt in     out     source
>> > > destination
>> > >  1914  183K IPST_ALL   all  --  *      *       0.0.0.0/0
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>> > > 0.0.0.0/0
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>>           recent: UPDATE seconds: 2 hit_count: 250 TTL-Match
>> > > name: limit_all_global side: source mask: 255.255.255.255
>> > >  149K   15M DROP_ALL   all  --  *      *       0.0.0.0/0
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>> > > 0.0.0.0/0
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>>           recent: UPDATE seconds: 2 reap hit_count: 150
>> > > TTL-Match name: limit_all_global side: source mask: 255.255.255.255
>> > >
>> > >
>> > > _______________________________________________
>> > > mod-security-users mailing list
>> > > [email protected]
>> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
>> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> > > http://www.modsecurity.org/projects/commercial/rules/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
>> > > http://www.modsecurity.org/projects/commercial/support/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
>> > >
>>
>>
>> > _______________________________________________
>> > mod-security-users mailing list
>> > [email protected]
>> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
>> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> > http://www.modsecurity.org/projects/commercial/rules/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
>> > http://www.modsecurity.org/projects/commercial/support/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
>>
>>
>>
>> _______________________________________________
>> mod-security-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
>> http://www.modsecurity.org/projects/commercial/support/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
>>
>> _______________________________________________
>> mod-security-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/mod-security-users
>> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
>> http://www.modsecurity.org/projects/commercial/rules/
>> http://www.modsecurity.org/projects/commercial/support/
>>
> _______________________________________________
> mod-security-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

_______________________________________________
mod-security-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.