Re: How to limit access rate by header?

Already tried that, but still the same message in debug log. Anyway, I'm
behind Cloudflare so I need to access that var (that contains the real IP)
from header. Moreover, I tried with global collection and same luck. Am I
missing some initialization step in modsec?

Thanks.

El vie., 7 dic. 2018 00:56, Scheblein, Adam <[email protected]>
escribió:

> I had a similar problem.  You need to initialize the collection with
> something like this:
>
>
>
>                 SecAction
> id:'2000000',phase:1,nolog,pass,initcol:IP=%{REMOTE_ADDR}
>
>
>
> *From: *Luciano Guillermo Fantuzzi <[email protected]>
> *Reply-To: *"[email protected]" <
> [email protected]>
> *Date: *Thursday, December 6, 2018 at 8:51 PM
> *To: *"[email protected]" <
> [email protected]>
> *Subject: *Re: [mod-security-users] How to limit access rate by header?
>
>
>
> I've very frustrated... I can't make it work, even for IP control. What am
> I doing wrong here? It always returns:
>
> Could not set variable "IP.access_count" as the collection does not exist.
>
>
>
> 105 <LocationMatch "^/.*">
>
> 109   SecRule REQUEST_HEADERS:CF-Connecting-IP "@unconditionalMatch"
> "phase:2,initcol:IP=%{MATCHED_VAR},pass,nolog,id:35003"
>
> 112   SecRule IP:ACCESS_COUNT "@gt 1"
> "phase:2,pause:300,deny,status:503,setenv:RATELIMITED,skip:1,nolog,id:35004"
>
> 116   SecAction "phase:2,setvar:IP.access_count=+1,pass,nolog,id:35005"
>
>
>
> 119   SecAction
> "phase:5,deprecatevar:IP.access_count=1/10,pass,nolog,id:35006"
>
> 122   Header always set Retry-After "10" env=RATELIMITED
>
> 123 </LocationMatch>
>
> 124
>
> 125 ErrorDocument 503 "Service Unavailable"
>
>
>
>
>
> El jue., 6 de dic. de 2018 a la(s) 20:38, Luciano Guillermo Fantuzzi (
> [email protected]) escribió:
>
> Thank you for your answer, Christian. Do you think it's possible for you
> to just build the first part of the rule (in Modsec)? I'm trying but I'm
> not understanding how variables work with the global scope. I was be able
> to build some basic rules like:
>
>
>
> # Banned Bots and Crawlers
>
> SecRule REQUEST_HEADERS:User-Agent "@pmFromFile blacklist-bots.data" \
>
>     "id:350001,phase:1,t:none,deny,log,msg:'BANNED BOT'"
>
>
>
> # Specific IPs
>
> SecRule REMOTE_ADDR "@pmFromFile blacklist-ip.data" \
>
>     "id:350002,phase:1,t:none,deny,log,msg:'BANNED IP'"
>
>
>
> I'm trying to understand examples from stackoverflow and different places,
> but they are all intended to limit by IP and for specific resources (the
> scope of the rule). Eg.:
>
> https://gist.github.com/josnidhin/91d1ea9cd71fde386c27a9228476834e
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_josnidhin_91d1ea9cd71fde386c27a9228476834e&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=L7iKlxwUA3exA-ByaKl7gyvQkoOevQwuEjv4ZKC6hOY&e=>
>
>
>
> I'm not asking for the entire rule, just an example of how var counters
> work in the global scope (directly in /etc/modsecurity/modsecurity.conf)
> and how can I connect them to sum by header instead of IP.
>
>
>
> Thank you!
>
>
>
> El jue., 6 de dic. de 2018 a la(s) 10:30, Christian Folini (
> [email protected]) escribió:
>
> Hello Luciano,
>
> You have a peculiar use case, but I see your thinking.
>
> There are examples in the ModSecurity books that are really close to your
> plan. They should be easy to adopt.
>
> Other than that, you may want to look into mod_qos. It has functionality
> that might be useful in your case.
>
> Best,
>
> Christian
>
>
> On Wed, Dec 05, 2018 at 06:26:03PM -0300, Luciano Guillermo Fantuzzi wrote:
> > Thank you for your answer, but maybe I'm not asking it the right way or
> > this is not the right place to ask(?).
> >
> > I need a Modsecurity rule (I'm using it through Apache) to be able to
> > control hits from clients with a specific header, like
> > "facebookexternalhit/1.1".
> > Ie. to stop some agressive bots hitting too often my webservers and
> taking
> > them down eventually. I don't want to block them at all because I need
> some
> > of them (like Facebook bot to parse shared content), but I need a way to
> > tell them "stop, retry in some seconds".
> >
> > Thanks.
> >
> > El mié., 5 de dic. de 2018 a la(s) 16:16, Reindl Harald (
> > [email protected]) escribió:
> >
> > >
> > >
> > > Am 05.12.18 um 16:57 schrieb Luciano Guillermo Fantuzzi:
> > > > First of all, I'm new here so I'm not sure this is the right place
> for
> > > > asking for help (free modsec version). If it's not, I'll really
> > > > appreciate it if you can tell me where should I go.
> > > >
> > > > I'm trying to limit hit rate by:
> > > >
> > > > 1. Request's header (like "facebookexternalhit").
> > > > 2. (All hits to non static resources)
> > > >
> > > > And then return a friendly "429 Too Many Requests" and "Retry-After:
> 3"
> > > > (seconds).
> > > > I know I can read a file of headers like:
> > > >
> > > > SecRule REQUEST_HEADERS:User-Agent "@pmFromFile ratelimit-bots.txt"
> > > >
> > > > But I'm getting trouble building the entire rule.
> > > >
> > > > Any help would be really appreciated. Thank you!
> > >
> > > this a non-iusse
> > >
> > > normally you have rate-limits per IP in place and they should not be
> > > within the application layer at all and in the best case not even on
> the
> > > same machine
> > >
> > > that below is from a firewall-vm on a complete /24 network before any
> > > packet reaches a server at all, and for the individual servers are
> > > simimlar rules with lower values per 2 seconds in place
> > >
> > > when the request reachs the webserver damage is long done and if no
> > > damage is done you are wasting expensive ressources with the rules
> > >
> > > Chain INBOUND (2 references)
> > >  pkts bytes target     prot opt in     out     source
> > > destination
> > >  1914  183K IPST_ALL   all  --  *      *       0.0.0.0/0
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
> > > 0.0.0.0/0
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>           recent: UPDATE seconds: 2 hit_count: 250 TTL-Match
> > > name: limit_all_global side: source mask: 255.255.255.255
> > >  149K   15M DROP_ALL   all  --  *      *       0.0.0.0/0
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
> > > 0.0.0.0/0
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=>
>           recent: UPDATE seconds: 2 reap hit_count: 150
> > > TTL-Match name: limit_all_global side: source mask: 255.255.255.255
> > >
> > >
> > > _______________________________________________
> > > mod-security-users mailing list
> > > [email protected]
> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
> > > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > > http://www.modsecurity.org/projects/commercial/rules/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
> > > http://www.modsecurity.org/projects/commercial/support/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
> > >
>
>
> > _______________________________________________
> > mod-security-users mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
> > http://www.modsecurity.org/projects/commercial/support/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
>
>
>
> _______________________________________________
> mod-security-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=>
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=>
> http://www.modsecurity.org/projects/commercial/support/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=>
>
> _______________________________________________
> mod-security-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

_______________________________________________
mod-security-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/