Re: How to limit access rate by header?

Yes, sorry to try to defend myself. It was not fear to telling the true to
someone suddenly attacking me for no reason. Trying to get help here was
clearly a bad idea. This is not a place for help, at least for newbies with
modsec. Sorry for bothering all of you. Have a nice day.

El dom., 9 dic. 2018 08:16, Franziska Buehler <
[email protected]> escribió:

> Luciano,
>
> It’s not fair to call someone, who publishes several free tutorials,
> invests his free time in open source, and over the years answers tons
> of questions on this mailing list, as arrogant, as someone who solves
> his problems with money.
> And in the end, you tell him he should never write you back again.
> I could imagine that your tone probably disappoints and discourages people.
>
> Best regards,
> Franziska
>
> Am Fr., 7. Dez. 2018 um 16:24 Uhr schrieb Luciano Guillermo Fantuzzi
> <[email protected]>:
> >
> > @Christian First of all, you are completly wrong. Let me explain:
> >
> > 1. I'm using Cloudflare (free plan) just to save traffic. So no, I'm not
> a rich guy and I'm looking for a solution server side, because since I'm in
> the free plan I can't use their WAF solution.
> > 2. I'm not a guy looking for someone to do my job. Maybe you didn't see
> my last email, where I wrote the rule I created and that is not working for
> some reason. Before spending my time creating an account here and writing
> emails, I googled the entire day and tried different approaches, but no one
> worked. I didn't find a single piece of code doing what I need to do, so I
> tried with different IP rate limit snippets, but after debugging some time
> I'm facing an error that no one else seem to see. This is why I'm stuck
> here.
> > 3. I'm not paying anyone to do something like this. Maybe you are the
> kind of people that solve your problems with money and in the easiest way.
> Sorry, I'm not that kind of people.
> >
> > And last, I don't want the help of arrogant guys like you. I don't know
> the reason you think you can talk to people the way you do, specially when
> they are asking for help. Like if you were born knowing about everything...
> I can only thank I don't know you and I don't have people like you in my
> life. Fortunately, you are part of a minority.
> >
> > Don't write me back again.
> >
> > El vie., 7 de dic. de 2018 a la(s) 09:31, Christian Folini (
> [email protected]) escribió:
> >>
> >> Luciano,
> >>
> >> I understand your troubles. But if you are behind cloudflare and you are
> >> giving cloudflare money to solve your problems, why are you coming here
> >> asking for free support?
> >>
> >> You are facing a special problem and you need a special solution. We
> have
> >> given you pointers and hints but it seems it is not enough, so it is
> >> likely you need to dig deeper and learn more - or you pay somebody
> >> to do that for you.
> >>
> >> What I am not going to do - and I doubt somebody else is willing to do
> that -
> >> is investing 1-2-3 hours into developing a solution for you in my spare
> >> time. And it is very likely this would take 1-2-3 hours for me, so do
> not
> >> expect a quick win.
> >>
> >> Just my 2 cents. Good luck,
> >>
> >> Christian
> >>
> >> On Fri, Dec 07, 2018 at 09:17:45AM -0300, Luciano Guillermo Fantuzzi
> wrote:
> >> > Already tried that, but still the same message in debug log. Anyway,
> I'm
> >> > behind Cloudflare so I need to access that var (that contains the
> real IP)
> >> > from header. Moreover, I tried with global collection and same luck.
> Am I
> >> > missing some initialization step in modsec?
> >> >
> >> > Thanks.
> >> >
> >> > El vie., 7 dic. 2018 00:56, Scheblein, Adam <
> [email protected]>
> >> > escribió:
> >> >
> >> > > I had a similar problem.  You need to initialize the collection with
> >> > > something like this:
> >> > >
> >> > >
> >> > >
> >> > >                 SecAction
> >> > > id:'2000000',phase:1,nolog,pass,initcol:IP=%{REMOTE_ADDR}
> >> > >
> >> > >
> >> > >
> >> > > *From: *Luciano Guillermo Fantuzzi <[email protected]>
> >> > > *Reply-To: *"[email protected]" <
> >> > > [email protected]>
> >> > > *Date: *Thursday, December 6, 2018 at 8:51 PM
> >> > > *To: *"[email protected]" <
> >> > > [email protected]>
> >> > > *Subject: *Re: [mod-security-users] How to limit access rate by
> header?
> >> > >
> >> > >
> >> > >
> >> > > I've very frustrated... I can't make it work, even for IP control.
> What am
> >> > > I doing wrong here? It always returns:
> >> > >
> >> > > Could not set variable "IP.access_count" as the collection does not
> exist.
> >> > >
> >> > >
> >> > >
> >> > > 105 <LocationMatch "^/.*">
> >> > >
> >> > > 109   SecRule REQUEST_HEADERS:CF-Connecting-IP "@unconditionalMatch"
> >> > > "phase:2,initcol:IP=%{MATCHED_VAR},pass,nolog,id:35003"
> >> > >
> >> > > 112   SecRule IP:ACCESS_COUNT "@gt 1"
> >> > >
> "phase:2,pause:300,deny,status:503,setenv:RATELIMITED,skip:1,nolog,id:35004"
> >> > >
> >> > > 116   SecAction
> "phase:2,setvar:IP.access_count=+1,pass,nolog,id:35005"
> >> > >
> >> > >
> >> > >
> >> > > 119   SecAction
> >> > > "phase:5,deprecatevar:IP.access_count=1/10,pass,nolog,id:35006"
> >> > >
> >> > > 122   Header always set Retry-After "10" env=RATELIMITED
> >> > >
> >> > > 123 </LocationMatch>
> >> > >
> >> > > 124
> >> > >
> >> > > 125 ErrorDocument 503 "Service Unavailable"
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > El jue., 6 de dic. de 2018 a la(s) 20:38, Luciano Guillermo
> Fantuzzi (
> >> > > [email protected]) escribió:
> >> > >
> >> > > Thank you for your answer, Christian. Do you think it's possible
> for you
> >> > > to just build the first part of the rule (in Modsec)? I'm trying
> but I'm
> >> > > not understanding how variables work with the global scope. I was
> be able
> >> > > to build some basic rules like:
> >> > >
> >> > >
> >> > >
> >> > > # Banned Bots and Crawlers
> >> > >
> >> > > SecRule REQUEST_HEADERS:User-Agent "@pmFromFile
> blacklist-bots.data" \
> >> > >
> >> > >     "id:350001,phase:1,t:none,deny,log,msg:'BANNED BOT'"
> >> > >
> >> > >
> >> > >
> >> > > # Specific IPs
> >> > >
> >> > > SecRule REMOTE_ADDR "@pmFromFile blacklist-ip.data" \
> >> > >
> >> > >     "id:350002,phase:1,t:none,deny,log,msg:'BANNED IP'"
> >> > >
> >> > >
> >> > >
> >> > > I'm trying to understand examples from stackoverflow and different
> places,
> >> > > but they are all intended to limit by IP and for specific resources
> (the
> >> > > scope of the rule). Eg.:
> >> > >
> >> > > https://gist.github.com/josnidhin/91d1ea9cd71fde386c27a9228476834e
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_josnidhin_91d1ea9cd71fde386c27a9228476834e&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=L7iKlxwUA3exA-ByaKl7gyvQkoOevQwuEjv4ZKC6hOY&e=
> >
> >> > >
> >> > >
> >> > >
> >> > > I'm not asking for the entire rule, just an example of how var
> counters
> >> > > work in the global scope (directly in
> /etc/modsecurity/modsecurity.conf)
> >> > > and how can I connect them to sum by header instead of IP.
> >> > >
> >> > >
> >> > >
> >> > > Thank you!
> >> > >
> >> > >
> >> > >
> >> > > El jue., 6 de dic. de 2018 a la(s) 10:30, Christian Folini (
> >> > > [email protected]) escribió:
> >> > >
> >> > > Hello Luciano,
> >> > >
> >> > > You have a peculiar use case, but I see your thinking.
> >> > >
> >> > > There are examples in the ModSecurity books that are really close
> to your
> >> > > plan. They should be easy to adopt.
> >> > >
> >> > > Other than that, you may want to look into mod_qos. It has
> functionality
> >> > > that might be useful in your case.
> >> > >
> >> > > Best,
> >> > >
> >> > > Christian
> >> > >
> >> > >
> >> > > On Wed, Dec 05, 2018 at 06:26:03PM -0300, Luciano Guillermo
> Fantuzzi wrote:
> >> > > > Thank you for your answer, but maybe I'm not asking it the right
> way or
> >> > > > this is not the right place to ask(?).
> >> > > >
> >> > > > I need a Modsecurity rule (I'm using it through Apache) to be
> able to
> >> > > > control hits from clients with a specific header, like
> >> > > > "facebookexternalhit/1.1".
> >> > > > Ie. to stop some agressive bots hitting too often my webservers
> and
> >> > > taking
> >> > > > them down eventually. I don't want to block them at all because I
> need
> >> > > some
> >> > > > of them (like Facebook bot to parse shared content), but I need a
> way to
> >> > > > tell them "stop, retry in some seconds".
> >> > > >
> >> > > > Thanks.
> >> > > >
> >> > > > El mié., 5 de dic. de 2018 a la(s) 16:16, Reindl Harald (
> >> > > > [email protected]) escribió:
> >> > > >
> >> > > > >
> >> > > > >
> >> > > > > Am 05.12.18 um 16:57 schrieb Luciano Guillermo Fantuzzi:
> >> > > > > > First of all, I'm new here so I'm not sure this is the right
> place
> >> > > for
> >> > > > > > asking for help (free modsec version). If it's not, I'll
> really
> >> > > > > > appreciate it if you can tell me where should I go.
> >> > > > > >
> >> > > > > > I'm trying to limit hit rate by:
> >> > > > > >
> >> > > > > > 1. Request's header (like "facebookexternalhit").
> >> > > > > > 2. (All hits to non static resources)
> >> > > > > >
> >> > > > > > And then return a friendly "429 Too Many Requests" and
> "Retry-After:
> >> > > 3"
> >> > > > > > (seconds).
> >> > > > > > I know I can read a file of headers like:
> >> > > > > >
> >> > > > > > SecRule REQUEST_HEADERS:User-Agent "@pmFromFile
> ratelimit-bots.txt"
> >> > > > > >
> >> > > > > > But I'm getting trouble building the entire rule.
> >> > > > > >
> >> > > > > > Any help would be really appreciated. Thank you!
> >> > > > >
> >> > > > > this a non-iusse
> >> > > > >
> >> > > > > normally you have rate-limits per IP in place and they should
> not be
> >> > > > > within the application layer at all and in the best case not
> even on
> >> > > the
> >> > > > > same machine
> >> > > > >
> >> > > > > that below is from a firewall-vm on a complete /24 network
> before any
> >> > > > > packet reaches a server at all, and for the individual servers
> are
> >> > > > > simimlar rules with lower values per 2 seconds in place
> >> > > > >
> >> > > > > when the request reachs the webserver damage is long done and
> if no
> >> > > > > damage is done you are wasting expensive ressources with the
> rules
> >> > > > >
> >> > > > > Chain INBOUND (2 references)
> >> > > > >  pkts bytes target     prot opt in     out     source
> >> > > > > destination
> >> > > > >  1914  183K IPST_ALL   all  --  *      *       0.0.0.0/0
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=
> >
> >> > > > > 0.0.0.0/0
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=
> >
> >> > >           recent: UPDATE seconds: 2 hit_count: 250 TTL-Match
> >> > > > > name: limit_all_global side: source mask: 255.255.255.255
> >> > > > >  149K   15M DROP_ALL   all  --  *      *       0.0.0.0/0
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=
> >
> >> > > > > 0.0.0.0/0
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__0.0.0.0_0&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=S69x5cd6GIukj5xdZEQNNUnYwCrOIQklblT0zUw7IVM&e=
> >
> >> > >           recent: UPDATE seconds: 2 reap hit_count: 150
> >> > > > > TTL-Match name: limit_all_global side: source mask:
> 255.255.255.255
> >> > > > >
> >> > > > >
> >> > > > > _______________________________________________
> >> > > > > mod-security-users mailing list
> >> > > > > [email protected]
> >> > > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=
> >
> >> > > > > Commercial ModSecurity Rules and Support from Trustwave's
> SpiderLabs:
> >> > > > > http://www.modsecurity.org/projects/commercial/rules/
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=
> >
> >> > > > > http://www.modsecurity.org/projects/commercial/support/
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=
> >
> >> > > > >
> >> > >
> >> > >
> >> > > > _______________________________________________
> >> > > > mod-security-users mailing list
> >> > > > [email protected]
> >> > > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=
> >
> >> > > > Commercial ModSecurity Rules and Support from Trustwave's
> SpiderLabs:
> >> > > > http://www.modsecurity.org/projects/commercial/rules/
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=
> >
> >> > > > http://www.modsecurity.org/projects/commercial/support/
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=
> >
> >> > >
> >> > >
> >> > >
> >> > > _______________________________________________
> >> > > mod-security-users mailing list
> >> > > [email protected]
> >> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_mod-2Dsecurity-2Dusers&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=RUDsPA0iq5WVkW20NWQOl8suSJ4RvNfYZ6TM3FXNtdM&e=
> >
> >> > > Commercial ModSecurity Rules and Support from Trustwave's
> SpiderLabs:
> >> > > http://www.modsecurity.org/projects/commercial/rules/
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_rules_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=rQF299h7fZhdJbhudnhzjEcP4e3Aa8qCG0KvKi4CKiM&e=
> >
> >> > > http://www.modsecurity.org/projects/commercial/support/
> >> > > <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.modsecurity.org_projects_commercial_support_&d=DwMFaQ&c=S1d2Gs1Y1NQV8Lx35_Qi5FnTH2uYWyh_OhOS94IqYCo&r=E28NzkfUnnOxyipWbMmVvps8QnGe_19SJDNMcPTyffU&m=0DeHAXm5x7u_63IG4vvHEiJ7cWQqPlE3mjexyj6AoOY&s=ofF4OfFPsr3nKEMOH7j-CQmBqLgK_51fvOnQTavYK3c&e=
> >
> >> > >
> >> > > _______________________________________________
> >> > > mod-security-users mailing list
> >> > > [email protected]
> >> > > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >> > > Commercial ModSecurity Rules and Support from Trustwave's
> SpiderLabs:
> >> > > http://www.modsecurity.org/projects/commercial/rules/
> >> > > http://www.modsecurity.org/projects/commercial/support/
> >> > >
> >>
> >>
> >> > _______________________________________________
> >> > mod-security-users mailing list
> >> > [email protected]
> >> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> >> > http://www.modsecurity.org/projects/commercial/rules/
> >> > http://www.modsecurity.org/projects/commercial/support/
> >>
> >>
> >>
> >> _______________________________________________
> >> mod-security-users mailing list
> >> [email protected]
> >> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> >> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> >> http://www.modsecurity.org/projects/commercial/rules/
> >> http://www.modsecurity.org/projects/commercial/support/
> >
> > _______________________________________________
> > mod-security-users mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/mod-security-users
> > Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> > http://www.modsecurity.org/projects/commercial/rules/
> > http://www.modsecurity.org/projects/commercial/support/
>
>
> _______________________________________________
> mod-security-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/mod-security-users
> Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
> http://www.modsecurity.org/projects/commercial/rules/
> http://www.modsecurity.org/projects/commercial/support/
>

_______________________________________________
mod-security-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/