Re: internal vs. external hostname in Request.getHost9)
Tom Most <[email protected]> Mon, 20 Mar 2017 11:30:27 -0700
| Newsgroups | gmane.comp.python.twisted.web |
|---|---|
| Message-ID | <CAK4RBD96CqP0vU7tzLf_kme2uO4MfPxfgbkzzUN82DAYdqMiVA@mail.gmail.com> |
--===============7404533629715231459== Content-Type: multipart/alternative; boundary=94eb2c0a3584ea030f054b2db9b7 --94eb2c0a3584ea030f054b2db9b7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Fri, Mar 17, 2017 at 2:52 AM, Ilya Skriblovsky <[email protected]= m > wrote: > > The code calling request.URLPath(), in a given Resource, or > application, is highly unlikely to know whether it wants to honor > (x-)forwarded-for. > You are right, I haven't thought about it. > But I'm in doubt whether trusting X-Forwarded-* by default can damage > security if Twisted app is running with naked HTTP(S) port exposed withou= t > reverse proxy that handles these headers. > There are three headers: > 1. X-Forwarded-For specifying original client IP and IPs of proxies > 2. X-Forwarded-Host specifying original Host header from the client > 3. X-Forwarded-Proto specifying original client's scheme > (there is also new-style "Forwarded:" header but it is not widely used > yet, AFAIK) > > X-Forwarded-For definetly can't be trusted if comes from untrusted client > client. Fortunately we don't need it at all for generating URLs :) It wil= l > be in question when refactoring getClientIP() somewhen later. > > But can we trust X-Forwarded-Host & X-Forwarded-Proto? From the first > glance it isn't a problem since we are using them to display URLs for the > same client, so nasty client will get his nasty URLs, that's it. But if a= pp > is doing something like storing URL in DB or (more likely) sending an ema= il > with a link to another client, this would be an issue. > It's not safe to enable support for X-Forwarded-For and friends by default, since you can't know how application code will use that information and in many configurations it may be spoofed by clients. A proper deployment which sets these headers looks like this: 1. Configure your frontend reverse-proxy (nginx or whatever) to discard incoming X-Forwarded-* headers and *set* X-Forwarded-* as appropriate. Note that X-Forwarded-For is actually a *list* of hops[1], so if you do this naively your server may append to the list! [2] 2. Configure your backend services to respect exactly the headers passed by your frontend. Note that this requires administrative action in two places, and that if you don't do the frontend config it will probably pass the headers through, allowing them to be spoofed. A non-exhaustive list of possible bad stuff I client could use spoofing for= : 1. Evade IP-based access control (a reasonable defense-in-depth measure). 2. Evade pinning of user sessions to IP addresses or subnets. 2. Evade IP-based rate limiting, e.g. as discussed at [3]. It's also probably worth noting that Django used to offer support for X-Forwarded-For and removed it[4]. X-Forwarded-* and friends just too varied in the wild to reasonably support. If Twisted is to support this in any way, I think that it should be opt-in support for the Forwarded header as specified in RFC 7239. This should be a parameter applicable to all of twisted.web.server rather than per-method call, since it's something the administrator needs to set. =E2=80=94Tom [1]: Standardized as Forwarded in https://tools.ietf.org/html/rfc7239 [2]: Your frontend proxy should also validate Host is a domain you control to prevent cookie theft. Plus lots of other stuff. Web security is hard, and every default everywhere sets users up for failure. [3]: http://django-ratelimit.readthedocs.io/en/v1.0.0/security.html [4]: https://www.djangoproject.com/weblog/2009/jul/28/security/# secondary-issue --94eb2c0a3584ea030f054b2db9b7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On F= ri, Mar 17, 2017 at 2:52 AM, Ilya Skriblovsky <span dir=3D"ltr"><<a href= =3D"mailto:[email protected]" target=3D"_blank">ilyaskriblovsky@gma= il.com</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"= margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-lef= t:1ex"><div dir=3D"ltr"><span class=3D"gmail-m_388361609629121005gmail-"><d= iv dir=3D"ltr" class=3D"gmail-m_388361609629121005gmail-m_-5250999540245141= 15gmail_msg">>=C2=A0<span style=3D"color:rgb(33,33,33)" class=3D"gmail-m= _388361609629121005gmail-m_-525099954024514115gmail_msg">The code calling r= equest.URLPath(), in a given Resource, or application, is highly unlikely t= o know whether it wants to honor (x-)forwarded-for.</span></div></span><div= dir=3D"ltr" class=3D"gmail-m_388361609629121005gmail-m_-525099954024514115= gmail_msg"><div class=3D"gmail-m_388361609629121005gmail-m_-525099954024514= 115gmail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail-m_38836160= 9629121005gmail-m_-525099954024514115gmail_msg">You are right, I haven'= t thought about it.</span></div><div class=3D"gmail-m_388361609629121005gma= il-m_-525099954024514115gmail_msg"><span style=3D"color:rgb(33,33,33)" clas= s=3D"gmail-m_388361609629121005gmail-m_-525099954024514115gmail_msg">But I&= #39;m in doubt whether trusting X-Forwarded-* by default can damage securit= y if Twisted app is running with naked HTTP(S) port exposed without reverse= proxy that handles these headers.</span></div><div class=3D"gmail-m_388361= 609629121005gmail-m_-525099954024514115gmail_msg"><span style=3D"color:rgb(= 33,33,33)" class=3D"gmail-m_388361609629121005gmail-m_-525099954024514115gm= ail_msg">There are three headers:</span></div><div class=3D"gmail-m_3883616= 09629121005gmail-m_-525099954024514115gmail_msg"><span style=3D"color:rgb(3= 3,33,33)" class=3D"gmail-m_388361609629121005gmail-m_-525099954024514115gma= il_msg">1. X-Forwarded-For specifying original client IP and IPs of proxies= </span></div><div class=3D"gmail-m_388361609629121005gmail-m_-5250999540245= 14115gmail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail-m_388361= 609629121005gmail-m_-525099954024514115gmail_msg">2. X-Forwarded-Host speci= fying original Host header from the client</span></div><div class=3D"gmail-= m_388361609629121005gmail-m_-525099954024514115gmail_msg"><span style=3D"co= lor:rgb(33,33,33)" class=3D"gmail-m_388361609629121005gmail-m_-525099954024= 514115gmail_msg">3. X-Forwarded-Proto specifying original client's sche= me</span></div><div class=3D"gmail-m_388361609629121005gmail-m_-52509995402= 4514115gmail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail-m_3883= 61609629121005gmail-m_-525099954024514115gmail_msg">(there is also new-styl= e "Forwarded:" header but it is not widely used yet, AFAIK)</span= ></div><div class=3D"gmail-m_388361609629121005gmail-m_-525099954024514115g= mail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail-m_388361609629= 121005gmail-m_-525099954024514115gmail_msg"><br class=3D"gmail-m_3883616096= 29121005gmail-m_-525099954024514115gmail_msg"></span></div><div class=3D"gm= ail-m_388361609629121005gmail-m_-525099954024514115gmail_msg"><span style= =3D"color:rgb(33,33,33)" class=3D"gmail-m_388361609629121005gmail-m_-525099= 954024514115gmail_msg">X-Forwarded-For definetly can't be trusted if co= mes from untrusted client client. Fortunately we don't need it at all f= or generating URLs :) It will be in question when refactoring getClientIP()= somewhen later.</span><br class=3D"gmail-m_388361609629121005gmail-m_-5250= 99954024514115gmail_msg"></div><div class=3D"gmail-m_388361609629121005gmai= l-m_-525099954024514115gmail_msg"><span style=3D"color:rgb(33,33,33)" class= =3D"gmail-m_388361609629121005gmail-m_-525099954024514115gmail_msg"><br></s= pan></div><div class=3D"gmail-m_388361609629121005gmail-m_-5250999540245141= 15gmail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail-m_388361609= 629121005gmail-m_-525099954024514115gmail_msg">But can we trust=C2=A0</span= ><span style=3D"color:rgb(33,33,33)" class=3D"gmail-m_388361609629121005gma= il-m_-525099954024514115gmail_msg">X-Forwarded-Host & X-Forwarded-Proto= ? From the first glance it isn't a problem since we are using them to d= isplay URLs for the same client, so nasty client will get his nasty URLs, t= hat's it. But if app is doing something like storing URL in DB or (more= likely) sending an email with a link to another client, this would be an i= ssue.</span></div></div></div></blockquote><div><br></div><div>It's not= safe to enable support for X-Forwarded-For and friends by default, since y= ou can't know how application code will use that information and in man= y configurations it may be spoofed by clients.<br><br>A proper deployment w= hich sets these headers looks like this:<br><br></div><div>1. Configure you= r frontend reverse-proxy (nginx or whatever) to discard incoming X-Forwarde= d-* headers and <i>set</i> X-Forwarded-* as appropriate. Note that X-Forwar= ded-For is actually a <i>list</i> of hops[1], so if you do this naively you= r server may append to the list! [2]<br></div><div>2. Configure your backen= d services to respect exactly the headers passed by your frontend.<br></div= ><div><br></div><div>Note that this requires administrative action in two p= laces, and that if you don't do the frontend config it will probably pa= ss the headers through, allowing them to be spoofed.<br></div><div><br></di= v><div>A non-exhaustive list of possible bad stuff I client could use spoof= ing for:<br><br></div><div>1. Evade IP-based access control (a reasonable d= efense-in-depth measure).<br></div><div>2. Evade pinning of user sessions t= o IP addresses or subnets.<br></div><div>2. Evade IP-based rate limiting, e= .g. as discussed at [3].<br><br></div><div>It's also probably worth not= ing that Django used to offer support for X-Forwarded-For and removed it[4]= . X-Forwarded-* and friends just too varied in the wild to reasonably supp= ort.<br><br></div><div>If Twisted is to support this in any way, I think th= at it should be opt-in support for the Forwarded header as specified in RFC= 7239. This should be a parameter applicable to all of twisted.web.server r= ather than per-method call, since it's something the administrator need= s to set.<br></div><div><br></div><div>=E2=80=94Tom<br></div><div><br>[1]: = Standardized as Forwarded in <a href=3D"https://tools.ietf.org/html/rfc7239= " target=3D"_blank">https://tools.ietf.org/html/<wbr>rfc7239</a><br></div><= div>[2]: Your frontend proxy should also validate Host is a domain you cont= rol to prevent cookie theft. Plus lots of other stuff. Web security is hard= , and every default everywhere sets users up for failure.<br>[3]: <a href= =3D"http://django-ratelimit.readthedocs.io/en/v1.0.0/security.html" target= =3D"_blank">http://django-ratelimit.<wbr>readthedocs.io/en/v1.0.0/<wbr>secu= rity.html</a><br>[4]: <a href=3D"https://www.djangoproject.com/weblog/2009/= jul/28/security/#secondary-issue" target=3D"_blank">https://www.djangoproje= ct.com/<wbr>weblog/2009/jul/28/security/#<wbr>secondary-issue</a></div></di= v></div></div> --94eb2c0a3584ea030f054b2db9b7-- --===============7404533629715231459== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Twisted-web mailing list [email protected] http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web --===============7404533629715231459==--