Re: internal vs. external hostname in Request.getHost9)
Ilya Skriblovsky <[email protected]> Fri, 17 Mar 2017 09:52:05 +0000
| Newsgroups | gmane.comp.python.twisted.web |
|---|---|
| Message-ID | <CAOG7vkyU8FHN0NJJtwJARWstb7paaXM2r0X2a1p2ssc7hV3dXQ@mail.gmail.com> |
--===============0304204159246243368== Content-Type: multipart/alternative; boundary=001a11c1695cfdc80b054aea2134 --001a11c1695cfdc80b054aea2134 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable > The code calling request.URLPath(), in a given Resource, or application, is highly unlikely to know whether it wants to honor (x-)forwarded-for. You are right, I haven't thought about it. But I'm in doubt whether trusting X-Forwarded-* by default can damage security if Twisted app is running with naked HTTP(S) port exposed without reverse proxy that handles these headers. There are three headers: 1. X-Forwarded-For specifying original client IP and IPs of proxies 2. X-Forwarded-Host specifying original Host header from the client 3. X-Forwarded-Proto specifying original client's scheme (there is also new-style "Forwarded:" header but it is not widely used yet, AFAIK) X-Forwarded-For definetly can't be trusted if comes from untrusted client client. Fortunately we don't need it at all for generating URLs :) It will be in question when refactoring getClientIP() somewhen later. But can we trust X-Forwarded-Host & X-Forwarded-Proto? From the first glance it isn't a problem since we are using them to display URLs for the same client, so nasty client will get his nasty URLs, that's it. But if app is doing something like storing URL in DB or (more likely) sending an email with a link to another client, this would be an issue. -- ilya =D0=BF=D1=82, 17 =D0=BC=D0=B0=D1=80. 2017 =D0=B3. =D0=B2 11:18, Glyph <glyp= [email protected]>: On Mar 15, 2017, at 1:20 AM, Ilya Skriblovsky <[email protected]> wrote: Ok, so in the sort term you are suggesting to change Request.URLPath Yes. (uppercased method? Hmm) Like I said, not a great interface, overall :-). to use Host header instead of getRequestHostname and to change Klein to use it instead of Request.getHost(), right? Sounds wise and reasonable :) OK, glad you agree :). But I'd like to add one more thing. In order to build correct external URL we need to know is it http or https. Currently URLPath is using Request.isSecure(), but it is not sufficient since Request.isSecure() just checks if backend connection is SSL while encryption is often terminated at a reverse proxy. Can we add "useXForwardedProto=3DFalse" argument to Request.URLPath() and check X-Forwarded-Proto header if it is true? And may be add "useXForwardedHost=3DFalse" too to simplify setting up a reverse pro= xy (with a bold red warning in docstring that it can be set to True only if reverse proxy is correctly configured to drop corresponding client-specified headers). What do you think? I think that for starters, it would make more sense to just fix it to *always* honor forwarded-for and x-forwarded-for headers. The code calling request.URLPath(), in a given Resource, or application, is highly unlikely to know whether it wants to honor (x-)forwarded-for. The code that might know about this sort of configuration would be the thing that constructs the Site object, but I'd be much happier to just get a change that always honors it first, and then figure out how to customize it later. -glyph _______________________________________________ Twisted-web mailing list [email protected] http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web --001a11c1695cfdc80b054aea2134 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr" class=3D"gmail_msg">>=C2=A0<span style= =3D"color:rgb(33,33,33)" class=3D"gmail_msg">The code calling request.URLPa= th(), in a given Resource, or application, is highly unlikely to know wheth= er it wants to honor (x-)forwarded-for.</span></div><div dir=3D"ltr" class= =3D"gmail_msg"><div class=3D"gmail_msg"><span style=3D"color:rgb(33,33,33)"= class=3D"gmail_msg">You are right, I haven't thought about it.</span><= /div><div class=3D"gmail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"= gmail_msg">But I'm in doubt whether trusting X-Forwarded-* by default c= an damage security if Twisted app is running with naked HTTP(S) port expose= d without reverse proxy that handles these headers.</span></div><div class= =3D"gmail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail_msg">Ther= e are three headers:</span></div><div class=3D"gmail_msg"><span style=3D"co= lor:rgb(33,33,33)" class=3D"gmail_msg">1. X-Forwarded-For specifying origin= al client IP and IPs of proxies</span></div><div class=3D"gmail_msg"><span = style=3D"color:rgb(33,33,33)" class=3D"gmail_msg">2. X-Forwarded-Host speci= fying original Host header from the client</span></div><div class=3D"gmail_= msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail_msg">3. X-Forwarded= -Proto specifying original client's scheme</span></div><div class=3D"gm= ail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail_msg">(there is = also new-style "Forwarded:" header but it is not widely used yet,= AFAIK)</span></div><div class=3D"gmail_msg"><span style=3D"color:rgb(33,33= ,33)" class=3D"gmail_msg"><br class=3D"gmail_msg"></span></div><div class= =3D"gmail_msg"><span style=3D"color:rgb(33,33,33)" class=3D"gmail_msg">X-Fo= rwarded-For definetly can't be trusted if comes from untrusted client c= lient. Fortunately we don't need it at all for generating URLs :) It wi= ll be in question when refactoring getClientIP() somewhen later.</span><br = class=3D"gmail_msg"></div><div class=3D"gmail_msg"><span style=3D"color:rgb= (33,33,33)" class=3D"gmail_msg"><br></span></div><div class=3D"gmail_msg"><= span style=3D"color:rgb(33,33,33)" class=3D"gmail_msg">But can we trust=C2= =A0</span><span style=3D"color:rgb(33,33,33)" class=3D"gmail_msg">X-Forward= ed-Host & X-Forwarded-Proto? From the first glance it isn't a probl= em since we are using them to display URLs for the same client, so nasty cl= ient will get his nasty URLs, that's it. But if app is doing something = like storing URL in DB or (more likely) sending an email with a link to ano= ther client, this would be an issue.</span></div><div class=3D"gmail_msg"><= br></div><div class=3D"gmail_msg"><span style=3D"color:rgb(33,33,33)" class= =3D"gmail_msg">-- ilya</span></div></div><br class=3D"gmail_msg"><div class= =3D"gmail_quote gmail_msg"><div dir=3D"ltr" class=3D"gmail_msg">=D0=BF=D1= =82, 17 =D0=BC=D0=B0=D1=80. 2017 =D0=B3. =D0=B2 11:18, Glyph <<a href=3D= "mailto:[email protected]" class=3D"gmail_msg" target=3D"_blank">glyp= [email protected]</a>>:<br class=3D"gmail_msg"></div></div><div class= =3D"gmail_quote gmail_msg"><blockquote class=3D"gmail_quote gmail_msg" styl= e=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div st= yle=3D"word-wrap:break-word" class=3D"gmail_msg"><div class=3D"gmail_msg"><= blockquote type=3D"cite" class=3D"gmail_msg"><div class=3D"gmail_msg">On Ma= r 15, 2017, at 1:20 AM, Ilya Skriblovsky <<a href=3D"mailto:ilyaskriblov= [email protected]" class=3D"gmail_msg" target=3D"_blank">ilyaskriblovsky@gmail.= com</a>> wrote:</div><br class=3D"m_-7434395177431637724m_29958264402532= 95589Apple-interchange-newline gmail_msg"><div class=3D"gmail_msg"><div dir= =3D"ltr" class=3D"gmail_msg">Ok, so in the sort term you are suggesting to = change Request.URLPath</div></div></blockquote><div class=3D"gmail_msg"><br= class=3D"gmail_msg"></div></div></div><div style=3D"word-wrap:break-word" = class=3D"gmail_msg"><div class=3D"gmail_msg"><div class=3D"gmail_msg">Yes.<= /div><br class=3D"gmail_msg"><blockquote type=3D"cite" class=3D"gmail_msg">= <div class=3D"gmail_msg"><div dir=3D"ltr" class=3D"gmail_msg"> (uppercased = method? Hmm)</div></div></blockquote><div class=3D"gmail_msg"><br class=3D"= gmail_msg"></div><div class=3D"gmail_msg">Like I said, not a great interfac= e, overall :-).</div></div></div><div style=3D"word-wrap:break-word" class= =3D"gmail_msg"><div class=3D"gmail_msg"><br class=3D"gmail_msg"><blockquote= type=3D"cite" class=3D"gmail_msg"><div class=3D"gmail_msg"><div dir=3D"ltr= " class=3D"gmail_msg"> to use Host header instead of getRequestHostname and= to change Klein to use it instead of Request.getHost(), right?<div class= =3D"gmail_msg">Sounds wise and reasonable :)</div></div></div></blockquote>= <div class=3D"gmail_msg"><br class=3D"gmail_msg"></div></div></div><div sty= le=3D"word-wrap:break-word" class=3D"gmail_msg"><div class=3D"gmail_msg"><d= iv class=3D"gmail_msg">OK, glad you agree :).</div></div></div><div style= =3D"word-wrap:break-word" class=3D"gmail_msg"><div class=3D"gmail_msg"><br = class=3D"gmail_msg"><blockquote type=3D"cite" class=3D"gmail_msg"><div dir= =3D"ltr" class=3D"gmail_msg"><div class=3D"gmail_msg">But I'd like to a= dd one more thing. In order to build correct external URL we need to know i= s it http or https. Currently URLPath is using Request.isSecure(), but it i= s not sufficient since Request.isSecure() just checks if backend connection= is SSL while encryption is often terminated at a reverse proxy. Can we add= "useXForwardedProto=3DFalse" argument to Request.URLPath() and c= heck X-Forwarded-Proto header if it is true? And may be add "useXForwa= rdedHost=3DFalse" too to simplify setting up a reverse proxy (with a b= old red warning in docstring that it can be set to True only if reverse pro= xy is correctly configured to drop corresponding client-specified headers).= What do you think?</div></div></blockquote><br class=3D"gmail_msg"></div><= /div><div style=3D"word-wrap:break-word" class=3D"gmail_msg"><div class=3D"= gmail_msg"></div><div class=3D"gmail_msg">I think that for starters, it wou= ld make more sense to just fix it to <i class=3D"gmail_msg">always</i> hono= r forwarded-for and x-forwarded-for headers.=C2=A0 The code calling request= .URLPath(), in a given Resource, or application, is highly unlikely to know= whether it wants to honor (x-)forwarded-for.=C2=A0 The code that might kno= w about this sort of configuration would be the thing that constructs the S= ite object, but I'd be much happier to just get a change that always ho= nors it first, and then figure out how to customize it later.</div></div><d= iv style=3D"word-wrap:break-word" class=3D"gmail_msg"><div class=3D"gmail_m= sg"><br class=3D"gmail_msg"></div><div class=3D"gmail_msg">-glyph</div><div= class=3D"gmail_msg"><br class=3D"gmail_msg"></div></div>__________________= _____________________________<br class=3D"gmail_msg"> Twisted-web mailing list<br class=3D"gmail_msg"> <a href=3D"mailto:[email protected]" class=3D"gmail_msg" target= =3D"_blank">[email protected]</a><br class=3D"gmail_msg"> <a href=3D"http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web" r= el=3D"noreferrer" class=3D"gmail_msg" target=3D"_blank">http://twistedmatri= x.com/cgi-bin/mailman/listinfo/twisted-web</a><br class=3D"gmail_msg"> </blockquote></div></div> --001a11c1695cfdc80b054aea2134-- --===============0304204159246243368== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Twisted-web mailing list [email protected] http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web --===============0304204159246243368==--