Re: WSGI 2.0 Round 2: requirements and call for interest
Graham Dumpleton <[email protected]> Wed, 6 Jan 2016 09:19:15 +1100
| Newsgroups | gmane.comp.python.web |
|---|---|
| Message-ID | <[email protected]> |
--===============2037259603723955753== Content-Type: multipart/alternative; boundary="Apple-Mail=_6FF9F962-6446-4671-AA6C-19E176D5B532" --Apple-Mail=_6FF9F962-6446-4671-AA6C-19E176D5B532 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 6 Jan 2016, at 12:09 AM, [email protected] wrote: >=20 > As someone who writes their WSGI applications as functions that take > `start_response` and `environ` and doesn't bother with much > framework the things I would like to see in a minor revision to WSGI > are: >=20 > * A consistent way to access the raw un-decoded request URI. This is > so I can reconstruct a realistic `PATH_INFO` that has not been > subjected to destructive handling by the server (e.g. apache > messing with `%2F`) before continuing on to a route dispatcher. This is already available in some servers by way of the REQUEST_URI = value. This is the original first line of any HTTP request and can be split = apart to get the path. The problem is that you cannot easily use it unless you want to = replicate normalisations that the underlying server may do. The key problem is working out where SCRIPT_NAME ends and PATH_INFO = starts with the original path given in REQUEST_URI. Sure if you only deal with a web application mounted at the root of the = host it is easier because SCRIPT_NAME would be empty, but when mounted = at a sub URL it gets trickier. This is because a web server will eliminate things like repeating = slashes in the part of the path that may match the mount point (sub url) = for the web application. The sub url here could be dictated by what is = defined in a configuration file, or could instead be due to matching = against a file system path. Further, the web server will eliminate attempts at relative directory = traversal using =E2=80=98..=E2=80=99 and =E2=80=98.=E2=80=99. So an original path may be something like: /a/b//c/../d/.//e/../f/g/h If the mount point was =E2=80=98/a/b/d=E2=80=99, then that is what gets = passed through SCRIPT_NAME. Now if you instead go to the raw path you would need to replicate all = the normalisations. Only then could you maybe based on length of = SCRIPT_NAME, number of component parts, or actual components in the = path, try and calculate where SCRIPT_NAME ended and PATH_INFO started in = the raw path. This will still all fail if a web server does internal rewrites though, = as the final SCRIPT_NAME may not even match the raw path, although at = that point URL reconstruction can be a problem as well if what the = application is given by way of the rewrite isn=E2=80=99t a public path. I have only looked at SCRIPT_NAME. Even in PATH_INFO servers will apply = same sort of normalisations. So even this isn=E2=80=99t so simple to do properly if you want to go = back and do it yourself using the raw path. I have never seen anyone trying to extract repeating slashes intact out = of a raw path even attempt to do it properly. They tend to assume that = the raw path is pure and doesn=E2=80=99t have stuff in it which needs to = be normalised and that rewrites aren=E2=80=99t occurring. As a result = they assume that they can just strip number of characters off raw path = based on length of SCRIPT_NAME passed through. This will be fragile = though if the raw path isn=E2=80=99t pure. Graham= --Apple-Mail=_6FF9F962-6446-4671-AA6C-19E176D5B532 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><br class=3D""><div><blockquote type=3D"cite" class=3D""><div = class=3D"">On 6 Jan 2016, at 12:09 AM, <a = href=3D"mailto:[email protected]" class=3D"">[email protected]</a> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><span = style=3D"font-family: Helvetica; font-size: 16px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; float: none; display: inline = !important;" class=3D"">As someone who writes their WSGI applications as = functions that take</span><br style=3D"font-family: Helvetica; = font-size: 16px; font-style: normal; font-variant: normal; font-weight: = normal; letter-spacing: normal; orphans: auto; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; widows: = auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><span= style=3D"font-family: Helvetica; font-size: 16px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; float: none; display: inline = !important;" class=3D"">`start_response` and `environ` and doesn't = bother with much</span><br style=3D"font-family: Helvetica; font-size: = 16px; font-style: normal; font-variant: normal; font-weight: normal; = letter-spacing: normal; orphans: auto; text-align: start; text-indent: = 0px; text-transform: none; white-space: normal; widows: auto; = word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><span = style=3D"font-family: Helvetica; font-size: 16px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; float: none; display: inline = !important;" class=3D"">framework the things I would like to see in a = minor revision to WSGI</span><br style=3D"font-family: Helvetica; = font-size: 16px; font-style: normal; font-variant: normal; font-weight: = normal; letter-spacing: normal; orphans: auto; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; widows: = auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><span= style=3D"font-family: Helvetica; font-size: 16px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; float: none; display: inline = !important;" class=3D"">are:</span><br style=3D"font-family: Helvetica; = font-size: 16px; font-style: normal; font-variant: normal; font-weight: = normal; letter-spacing: normal; orphans: auto; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; widows: = auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><br = style=3D"font-family: Helvetica; font-size: 16px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px;" class=3D""><span style=3D"font-family: = Helvetica; font-size: 16px; font-style: normal; font-variant: normal; = font-weight: normal; letter-spacing: normal; orphans: auto; text-align: = start; text-indent: 0px; text-transform: none; white-space: normal; = widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: = none; display: inline !important;" class=3D"">* A consistent way to = access the raw un-decoded request URI. This is</span><br = style=3D"font-family: Helvetica; font-size: 16px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px;" class=3D""><span style=3D"font-family: = Helvetica; font-size: 16px; font-style: normal; font-variant: normal; = font-weight: normal; letter-spacing: normal; orphans: auto; text-align: = start; text-indent: 0px; text-transform: none; white-space: normal; = widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: = none; display: inline !important;" class=3D""> so I can reconstruct = a realistic `PATH_INFO` that has not been</span><br style=3D"font-family: = Helvetica; font-size: 16px; font-style: normal; font-variant: normal; = font-weight: normal; letter-spacing: normal; orphans: auto; text-align: = start; text-indent: 0px; text-transform: none; white-space: normal; = widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" = class=3D""><span style=3D"font-family: Helvetica; font-size: 16px; = font-style: normal; font-variant: normal; font-weight: normal; = letter-spacing: normal; orphans: auto; text-align: start; text-indent: = 0px; text-transform: none; white-space: normal; widows: auto; = word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: = inline !important;" class=3D""> subjected to destructive handling = by the server (e.g. apache</span><br style=3D"font-family: Helvetica; = font-size: 16px; font-style: normal; font-variant: normal; font-weight: = normal; letter-spacing: normal; orphans: auto; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; widows: = auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><span= style=3D"font-family: Helvetica; font-size: 16px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; float: none; display: inline = !important;" class=3D""> messing with `%2F`) before continuing on = to a route dispatcher.</span></div></blockquote></div><br class=3D""><div = class=3D"">This is already available in some servers by way of the = REQUEST_URI value.</div><div class=3D""><br class=3D""></div><div = class=3D"">This is the original first line of any HTTP request and can = be split apart to get the path.</div><div class=3D""><br = class=3D""></div><div class=3D"">The problem is that you cannot easily = use it unless you want to replicate normalisations that the underlying = server may do.</div><div class=3D""><br class=3D""></div><div = class=3D"">The key problem is working out where SCRIPT_NAME ends and = PATH_INFO starts with the original path given in REQUEST_URI.</div><div = class=3D""><br class=3D""></div><div class=3D"">Sure if you only deal = with a web application mounted at the root of the host it is easier = because SCRIPT_NAME would be empty, but when mounted at a sub URL it = gets trickier.</div><div class=3D""><br class=3D""></div><div = class=3D"">This is because a web server will eliminate things like = repeating slashes in the part of the path that may match the mount point = (sub url) for the web application. The sub url here could be dictated by = what is defined in a configuration file, or could instead be due to = matching against a file system path.</div><div class=3D""><br = class=3D""></div><div class=3D"">Further, the web server will eliminate = attempts at relative directory traversal using =E2=80=98..=E2=80=99 and = =E2=80=98.=E2=80=99.</div><div class=3D""><br class=3D""></div><div = class=3D"">So an original path may be something like:</div><div = class=3D""><br class=3D""></div><div class=3D""> = /a/b//c/../d/.//e/../f/g/h</div><div class=3D""><br class=3D""></div><div = class=3D"">If the mount point was =E2=80=98/a/b/d=E2=80=99, then that is = what gets passed through SCRIPT_NAME.</div><div class=3D""><br = class=3D""></div><div class=3D"">Now if you instead go to the raw path = you would need to replicate all the normalisations. Only then could you = maybe based on length of SCRIPT_NAME, number of component parts, or = actual components in the path, try and calculate where SCRIPT_NAME ended = and PATH_INFO started in the raw path.</div><div class=3D""><br = class=3D""></div><div class=3D"">This will still all fail if a web = server does internal rewrites though, as the final SCRIPT_NAME may not = even match the raw path, although at that point URL reconstruction can = be a problem as well if what the application is given by way of the = rewrite isn=E2=80=99t a public path.</div><div class=3D""><br = class=3D""></div><div class=3D"">I have only looked at SCRIPT_NAME. Even = in PATH_INFO servers will apply same sort of normalisations.</div><div = class=3D""><br class=3D""></div><div class=3D"">So even this isn=E2=80=99t= so simple to do properly if you want to go back and do it yourself = using the raw path.</div><div class=3D""><br class=3D""></div><div = class=3D"">I have never seen anyone trying to extract repeating slashes = intact out of a raw path even attempt to do it properly. They tend to = assume that the raw path is pure and doesn=E2=80=99t have stuff in it = which needs to be normalised and that rewrites aren=E2=80=99t occurring. = As a result they assume that they can just strip number of characters = off raw path based on length of SCRIPT_NAME passed through. This will be = fragile though if the raw path isn=E2=80=99t pure.</div><div = class=3D""><br class=3D""></div><div class=3D"">Graham</div></body></html>= --Apple-Mail=_6FF9F962-6446-4671-AA6C-19E176D5B532-- --===============2037259603723955753== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Web-SIG mailing list [email protected] Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org --===============2037259603723955753==--