RE: [INTERNALS-WIN] Question about using UTF-8 encoding for server variables used by PHP

I have some concerns about putting the raw URL into the REQUEST_URI variable.  If applications are using data from that value to open files, there is a potential for security vulnerabilities.  The problem can occur because IIS looks up configuration information using the canonicalized version of the URL for the request.  This process of canonicalization includes decoding the URL, but also some other things.  For example, the kernel mode HTTP parser underneath IIS does the work of deciding the code page of the incoming URL and then hands IIS a Unicode version of the URL for processing.  If an application starts with the raw URL and does its own decoding, it could potentially use a different code page form what the HTTP parser used.  If the application then opens a file, it could be using a path that does not match the IIS configuration for the request.  Since some of the configuration can affect security, problems can be introduced.  The most common security bugs that arise from this problem are bypassing authentication or authorization rules, or serving script source code back to the client because a clever URL managed to trick the application into opening a script file meant for another script handler.

In general, we would be happy to do the right thing for PHP so that support for non-ASCII URLs and filenames works well.  It looks like we could potentially get there by ensuring that SCRIPT_FILENAME is correct.  Unfortunately, I understand that not all of the file I/O in PHP uses Unicode APIs.  This is not a problem if the URL is sent using a code page that matches the server's system because our default is to populate all of the server variables using the system's default code page.  We've introduced an option in the FastCGI 1.5 handler that we just released to encode any or all of the server variables using UTF-8 encoding.  This is effective at preserving the URL for the application, but the non-Unicode file APIs in Windows do not work with UTF-8.  That means that PHP would have to convert the paths to Unicode and use only Unicode APIs to access the file system.

I would be interested in hearing your thoughts on these issues.

Thanks,
-Wade

 
-----Original Message-----
From: Johannes Schlüter [mailto:[email protected]] 
Sent: Wednesday, January 20, 2010 7:39 AM
To: Pierre Joye
Cc: Ruslan Yakushev; Stanislav Malyshev; [email protected]
Subject: Re: [INTERNALS-WIN] Question about using UTF-8 encoding for server variables used by PHP

At least REQUEST_URI should still be in the urlencoded form received
from the client. SCRIPT_FILENAME should be be filesystem encoding (it's
used exactly in that form to open the file ...) PATH_TRANSLATED probably
might be urlencoded, not sure there, maybe the RFC has some information
on this.

further encoding support is hard as the HTTP protocol doesn'tcare
encoding information for the request header and url encoding is a simple
translation of the byte values and an be applied to all binary data ...

johannes

On Tue, 2010-01-19 at 15:11 +0100, Pierre Joye wrote:
> hi Ruslan,
> 
> Btw, what are the specs you use as reference about URL encoding and
> FCGI environment variable encoding, to consider this behavior as
> correct? As far as I remember it is the role of the application to do
> this conversion, not the web server itself. I wonder if it is actually
> a good idea to do it in IIS directly.
> 
> Cheers,
> 
> 2010/1/9 Ruslan Yakushev <[email protected]>:
> > Thanks Stas and Pierre for your comments! Based on this feedback we are planning to make this change only for REQUEST_URI and for PATH_INFO and make it configurable so that if it breaks anything then users can switch back to existing behavior.
> >
> >> -----Original Message-----
> >> From: Stanislav Malyshev [mailto:[email protected]]
> >> Sent: Friday, January 08, 2010 4:04 PM
> >> To: Ruslan Yakushev
> >> Cc: 'Pierre Joye'; '[email protected]'
> >> Subject: Re: [INTERNALS-WIN] Question about using UTF-8 encoding for
> >> server variables used by PHP
> >>
> >> Hi!
> >>
> >> > If url contains non-ascii characters, e.g.
> >> > http://mysite.com/2009/10/русские-символ, then this URL results in
> >> > 404 in WordPress. WordPress uses REQUEST_URI server variable to get
> >> > the originally requested URL. Over HTTP the non-english characters
> >> > come as URL encoded, but IIS internally converts them to unicode.
> >> > Then FastCGI module converts it back to ANSI before passing them to
> >> > PHP.
> >>
> >> I think having REQUEST_URI in some salvageable form is better than
> >> getting ???'s. Also, REQUEST_URI probably won't be used by the engine
> >> directly for filename access, though may be used by application - like
> >> MVC frameworks which route actions depending on REQUEST_URL.
> >> IIRC PHP uses only SCRIPT_FILENAME and PATH_TRANSLATED from the request
> >> for filesystem access, the rest is stored but not used to find the
> >> script.
> >> --
> >> Stanislav Malyshev, Zend Software Architect
> >> [email protected]   http://www.zend.com/
> >> (408)253-8829   MSN: [email protected]
> >>
> >> --
> >> Windows Internals Mailing List (http://www.php.net/)
> >> To unsubscribe, visit: http://www.php.net/unsub.php
> >>
> >
> >
> 
> 
> 



-- 
Windows Internals Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php