Re: [INTERNALS-WIN] Question about using UTF-8 encoding for server variables used by PHP

hi Wade,

2010/1/22 Wade Hilmo <[email protected]>:

> I have some concerns about putting the raw URL into the REQUEST_URI variable.

I think there is a confusion about the problem Ruslan is trying to
solve, manage url like: http://mysite.com/2009/10/русские-символ,
which are encoded as described in the RFC. None of the files actually
use UTF-8, index.php is not suddenly named индекс.php, even for
Wordpress :).

http://mysite.com/2009/10/русские-символ is actually processed by the
rewrite module and will end (example) as:

http://mysite.com/index.php?year=2009&month=10&title=русские-символ

Or whatever else is defined as rewrite rule.

> If applications are using data from that value to open files, there is a potential for security vulnerabilities.  The problem can occur because IIS looks up configuration information using the canonicalized version of the URL for the request.  This process of canonicalization includes decoding the URL, but also some other things.  For example, the kernel mode HTTP parser underneath IIS does the work of deciding the code page of the incoming URL and then hands IIS a Unicode version of the URL for processing.  If an application starts with the raw URL and does its own decoding, it could potentially use a different code page form what the HTTP parser used.  If the application then opens a file, it could be using a path that does not match the IIS configuration for the request.  Since some of the configuration can affect security, problems can be introduced.  The most common security bugs that arise from this problem are bypassing authentication or authorization rules, or serving script source code back to the client because a clever URL managed to trick the application into opening a script file meant for another script handler.
>
> In general, we would be happy to do the right thing for PHP so that support for non-ASCII URLs and filenames works well.  It looks like we could potentially get there by ensuring that SCRIPT_FILENAME is correct.

Exactly, see php-src/sapi/cgi/cgi_main.c, line 1000+, there is a
simple explanation about how php will process the (f)cgi inputs and
the implementation follows this text.

>  Unfortunately, I understand that not all of the file I/O in PHP uses Unicode APIs.

Nothing in PHP uses the Unicode APIs (except the IO_REPARSE
implementation I added in 5.3.1, in TSRM) , and that won't be possible
to change any time soon (5.2.x or 5.3.x).

>  This is not a problem if the URL is sent using a code page that matches the server's system because our default is to populate all of the server variables using the system's default code page.  We've introduced an option in the FastCGI 1.5 handler that we just released to encode any or all of the server variables using UTF-8 encoding.  This is effective at preserving the URL for the application, but the non-Unicode file APIs in Windows do not work with UTF-8.  That means that PHP would have to convert the paths to Unicode and use only Unicode APIs to access the file system.

Why introduce such thing now at this stage? URL are encoded and can be
easily decode in PHP applications, I do that since more than a decade
without issues. It was always the responsability of the application to
deal with that. However as long as this option is disabled by default,
it won't break anything.

> I would be interested in hearing your thoughts on these issues.

Cheers,
-- 
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org