RE: [INTERNALS-WIN] Question about using UTF-8 encoding for server variables used by PHP

Hi Pierre,

> 
> Not only PHP, I was refering to (F)CGI as well here.
> 
> As it could make sense (still not sure it is a good thing to do) to
> use Unicode (or whatever other encoding IIS considers as the way to
> go) in the IIS+asp.net combination, the idea behind the two being a
> kind of full application server, even if it gets modularized in
> version 7 (very good thing btw).
> 
> I can imagine it would be possible to do it if PHP was still using the
> ISAPI instead of FastCGI, as we could access the internal IIS or
> http.sys API directly. We could get the physical path of the file
> directly as well using IHttpContext::GetPhysicalPath, or using other
> valuable APIs. But we use FastCGI, there is no linking between IIS and
> PHP and there are standard (more or less well documented :) for it.
> 
> Please note that for (F)CGI some variables should not be decoded
> (never), as specified here
> http://hoohoo.ncsa.illinois.edu/cgi/env.html
> 

The referenced spec is for CGI and predates FastCGI.  It mentions decoding in two cases: PATH_INFO and QUERY_STRING.  In the case of PATH_INFO, it says that the value *should* be coded by the server if it comes from a URL.  While it is not stated specifically, this sort of implies to me that URL data should be decoded by the server.  In the case of QUERY_STRING, it says that the value should *never* be decoded by the server.  IIS is compliant with this, as it has never decoded the QUERY_STRING server variable.  Note that it's possible for code outside of IIS to modify this behavior.  For example, I believe that the URL Rewriter will, in some cases, decode the query string before it writes the new URL.

As for getting the physical path to the script, IIS provides this information in the SCRIPT_TRANSLATED server variable, which we do make available via FastCGI.  Note that IIS also provides the raw URL information via the UNENCODED_URL server variable, but I would strongly recommend against using this value to derive any string that will be passed to a file system API.

> 
> I would like to discuss the security risk more closely as I'm not
> aware of any open ones to date. If you have references to past flaws
> related to FastCGI and encoding, I would also like to get an eye on
> them. Please drop me a mail to my msft addy so we can discuss it off
> list if necessary (not willing to end the discussions here but
> security issues should not be discussed here).
> 

There have not been any reported vulnerabilities specifically involving FastCGI and encoding.  There have been reported vulnerabilities using other interfaces and encoding.  My desire is to prevent any new vulnerabilities.  As for providing a list of references, that is a bit difficult to do.  We took a hard line on the decoding issue back in 2003 and there have been few reported issues since then and none that could not be mitigated.  Searching for issues older than 2003 is possible, but not a trivial work item.

>
> However whether we use the unicode API or not does not solve the
> problem about why IIS should suddenly use Unicode to pass the
> environments to the fastcgi instances. I understand why you would like
> to do it but I don't see why this change must be done.
> 

IIS is 100% Unicode internally with regard to URLs, but it is not suddenly using Unicode to pass environments.  In fact, it cannot use Unicode for this because FastCGI does not support it.  The environment variables passed via FastCGI are all ANSI.  The question is about encoding.  By default, we pass the server variables using the default system code page.  This works fine as long as the client encoded the URL with that same code page when it sent the request (and it even works if the client sent the URL encoded with UTF-8, since IIS received it from http.sys as a Unicode value.)  If the client uses code page that is not the same as the server's code page, then our defaults will not work.

In the new FastCGI release, we've added an option to have FastCGI send server variables with UTF-8 encoding instead of using the default system code page.  This specifically enables some scenarios in Wordpress that were broken in our previous FastCGI releases (Ruslan can provide more details if needed.)  The option to do this is not all or nothing.  It can be turned on and off on a variable-by-variable basis as needed.

Thanks,
-Wade