Don't encode ~ in URLs

Hrvoje Niksic <[email protected]>
Newsgroups gmane.comp.web.wget.patches
Message-ID <[email protected]>
Following the recommendation of (now ancient) RFC 1738, Wget treats
"~" as a char unsafe to use in URLs unencoded.  This causes problems
with many broken servers, as witnessed by
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=301624 .

This patch fixes this long-standing problem.  Note that this doesn't
affect treatment of ~ in file names, where ~ remains unencoded.

2005-04-09  Hrvoje Niksic  <[email protected]>

	* url.c: Use "static const" in preference to "const static".
	Sun's cc warns that "storage class after type is obsolescent".

	* url.c (urlchr_table): Don't mark ~ as unsafe, too many broken
	web sites are confused when ~ is changed to %7E.  Their servers
	redirect /%7Efoo/ to /~foo/, which Wget again accesses using %7E,
	causing further redirections, therefore looping infinitely.  See
	Debian bug #301624 for an example.

Index: src/url.c
===================================================================
RCS file: /pack/anoncvs/wget/src/url.c,v
retrieving revision 1.119
diff -u -r1.119 url.c
--- src/url.c	2005/04/01 23:59:48	1.119
+++ src/url.c	2005/04/09 11:29:42
@@ -87,13 +87,14 @@
    changing the meaning of the URL.  For example, you can't decode
    "/foo/%2f/bar" into "/foo///bar" because the number and contents of
    path components is different.  Non-reserved characters can be
-   changed, so "/foo/%78/bar" is safe to change to "/foo/x/bar".  Wget
-   uses the rfc1738 set of reserved characters, plus "$" and ",", as
-   recommended by rfc2396.
-
-   An unsafe characters is the one that should be encoded when URLs
-   are placed in foreign environments.  E.g. space and newline are
-   unsafe in HTTP contexts because HTTP uses them as separator and
+   changed, so "/foo/%78/bar" is safe to change to "/foo/x/bar".  The
+   unsafe characters are loosely based on rfc1738, plus "$" and ",",
+   as recommended by rfc2396, and minus "~", which is very frequently
+   used (and sometimes unrecognized as %7E by broken servers).
+
+   An unsafe character is the one that should be encoded when URLs are
+   placed in foreign environments.  E.g. space and newline are unsafe
+   in HTTP contexts because HTTP uses them as separator and line
    terminator, so they must be encoded to %20 and %0A respectively.
    "*" is unsafe in shell context, etc.
 
@@ -117,7 +118,7 @@
 #define U  urlchr_unsafe
 #define RU R|U
 
-const static unsigned char urlchr_table[256] =
+static const unsigned char urlchr_table[256] =
 {
   U,  U,  U,  U,   U,  U,  U,  U,   /* NUL SOH STX ETX  EOT ENQ ACK BEL */
   U,  U,  U,  U,   U,  U,  U,  U,   /* BS  HT  LF  VT   FF  CR  SO  SI  */
@@ -134,7 +135,7 @@
   U,  0,  0,  0,   0,  0,  0,  0,   /* `   a   b   c    d   e   f   g   */
   0,  0,  0,  0,   0,  0,  0,  0,   /* h   i   j   k    l   m   n   o   */
   0,  0,  0,  0,   0,  0,  0,  0,   /* p   q   r   s    t   u   v   w   */
-  0,  0,  0,  U,   U,  U,  U,  U,   /* x   y   z   {    |   }   ~   DEL */
+  0,  0,  0,  U,   U,  U,  0,  U,   /* x   y   z   {    |   }   ~   DEL */
 
   U, U, U, U,  U, U, U, U,  U, U, U, U,  U, U, U, U,
   U, U, U, U,  U, U, U, U,  U, U, U, U,  U, U, U, U,
@@ -1269,7 +1270,7 @@
    translate file name back to URL, this would become important
    crucial.  Right now, it's better to be minimal in escaping.  */
 
-const static unsigned char filechr_table[256] =
+static const unsigned char filechr_table[256] =
 {
 UWC,  C,  C,  C,   C,  C,  C,  C,   /* NUL SOH STX ETX  EOT ENQ ACK BEL */
   C,  C,  C,  C,   C,  C,  C,  C,   /* BS  HT  LF  VT   FF  CR  SO  SI  */
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.