[ ijbswa-Bugs-3481589 ] URL with IPv6 address can't be accessed

Bugs item #3481589, was opened at 2012-01-30 02:28
Message generated for change (Comment added) made by zoolooo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=111118&aid=3481589&group_id=11118

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: debian specific
Group: version 3.0.19
>Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: zooloo (zoolooo)
Assigned to: Fabian Keil (fabiankeil)
Summary: URL with IPv6 address can't be accessed

Initial Comment:
My test set has eth0/1 interfaces configured on different IPv6 subnets
Test client uses curl to access IPv6 in different from it's own subnet trough proxy.

When trying to access URL like https:://[fc00:1:2::5:6:b]:8443/testsrv/index.php 

All addresses and URLs - are my local internal use services, but it seems like problem is not related to exact service or privoxy configuration

[root@sh5 ~]# curl -v -g -k "https://[fc00:1:2::5:6:b]:8443/enterprise/control/agent.php"
* About to connect() to proxy 105.168.1.1 port 8008
*   Trying 105.168.1.1... connected
* Connected to 105.168.1.1 (105.168.1.1) port 8008
* Establish HTTP proxy tunnel to fc00:1:2::5:6:b:8443
> CONNECT fc00:1:2::5:6:b:8443 HTTP/1.0
> Host: fc00:1:2::5:6:b:8443
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.0 404 No such domain
< Content-Length: 7377
< Content-Type: text/html
< Cache-Control: no-cache
< Date: Mon, 30 Jan 2012 11:06:06 GMT
< Last-Modified: Wed, 08 Jun 1955 12:00:00 GMT
< Expires: Sat, 17 Jun 2000 12:00:00 GMT
< Pragma: no-cache
< Connection: close
< 
* Received HTTP code 404 from proxy after CONNECT
* Closing connection #0
curl: (56) Received HTTP code 404 from proxy after CONNECT

I'm using:

curl --version
curl 7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
Protocols: tftp ftp telnet dict ldap http file https ftps 
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

----------------------------------------------------------------------

>Comment By: zooloo (zoolooo)
Date: 2012-02-08 10:05

Message:
Now I see that patch wasn't applied. Well now the initial request
(https://[fc00:1:2::5:6:b]:8443/enterprise/control/agent.php) seems
working.
According to source... What will happen if URL like
"https:://[fc00:1:2::5:6:b]/testsrv/index.php" being used.
There is no ":port" signature and strrchr(header, ':') will find something
wrong.

----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2012-02-08 06:27

Message:
The Host header isn't that important here, as proxies are supposed to take
the host from the request line.

Also the patch should have no effect on the "scan:" lines as they are
always taken from the client request as-is.

The patch should, however, let Privoxy split "fc00:1:2::5:6:b:8443" (from
the request line) at the last colon and from your log it looks like it may
still be splitting at the first one instead.

The patch seems to work for me:

2012-02-08 15:13:08.972 801807400 Connect: Listening for new connections
...
2012-02-08 15:13:08.973 801808800 Header: scan: CONNECT
fc00:1:2::5:6:b:8443 HTTP/1.1
2012-02-08 15:13:08.974 801808800 Header: Tagger 'client-ip-address' added
tag 'IP-ADDRESS: 10.0.0.1'. No action bits update necessary.
2012-02-08 15:13:08.974 801808800 Header: Tagger 'http-method' added tag
'CONNECT'. Action bits updated accordingly.
2012-02-08 15:13:08.974 801808800 Header: scan: Host:
[fc00:1:2::5:6:b]:8443
2012-02-08 15:13:08.974 801808800 Header: Adding: Connection: close
2012-02-08 15:13:08.974 801808800 Connect: Overriding forwarding settings
based on 'forward .'
2012-02-08 15:13:08.974 801808800 Request: fc00:1:2::5:6:b:8443/
2012-02-08 15:13:08.974 801808800 Connect: to fc00:1:2::5:6:b:8443
2012-02-08 15:13:08.974 801808800 Connect: Connected to
fc00:1:2::5:6:b[fc00:1:2::5:6:b]:8443.
2012-02-08 15:13:08.975 801808800 Connect: Created new connection to
fc00:1:2::5:6:b:8443 on socket 5.
2012-02-08 15:13:08.975 801808800 Connect: to fc00:1:2::5:6:b:8443
successful
10.0.0.1 - - [08/Feb/2012:15:13:08 +0100] "CONNECT fc00:1:2::5:6:b:8443
HTTP/1.1" 200 0
2012-02-08 15:13:08.975 801808800 Connect: The connection on server socket
5 to fc00:1:2::5:6:b isn't reusable. Closing. Server connection: keep-alive
0, tainted: 1, socket alive 1. Client connection: socket alive: 0. Server
timeout: 300. Configuration file change detected: 0

Can you please double-check that the patch has been properly applied?

If you grep for strrchr, you should get two hits:

fk@r500 ~/git/privoxy $grep strrchr *.c
parsers.c:   p = strrchr(header, ':');
urlmatch.c:         port = strrchr(host, ':');

If you get the same result, please additionally enable "debug 2" and post a
new log excerpt.

----------------------------------------------------------------------

Comment By: zooloo (zoolooo)
Date: 2012-02-08 05:55

Message:
Actually I did.
According to log in my previous post Parser shows that it parses host well
enough 
'2012-01-31 21:45:44.337 b7749b90 Header: scan: Host:
[fc00:1:2::5:6:b]:8443'
But still privoxy eager to connect to host fc00:1:2::5:6:b:8443... 
'2012-01-31 21:45:44.337 b7749b90 Request: fc00:1:2::5:6:b:8443/'

Did I miss something from the patch?

----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2012-02-08 05:36

Message:
Did you try the patch I attached? To me it looks like you didn't.

More recent Firefox versions don't seem to be affected anyway. At least my
Firefox 10 on FreeBSD is using brackets around the IPv6 address: CONNECT
[fc00:1:2::5:6:b]:8443 HTTP/1.1

----------------------------------------------------------------------

Comment By: zooloo (zoolooo)
Date: 2012-01-31 09:53

Message:
According to log of 'CONNECT' request from my Mozilla privoxy still has a
problem with numeric IPv6 handling

2012-01-31 21:44:06.105 b774a6c0 Info: Privoxy version 3.0.19
2012-01-31 21:44:06.207 b774a6c0 Info: Program name: privoxy
2012-01-31 21:44:06.207 b774a6c0 Info: Loading filter file:
/etc/privoxy/default.filter
2012-01-31 21:44:06.804 b774a6c0 Info: Loading actions file:
/etc/privoxy/match-all.action
2012-01-31 21:44:07.186 b774a6c0 Info: Loading actions file:
/etc/privoxy/default.action
2012-01-31 21:44:07.201 b774a6c0 Info: Loading actions file:
/etc/privoxy/user.action
2012-01-31 21:44:07.411 b774a6c0 Info: Listening on port 8008 on IP address
105.168.1.1
2012-01-31 21:44:07.411 b774a6c0 Info: Listening on port 8008 on IP address
fc00:1:2:2::2
2012-01-31 21:44:07.411 b774a6c0 Info: Listening on port 8118 on IP address
10.27.145.104
2012-01-31 21:45:44.325 b7749b90 Header: scan: CONNECT fc00:1:2::5:6:b:8443
HTTP/1.1
2012-01-31 21:45:44.337 b7749b90 Header: scan: User-Agent: Mozilla/5.0
(X11; U; Linux i686; en-US; rv:1.9.2.24) Gecko/20111109
CentOS/3.6.24-3.el6.centos Firefox/3.6.24
2012-01-31 21:45:44.337 b7749b90 Header: scan: Proxy-Connection:
keep-alive
2012-01-31 21:45:44.337 b7749b90 Header: scan: Host:
[fc00:1:2::5:6:b]:8443
2012-01-31 21:45:44.337 b7749b90 Header: crumble crunched:
Proxy-Connection: keep-alive!
2012-01-31 21:45:44.337 b7749b90 Header: Adding: Connection: close
2012-01-31 21:45:44.337 b7749b90 Request: fc00:1:2::5:6:b:8443/
2012-01-31 21:45:45.032 b7749b90 Info: Can not resolve fc00: Name or
service not known
2012-01-31 21:45:45.035 b7749b90 Crunch: DNS failure: fc00:1:2::5:6:b:8443



----------------------------------------------------------------------

Comment By: zooloo (zoolooo)
Date: 2012-01-31 03:59

Message:
I don't know if this is  a false alarm but seems curl is the one who makes
troubles
I've added bug to curl branch
https://sourceforge.net/tracker/?func=detail&aid=3482093&group_id=976&atid=100976

And I wander if there is need to make any fixes for case when CONNECT
request is in wrong format (e.g. check for multiple ':' symbols in host
name without any '[ ]' symbols in case of wrong IPv6 address)

----------------------------------------------------------------------

Comment By: zooloo (zoolooo)
Date: 2012-01-31 03:59

Message:
Turned into feature request


----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2012-01-30 12:28

Message:
Thanks a lot for the report.

I confirmed that Privoxy doesn't properly deal with the request, but with
FEATURE_IPV6_SUPPORT the error message I get is "HTTP/1.1 503 Connect
failed". This difference might be due to GNU libc vs. FreeBSD's libc.

While testing this, I also found a related bug with CONNECT requests to
::1.

Potential fixes (only briefly tested) for both issues are attached, please
given them a try.

----------------------------------------------------------------------

Comment By: zooloo (zoolooo)
Date: 2012-01-30 07:05

Message:
Looks privoxy resolve IPv6 address like it was domain name. Anyway it looks
weird.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=111118&aid=3481589&group_id=11118

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d