[ ijbswa-Feature Requests-3603913 ] socks5 message / error passthrough

Feature Requests item #3603913, was opened at 2013-02-08 22:10
Message generated for change (Comment added) made by fabiankeil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=361118&aid=3603913&group_id=11118

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Pending
Resolution: None
Priority: 5
Private: No
Submitted By: grarpamp (grarpamp)
Assigned to: Fabian Keil (fabiankeil)
Summary: socks5 message / error passthrough

Initial Comment:
Privoxy 3.0.19 on FreeBSD 8.3 i386
 
With polipo I'd see helpful diagnostic messages from Tor's socks5 implementation
passed through polipo and eventually reaching wget's log file. Though I'm not sure
if it was the exact socks5 error code number, the text was still useful. Example...
 
1300 504 Connect to onion failed: SOCKS error: host unreachable.
    80 504 Connect to onion failed: SOCKS error: TTL expired.
    30 504 Connect to onion failed: SOCKS connection not allowed.
    15 504 Connect to onion failed: General SOCKS server failure.
    10 504 Connect to onion failed: SOCKS error: connection refused.
 
With Privoxy, I get vague 500 level messages that don't really tell me anything
about the far end. (I don't have relative counts for this example right now).
 
ERROR 500: Internal Privoxy Error.
ERROR 500: Internal Server Error.
ERROR 502: Server or forwarder response invalid.
ERROR 503: Service unavailable.
ERROR 504: Gateway Time-out.
Proxy tunneling failed: Internal Privoxy ErrorUnable to establish SSL connection.
 
So I'm hereby suggesting that...
1) more detailed messages be passed through wherever available from the far end.
2) these 'internal' errors receive more verbose or coded treatment as to what function
is kicking them out. (that's probably the bug-ish part of this feature-ish report).
 
Thx.


----------------------------------------------------------------------

>Comment By: Fabian Keil (fabiankeil)
Date: 2014-02-02 04:39

Message:
Given that this ticket contains lots of feature requests of which some have
been accepted while others have been rejected I'm not sure what "these
concepts" refers to.

If you are interested in providing patches for review, I'd suggest you
start with the ones to duplicate the socks error description in the
response line.

----------------------------------------------------------------------

Comment By: grarpamp (grarpamp)
Date: 2014-02-01 13:25

Message:
Would like to see these concepts patched for testing and committed :) After
a reread are there any remaining questions/discuss needed for that? Thx :)

----------------------------------------------------------------------

Comment By: grarpamp (grarpamp)
Date: 2013-02-13 16:28

Message:
> In my opinion using status code 500 for socks errors would be
> worse than 502 or 504, even though I agree that they don't fit
> perfectly either.

502 and 504 were the only ones that defined any 'gateway' context.
For socks, privoxy currently utilizes 503 which is not defined to
be a gateway message, and is defined for a fairly specific problem,
just like 502 and 504 are.

 503 Service Unavailable
   The server is currently unable to handle the request due to a
   temporary overloading or maintenance of the server.

So I figured 500 was a generic enough place to append the specific
socks codes. 503 is reasonable if considering its short definition
'Service Unavailable'. Choose whatever works. However I would put
all the SOCKS reply codes/msgs under the same extended HTTP code.


With temporary templates, here's wget's vision of 8000 fetches...
   8 ERROR 502: No data received from server or forwarder.
  12 ERROR 502: Server or forwarder response invalid.
3421 ERROR 503: Forwarding failure.
   1 ERROR 503: Service Temporarily Unavailable.
   1 ERROR 503: Service Unavailable.
3385 Proxy tunneling failed: Forwarding failureUnable to establish SSL
connection.


> The templates

I'm doing template handling in a separate ticket when I'm able to
present it.

> In my opinion socks4(a) improvements would be a waste of time
> given that all the relevant gateways support the mostly-superior
> socks5(t).

I have no use case for socks4/4a either and wish people would not
deploy those servers. I just mentioned it to be complete :) Though
if 4/4a are still in Privoxy, it's probably better to fix them up
too.

> I don't see the point of mentioning Privoxy in all its response
> lines. It's already mentioned in some, though.

Not for all lines, only lines where it would help distinguish who
is actually emitting the error. Another example, chained proxies.
Who is to know if the local privoxy got the error, or was passed
it from farther away.

Though for the socks error case, if the literal 'Privoxy: ' were
trimmed out, the strict format of the remaining line would give a
clue to whoever looks up that string in the Privoxy docs.

Whichever way, at least the 'SOCKS<ver>: <nn> <msg>' needs added.

The '<ver>' could be trimmed because the socks4/4a/5 '<nn> <msg>'
codes do not overlap.

> BTW, you can already do the opposite and use a server-header
> filter to mark error-indicating response lines that aren't coming
> from Privoxy.

In some projects I wish to preserve the original headers from the
far side. In others I need to translate. Privoxy is amazing to do
that.

> I think the correlation issues can only be solved with another
> Tor-specific socks extension but haven't had time to write a
> proposal or implement it

Too, if I serialized things for debugging I could line up multiple
logs by time or content, but serial just isn't a production option.
I do some testing by hand as needed. Polipo locks up under heavy
parallel use, and seems a dead project, so I switched back to Privoxy
for everything :)


----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2013-02-11 06:42

Message:
In my opinion using status code 500 for socks errors would be worse than
502 or 504, even though I agree that they don't fit perfectly either.

The templates aren't considered optional and deleting them is supposed to
result in error messages.

You are welcome to empty or edit the templates, though.

In my opinion socks4(a) improvements would be a waste of time given that
all the relevant gateways support the mostly-superior socks5(t).

I don't see the point of mentioning Privoxy in all its response lines. It's
already mentioned in some, though.

BTW, you can already do the opposite and use a server-header filter to mark
error-indicating response lines that aren't coming from Privoxy.

I think the correlation issues can only be solved with another Tor-specific
socks extension but haven't had time to write a proposal or implement it
yet.

----------------------------------------------------------------------

Comment By: grarpamp (grarpamp)
Date: 2013-02-09 18:18

Message:
Did more work on this...

https://tools.ietf.org/html/rfc2616

Per HTTP rfc2616 sec:10.5, user agents SHOULD display any entity...
ok, displayed - firefox, curl, elinks, lynx, telnet, netcat
broken, not displayed - wget, fetch, socat[!]

This entity, 'forwarding-failed', cannot know any more
info than the socks5 rfc1928 sec:6 reply itself.
We can pick an HTTP rfc2616 sec:10.5 error code to ride on top of.

Unusuitable since the error is already specifically defined...
502 Bad Gateway - invalid response from webserver (or auxiliary: Tor)
504 Gateway Timeout - timeout on webserver (or auxiliary: Tor)

The most sensible HTTP error code seems to be...
500 Internal Server Error - unexpected condition prevented fulfillment

Privoxy can pass that through from webserver when not using socks,
or when the socks5 reply code is 00h.

And when using socks5, if reply code of 01h through FFh occur,
Privoxy could then reissue the 500 in that case as...

500 Internal Server Error - Privoxy: SOCKS<ver>: <nn> <msg>

Where <ver> is socks version. If version 5, <nn> is 01~FF, and
<msg> is as shown in socks5 rfc1928 or highly abbreviated.

socks4 and socks4a follow similarly.

There is merit in optionally including the attempted URL in the 
'forwarding-failed' entity. I removed that file, so it emits this
header instead: 500 Internal Privoxy Error

Privoxy should probably not do that, or have a 'support fileless
normally' mode. That is for another ticket.


----------------------------------------------------------------------

Comment By: grarpamp (grarpamp)
Date: 2013-02-09 14:26

Message:
> Adding the socks error description to the response line seems useful to
me.
 
Ok, cool. Section 6 would be apropriate:
https://tools.ietf.org/html/rfc1928
 
> BTW, at least some of your example error messages don't actually come
from Privoxy
 
Ahh, so maybe there is ability to indicate in the status response header
from privoxy to
the client (wget), in general, if it was privoxy that generated the error.
A formatted
privoxy: label perhaps.
 
> Note that Privoxy already provides the details as part of the response
body
> I don't think discarding the problem description in the body ... [is]
reasonable
 
Hmm, I think wget deems any such reply body received along with an error
header
as erroneous to the intended body and thus does not save them to disk. I
will look
at this and possibly ticket or replace wget. Thanks.
 
> Privoxy already provides the details ... and properly logs them when
configured.
> checking Privoxy's log would make their origin more obvious.
> correlation socks errors to Tor circuits currently isn't trivial.
 
Yes, for reproducing the error as well. And matching up thousands of
lines in wget, privoxy, and Tor logs, from parallel fetches, is tough too.
Divide and conquer may work here for whatever is not done by this ticket.
In the meantime I'll bump 'debug' to see what privoxy currently thinks
about 'privoxy <--> Tor' regarding socks.
 
wget <--> privoxy <--> Tor <--> webserver
'far end' - I mean reply from the webserver (the far end) back through to
wget.
 
Thanks.


----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2013-02-09 03:28

Message:
Adding the socks error description to the response line seems useful to me.
Thanks for the suggestion.

Note that Privoxy already provides the details as part of the response body
and properly logs them when configured to do so.

Internal Privoxy Errors usually aren't bugs but configuration problems and
I don't think discarding the problem description in the body and only
looking at the response line is a reasonable debugging strategy, even when
doing it from the "far end".

BTW, at least some of your example error messages don't actually come from
Privoxy and checking Privoxy's log would make their origin more obvious.
Unfortunately correlation socks errors to Tor circuits currently isn't
trivial.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=361118&aid=3603913&group_id=11118

------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable 
security intelligence. It gives you real-time visual feedback on key
security issues and trends.  Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk