AW: AW: Problems using robots with authentication

"Hohl, Gerrit" <[email protected]> Tue, 14 Feb 2012 16:25:00 +0100
Newsgroups	gmane.comp.web.web-polygraph.user
Message-ID	<[email protected]>
Hello Dmitry,

okay, the OpenSSL package was installed on my system, but the libssl-dev not. I executed ./configure and everything went fine: All SSL test were green. I execute make and make install. But I still facing some problems.

I have credentials in my polygraph test case like this one:

string[] cred = [ "TEST/[email protected]:p1FXn2S165", [...] ];

I've running a squid with NTLM support. But the polygraph-client can't establish a connection to the server using this proxy. It always runs into "407 Proxy Authentication Required" messages. So I used Ethereal Version 0.10.14 on that squid machine - one time I analyzed the communication with the polygraph-client, the second time I used a Firefox on a Window machine.

polygraph-client communication
------------------------------
1st request:

GET http://w1141.h1128o1005s1010.bench.tst/w18d91ae1.2d680f1b:00000120/t06/_00000001 HTTP/1.1
Accept: */*
Host: w1141.h1128o1005s1010.bench.tst
X-Xact: 18d91ae1.2d680f1b:00000002 18d91ae1.2d680f1b:0000042c 0
X-Loc-World: 18d91ae1.2d680f1b:00000120 -1/1 0
X-Rem-World: 18d91ae1.2d680f1b:00000120 -1/1 0
X-Target: w1141.h1128o1005s1010.bench.tst:80
X-Abort: 1412400744 2082554117
X-Phase-Sync-Pos: 0

1st response:

HTTP/1.0 407 Proxy Authentication Required
Server: squid/2.7.STABLE6
Date: Tue, 14 Feb 2012 13:44:50 GMT
Content-Type: text/html
Content-Length: 1550
X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
Proxy-Authenticate: NTLM
Proxy-Authenticate: Basic realm="squid server"
X-Cache: MISS from polygraph-squid.windows.local
X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128
Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6)
Connection: close

2nd request:

GET http://w1141.h1128o1005s1010.bench.tst/w18d91ae1.2d680f1b:00000120/t06/_00000001 HTTP/1.1
Accept: */*
Host: w1141.h1128o1005s1010.bench.tst
X-Xact: 18d91ae1.2d680f1b:00000002 18d91ae1.2d680f1b:0000042e 0
X-Loc-World: 18d91ae1.2d680f1b:00000120 -1/1 0
X-Rem-World: 18d91ae1.2d680f1b:00000120 -1/1 0
X-Target: w1141.h1128o1005s1010.bench.tst:80
X-Abort: 1798565613 512442519
X-Phase-Sync-Pos: 0
Proxy-Authorization: NTLM TlRMTVNTUAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA=

Ethereal shows the following information in Proxy-Authorization > NTLMSSP
Flags: 0x00088206
Calling workstation domain: NULL
Calling workstation name: NULL

2nd response:

HTTP/1.0 407 Proxy Authentication Required
Server: squid/2.7.STABLE6
Date: Tue, 14 Feb 2012 13:44:51 GMT
Content-Type: text/html
Content-Length: 1550
X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
Proxy-Authenticate: Basic realm="squid server"
X-Cache: MISS from polygraph-squid.windows.local
X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128
Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6)
Connection: close

Windows / Firefox communication
-------------------------------
1st request:

GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1
Host: download.mozilla.org
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Proxy-Connection: keep-alive
Range: bytes=300000-599999
Cookie: dmo=10.8.84.211.1329128259926354

1st response:

HTTP/1.0 407 Proxy Authentication Required
Server: squid/2.7.STABLE6
Date: Tue, 14 Feb 2012 14:16:05 GMT
Content-Type: text/html
Content-Length: 1397
X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
Proxy-Authenticate: NTLM
Proxy-Authenticate: Basic realm="squid server"
X-Cache: MISS from polygraph-squid.windows.local
X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128
Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6)
Connection: close

2nd request:

GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1
Host: download.mozilla.org
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Proxy-Connection: keep-alive
Range: bytes=300000-599999
Cookie: dmo=10.8.84.211.1329128259926354
Proxy-Authorization: NTLM TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAGAbAdAAAADw==

Ethereal shows the following information in Proxy-Authorization > NTLMSSP
Flags: 0xA2088207
Calling workstation domain: NULL
Calling workstation name: NULL

2nd response:

HTTP/1.0 407 Proxy Authentication Required
Server: squid/2.7.STABLE6
Date: Tue, 14 Feb 2012 14:16:05 GMT
Content-Type: text/html
Content-Length: 1397
X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
Proxy-Authenticate: NTLM TlRMTVNTUAACAAAACAAIADgAAAAFgomiCEQRwpFGL1oAAAAAAAAAAMgAyABAAAAABgGwHQAAAA9UAEUAUwBUAAIACABUAEUAUwBUAAEAGgBXADIASwA4AFIAMgBTAFIAVgAtAEQATwBNAAQAIgB0AGUAcwB0AC4AYQB1AHIAZQBuAHoALgBsAG8AYwBhAGwAAwA+AFcAMgBLADgAUgAyAFMAUgBWAC0ARABPAE0ALgB0AGUAcwB0AC4AYQB1AHIAZQBuAHoALgBsAG8AYwBhAGwABQAiAHQAZQBzAHQALgBhAHUAcgBlAG4AegAuAGwAbwBjAGEAbAAHAAgAcxVmHSPrzAEAAAAA
X-Cache: MISS from polygraph-squid.windows.local
X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128
Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6)
Connection: keep-alive
Proxy-Connection: keep-alive

Ethereal shows the following information in Proxy-Authorization > NTLMSSP
Flags: 0xA2908205

3rd request:

GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1
Host: download.mozilla.org
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Proxy-Connection: keep-alive
Range: bytes=300000-599999
Cookie: dmo=10.8.84.211.1329128259926354
Proxy-Authorization: NTLM TlRMTVNTUAADAAAAGAAYAH4AAAAYABgAlgAAAAgACABYAAAADgAOAGAAAAAQABAAbgAAAAAAAACuAAAABYKIogYBsB0AAAAPn8qbQDsUUb8Odt0FrfLvDVQARQBTAFQAdABlAHMAdABlAHIANQBDAEwASQBFAE4AVAAtADMAoHqz/a+76EoAAAAAAAAAAAAAAAAAAAAAwOMG9tdnijCslk8x46O5Jk5+0GXpoiPd

Ethereal shows the following information in Proxy-Authorization > NTLMSSP
Flags: 0xA2888205

3rd response:

HTTP/1.0 302 Moved Temporarily
Date: Tue, 14 Feb 2012 14:15:34 GMT
Server: Apache
X-Backend-Server: pp-app-dist01
X-Powered-By: PHP/5.1.6
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0, private
Pragma: no-cache
Location: http://mirror01.th.ifl.net/mozilla-releases/firefox/releases/10.0.1/update/win32/de/firefox-10.0.1.complete.mar
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Cache: MISS from polygraph-squid.windows.local
X-Cache-Lookup: MISS from polygraph-squid.windows.local:3128
Via: 1.1 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6)
Connection: keep-alive
Proxy-Connection: keep-alive

Because of the NULL values I first thought that this was the problem. But in the second test there are also NULL values. The most significant difference are the NTLM flags. Can this be the cause of the problem?
I made a table which compares the different flags. And it shows that Firefox supports 56-bit as well as 128-bit encryption, but WebPolgraph doesn't. Maybe we (mean squid) need this?

WP2Q -> WebPolygraph 2nd reQuest
WF2Q -> Windows Firefox 2nd reQuest
WF2S -> Windows Firefox 2nd reSponse
WF3Q -> Windows Firefox 3rd reQuest

Value (from highest to lowest bit)     | WP2Q | WF2Q | WF2S | WF3Q 
---------------------------------------|------|------|------|------
Negotiate 56                           |   -  |   x  |   x  |   x  
Negotiate Key Exchange                 |   -  |   -  |   -  |   -  
Negotiate 128                          |   -  |   x  |   x  |   x  
Negotiate 0x10000000                   |   -  |   -  |   -  |   -  
Negotiate 0x08000000                   |   -  |   -  |   -  |   -  
Negotiate 0x04000000                   |   -  |   -  |   -  |   -  
Negotiate 0x02000000                   |   -  |   x  |   x  |   x  
Negotiate 0x01000000                   |   -  |   -  |   -  |   -  
Negotiate Target Info                  |   -  |   -  |   x  |   x  
Negotiate 0x00400000                   |   -  |   -  |   -  |   -  
Negotiate 0x00200000                   |   -  |   -  |   -  |   -  
Negotiate 0x00100000                   |   -  |   -  |   x  |   -  
Negotiate NTLM2 key                    |   x  |   x  |   -  |   x  
Negotiate Challenge Non NT session Key |   -  |   -  |   -  |   -  
Negotiate Challenge Accept Response    |   -  |   -  |   -  |   -  
Negotiate Challenge Init Reponse       |   -  |   -  |   -  |   -  
Negotiate Always Sign                  |   x  |   x  |   x  |   x  
Negotiate This is Local Call           |   -  |   -  |   -  |   -  
Negotiate Workstation Supplied         |   -  |   -  |   -  |   -  
Negotiate Domain Supplied              |   -  |   -  |   -  |   -  
Negotiate 0x00000800                   |   -  |   -  |   -  |   -  
Negotiate 0x00000400                   |   -  |   -  |   -  |   -  
Negotiate NTLM key                     |   x  |   x  |   x  |   x  
Negotiate Netware                      |   -  |   -  |   -  |   -  
Negotiate Lan Manager Key              |   -  |   -  |   -  |   -  
Negotiate Datagramm Style              |   -  |   -  |   -  |   -  
Negotiate Seal                         |   -  |   -  |   -  |   -  
Negotiate Sign                         |   -  |   -  |   -  |   -  
Request 0x00000008                     |   -  |   -  |   -  |   -  
Request Target                         |   x  |   x  |   x  |   x  
Negotiate OEM                          |   x  |   x  |   -  |   -  
Negotiate UNICODE                      |   -  |   x  |   x  |   x  

I hope my mail didn't grow too long. But I wanted it be as detailed as possible. Maybe it will help to find my mistake.

Regards,
Gerrit


-----Ursprüngliche Nachricht-----
Von: Dmitry Kurochkin [mailto:[email protected]] 
Gesendet: Donnerstag, 15. Dezember 2011 21:12
An: Hohl, Gerrit; [email protected]
Betreff: Re: AW: Problems using robots with authentication

Hi Gerrit.

On Thu, 15 Dec 2011 17:36:34 +0100, "Hohl, Gerrit" <[email protected]> wrote:
> Hello everyone,
> 
> I read the article "Prerequisites" in the documentation:
> http://www.web-polygraph.org/docs/reference/models/ssl.html#Sect:2
> 
> Polygraph SSL support is based on the OpenSSL library. A recent version of the library is required to compile Polygraph. We have tested with OpenSSL versions 0.9.6g and 0.9.7b. The presence of OpenSSL is determined at ./configure time. Please check that ./configure actually found SSL library and headers if you install Polygraph and want SSL support:
> 
>     ...
>     checking for CRYPTO_lock in -lcrypto... yes
>     checking for SSL_connect in -lssl... yes
>     checking for openssl/ssl.h... yes
>     checking for openssl/err.h... yes
>     checking for openssl/rand.h... yes
> 
> Eh, I don't get these lines when I call the script. I assume that the script was modified, but the documentation not.
> 

The exact messages may have change, but they are still there:

  $ ./configure | grep -i ssl
  checking for SSL_connect in -lssl... yes
  checking openssl/ssl.h usability... yes
  checking openssl/ssl.h presence... yes
  checking for openssl/ssl.h... yes
  checking openssl/err.h usability... yes
  checking openssl/err.h presence... yes
  checking for openssl/err.h... yes
  checking openssl/rand.h usability... yes
  checking openssl/rand.h presence... yes
  checking for openssl/rand.h... yes

Assertion at NtlmAuth.cc:798 means that you built without OpenSSL support.  You should install it and rebuild.  On Debian-based systems you should install libssl-dev package.

The assertion is always a bug.  Web Polygraph should print a proper error here.  There is an open bug #878881 [1] for this issue.

> One remark: I only want NTLM authentication and not HTTPS benchmarking.
> 

NTLM needs some crypto functions (MD5, at least).  That is why OpenSSL is needed for it.

Regards,
  Dmitry

[1] https://bugs.launchpad.net/polygraph/+bug/878881

> Regards,
> Gerrit
> 
> -----Ursprüngliche Nachricht-----
> Von: [email protected] 
> [mailto:[email protected]] Im Auftrag von Hohl, Gerrit
> Gesendet: Donnerstag, 15. Dezember 2011 17:08
> An: [email protected]
> Betreff: Problems using robots with authentication
> 
> Hello everyone,
> 
> I have the same problem that ufa faced at the end of October:
> I use NTLM authentication and get the message
> 
>    NtlmAuth.cc:798: assertion failed: 'false'
>    Aborted
> 
> from the polygraph-client program. After the first time I received that message I included the following line in my PGL file:
> 
> Robot robot = {
>         [...]
>         pconn_use_lmt = const(2147483647);
>         [...]
> };
> 
> But I still get the same message. I read something in Dmitry response about OpenSSL. The OpenSSL package ('openssl') is installed on that Ubuntu machine I'm currently using. But I'm not sure if polygraph was compiled with or without SSL. Is there a way to test it? Or what do I have to do to make sure that polygraph compiles with SSL support?
> 
> Regards,
> Gerrit
> _______________________________________________
> Users mailing list
> [email protected]
> http://www.web-polygraph.org/mailman/listinfo/users
> _______________________________________________
> Users mailing list
> [email protected]
> http://www.web-polygraph.org/mailman/listinfo/users
_______________________________________________
Users mailing list
[email protected]
http://www.web-polygraph.org/mailman/listinfo/users