Re: a issue when running the "Simple tests" with a proxy
xu alex <[email protected]> Tue, 11 Mar 2014 14:23:00 +0800
| Newsgroups | gmane.comp.web.web-polygraph.user |
|---|---|
| Message-ID | <CADtOdmVK3J8j=iLtoOQz=0AJoAFVgaOxb=FoN65YkS7=+wx-cw@mail.gmail.com> |
--===============1452000669==
Content-Type: multipart/alternative; boundary=047d7b6d8f167f22ca04f44ec04a
--047d7b6d8f167f22ca04f44ec04a
Content-Type: text/plain; charset=GB2312
Content-Transfer-Encoding: quoted-printable
Hi Alex=A3=AC
Thanks for your reply. Yes, I has been testing squid using as a reverse
proxy. I tried the way of adding PGL AddrMap in my config file, but errors
showed up when starting polygraph-client. My config file is as following:
---------------------------------------------------------------------------=
-------------------------------------------------------
ObjLifeCycle olcStatic =3D {
birthday =3D now + const(-1year); // born a year ago
length =3D const(2year); // two year cycle
variance =3D 0%; // no variance
with_lmt =3D 100%; // all responses have LMT
expires =3D [nmt + const(0sec)]; // everything expires when modified
};
Content SimpleContent =3D {
size =3D exp(13000KB); // response sizes distributed exponentially
cachable =3D 100%; // 20% of content is uncachable
obj_life_cycle =3D olcStatic;
};
Server S =3D {
kind =3D "S101";
contents =3D [ SimpleContent ];
direct_access =3D contents;
addresses =3D ['218.202.111.138:80']; // where to create these serv=
er
agents
};
AddrMap Map =3D {
addresses =3D S.addresses;
names =3D ['218.202.111.5:8101'];
};
// a primitive robot
Robot R =3D {
kind =3D "R101";
pop_model =3D { pop_distr =3D popUnif(); };
recurrence =3D 55% / SimpleContent.cachable; // adjusted to get 55%
DHR
req_rate =3D 10/sec;
origins =3D Map.names; // where the origin servers are
addresses =3D ['218.202.111.138' ** 10]; // where these robot agent=
s
will be created
};
// commit to using these servers and robots
use(S, R);
The error info showed up when i started polygraph-clients is as following:
---------------------------------------------------------------------------=
---------------------------------------------------------------------------
fyi: PGL configuration stored (664bytes)
000.00| fyi: all 10 Robot IP address(es) lack interface name or netmask
000.00| found 10 Robot IP address(es) total (0 unique address(es) with
interface name and netmask).
000.00| fyi: no bench selected with use()
000.00| ./my_simple.pg:41: error: Robot cannot find configuration for
[email protected]:8101
I am a newer for polygraph, and can not unstand WebAxe workload
configuration completely, so i think there are some mistakes in my
configuration file. Can you help me take a look at my config file showed
above for which part is wrong, and how i should modify it?
Currently, I just modify the source code as following to make it work
with my squid server:
---------------------------------------------------------------------------=
------------------------------------------------------------
ostream &Oid2Url(const ObjId &oid, ostream &os) {
if (oid.foreignUrl())
return os << oid.foreignUrl();
if (oid.secure())
os << "https://";
else
- os << (oid.scheme() =3D=3D Agent::pHTTP ? "http://" : "ftp://");
+ os << (oid.scheme() =3D=3D Agent::pHTTP ? "/" : "ftp://");
Oid2UrlHost(oid, false, os);
Oid2UrlPath(oid, os);
return os;
}
I think that is not right way to use polygraph for test reverse proxy, so
look forward to your reply!
2014-03-11 0:05 GMT+08:00 Alex Rousskov <[email protected]>:
> On 03/10/2014 02:02 AM, xu tony wrote:
> > I've been using the "Simple tests" introdued by polygraph's User
> > manual to test my squid server. During testing, I found my squid can't
> > pull the resource from orign(that is polygraph-server) successfully.
>
> > 3=A1=A2The command i used for starting polygraph-server and polygraph-c=
lient
> > is ,
> > /usr/local/polygraph/bin/polygraph-client --config
> > ../workdir/my_simple.pg --verb_lvl 10 --proxy
> > 218.202.111.5:8101
>
>
> > During test, i found squid have got requests from
> > ploygraph-client,but using a wrong url to pull resource (that is
> > polygraph-server).Part of my squid access log is showed as below:
> >
> -------------------------------------------------------------------------=
-------------------------------------
> > 218.202.111.138 - - [10/Mar/2014:15:58:49 +0800] "GET
> > http://w1cbc2601.7f1c694f:8/t03/_00000003 HTTP/1.1" 503 1513 "-" "-"
> > TCP_MISS DIRECT 0 -
>
> The wrong domain name in the above log implies that your Squid is
> confused as to where the requests are going. This would not happen with
> a regular Squid configuration. Starting with a simple forward proxy
> configuration for Squid may be a good idea before moving on to more
> complex stuff (squid.conf.default may work after adding http_access
> rules to allow Polygraph traffic).
>
>
> > I snapped http-request packets from polygraph-client, and found
> > request-line of http-request is not what squid expected,which caused th=
e
> > squid can't pull resource from polyserver.The request-line in
> > http-request sent by polyclient is like that :
> > "GET http://218.202.111.138/w1cbc0feb.0fea1bae:00000008/t03/_00000001
> > HTTP/1.1"
>
> Yes, this is what you told Polygraph to send. In a forward proxy test
> that you are running (from Polygraph point of view), the URL should look
> like the one logged above.
>
>
> > The right request-line which the squid expected ,i think, shoud be like=
:
> > "GET /218.202.111.138/w1cbc02ee.0c726ba0:00000008/t03/_00000015HTTP/1.=
0"
>
> No, but if you are testing a reverse proxy, the Request URI that Squid
> expects should look like this:
>
> GET /w1cbc02ee.0c726ba0:00000008/t03/_00000015 HTTP/...
>
> It would still be logged as a full URL (with a host name) by Squid IIRC,
> but the packet capture should show from-Polygraph Request URIs without
> the protocol scheme and host name components.
>
> Please note that some packet capture GUIs such as Wireshark sometimes
> "restore" the full URL when rendering high-level details of the captured
> packet, even when a full URL is not present in the packet. Make sure you
> look at what has been actually sent and what what was rendered by GUI.
>
>
> > So, i want to know which part is wrong in my test, if i have made
> > some mistakes in test, or polyclient sent http-request with wrong
> > request-line, or my squid had some problems ,or anything else? How can =
i
> > fix it?
>
> You have configured Polygraph for a forward proxy test (the --proxy
> option does that). One possibility is that you have also configured your
> Squid to be a reverse proxy. If the two configurations are out of sync,
> bad things like the one you have described may happen.
>
> If you want to test a reverse proxy, you have to change Polygraph
> configuration. Do not use --proxy at all (reverse proxies are origin
> servers from the HTTP protocol point of view). Add a PGL AddrMap that
> maps proxy address (218.202.111.5:8101) to the Polygraph server address
> (S.addresses). And use map.names for Robot's origins field (R.origins)
> instead of S.addresses you are using there now. The WebAxe workload
> configuration distributed with Polygraph has a [more complex] example on
> how to do that.
>
> If you do not want to test a reverse proxy, reconfigure your Squid to be
> a regular forward proxy (starting with squid.conf.default may be a good
> idea).
>
>
> HTH,
>
> Alex.
>
>
--047d7b6d8f167f22ca04f44ec04a
Content-Type: text/html; charset=GB2312
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Hi Alex=A3=AC<div> Thanks for your reply. Yes=
, I has been testing squid using as a reverse proxy. I tried the way of add=
ing <span style=3D"font-family:arial,sans-serif;font-size:14px">PGL Ad=
drMap in my config file, but errors showed up when starting polygraph-clien=
t. My config file is as following:</span></div>
<div><span style=3D"font-family:arial,sans-serif;font-size:14px">----------=
---------------------------------------------------------------------------=
---------------------------------------------</span></div><div><div><font f=
ace=3D"arial, sans-serif"><span style=3D"font-size:14px">ObjLifeCycle olcSt=
atic =3D {</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
birthday =3D now + const(-1year); // born a year ago</span></font><=
/div><div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px">&=
nbsp; length =3D const(2year); &n=
bsp;// two year cycle</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
variance =3D 0%; &=
nbsp; // no variance</span></font></div><div><font face=3D"arial, san=
s-serif"><span style=3D"font-size:14px"> with_lmt =3D 100%; &n=
bsp; // all responses have =
LMT</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
expires =3D [nmt + const(0sec)]; // everything expires when m=
odified</span></font></div><div><font face=3D"arial, sans-serif"><span styl=
e=3D"font-size:14px">};</span></font></div>
<div><br></div><div><font face=3D"arial, sans-serif"><span style=3D"font-si=
ze:14px">Content SimpleContent =3D {</span></font></div><div><font face=3D"=
arial, sans-serif"><span style=3D"font-size:14px"> &nbs=
p; size =3D exp(13000KB); // response sizes distributed exponentially</span=
></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
cachable =3D 100%; // 20% of content is uncach=
able</span></font></div><div><font face=3D"arial, sans-serif"><span style=
=3D"font-size:14px"> obj_life_cycle =3D olcStati=
c;</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px">};</sp=
an></font></div><div><br></div><div><font face=3D"arial, sans-serif"><span =
style=3D"font-size:14px">Server S =3D {</span></font></div><div><font face=
=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
kind =3D "S101";</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
contents =3D [ SimpleContent ];</span></font></div><d=
iv><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> &=
nbsp; direct_access =3D contents;</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"><br></=
span></font></div><div><font face=3D"arial, sans-serif"><span style=3D"font=
-size:14px"> addresses =3D ['218.202.111.138=
:80']; // where to create these server agents</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px">};</sp=
an></font></div><div><font face=3D"arial, sans-serif"><span style=3D"font-s=
ize:14px"><br></span></font></div><div><font face=3D"arial, sans-serif"><sp=
an style=3D"font-size:14px">AddrMap Map =3D {</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
addresses =3D S.addresses;</span></font></div><div><f=
ont face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
names =3D ['218.202.111.5:8101'];</span></font></div=
>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px">};</sp=
an></font></div><div><font face=3D"arial, sans-serif"><span style=3D"font-s=
ize:14px"><br></span></font></div><div><font face=3D"arial, sans-serif"><sp=
an style=3D"font-size:14px">// a primitive robot</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px">Robot =
R =3D {</span></font></div><div><font face=3D"arial, sans-serif"><span styl=
e=3D"font-size:14px"> kind =3D "R101";=
</span></font></div><div><font face=3D"arial, sans-serif"><span style=3D"fo=
nt-size:14px"> pop_model =3D { pop_distr =3D pop=
Unif(); };</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
recurrence =3D 55% / SimpleContent.cachable; // adjus=
ted to get 55% DHR</span></font></div><div><font face=3D"arial, sans-serif"=
><span style=3D"font-size:14px"><br>
</span></font></div><div><font face=3D"arial, sans-serif"><span style=3D"fo=
nt-size:14px"> req_rate =3D 10/sec;</span></font=
></div><div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"=
> origins =3D Map.names; // =
where the origin servers are</span></font></div>
<div><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"> =
addresses =3D ['218.202.111.138' ** 10]; // w=
here these robot agents will be created</span></font></div><div><font face=
=3D"arial, sans-serif"><span style=3D"font-size:14px">};</span></font></div=
>
<div><span style=3D"font-size:14px;font-family:arial,sans-serif"><br></span=
></div><div><span style=3D"font-size:14px;font-family:arial,sans-serif">// =
commit to using these servers and robots</span><br></div><div><font face=3D=
"arial, sans-serif"><span style=3D"font-size:14px">use(S, R);</span></font>=
</div>
</div><div><br></div><div>The error info showed up when i started polygraph=
-clients is as following:</div><div>---------------------------------------=
---------------------------------------------------------------------------=
------------------------------------</div>
<div><div>fyi: PGL configuration stored (664bytes)</div><div>000.00| fyi: a=
ll 10 Robot IP address(es) lack interface name or netmask</div><div>000.00|=
found 10 Robot IP address(es) total (0 unique address(es) with interface n=
ame and netmask).</div>
<div>000.00| fyi: no bench selected with use()</div><div>000.00| ./<a href=
=3D"http://my_simple.pg:41" target=3D"_blank">my_simple.pg:41</a>: error: R=
obot cannot find configuration for <a href=3D"http://[email protected]:8=
101" target=3D"_blank">[email protected]:8101</a></div>
</div><div>I am a newer for polygraph, and can not unstand <span style=
=3D"font-family:arial,sans-serif;font-size:14px">WebAxe workload configurat=
ion c</span><font face=3D"arial, sans-serif"><span style=3D"font-size:14px"=
>ompletely, so i think there are some mistakes in my configuration file. Ca=
n you help me take a look at my config file showed above for which part is =
wrong, and how i should modify it?</span></font></div>
<div><br></div><div> Currently, I just modify the source code =
as following to make it work with my squid server:<br>---------------------=
---------------------------------------------------------------------------=
---------------------------------------</div>
<div><div>ostream &Oid2Url(const ObjId &oid, ostream &os) {</di=
v><div><span class=3D"" style=3D"white-space:pre"> </span>if (oid.foreignUr=
l())</div><div><span class=3D"" style=3D"white-space:pre"> </span>return o=
s << oid.foreignUrl();</div>
<div><br></div><div><span class=3D"" style=3D"white-space:pre"> </span>if (=
oid.secure())</div><div><span class=3D"" style=3D"white-space:pre"> </span=
>os << "https://";</div><div><span class=3D"" style=3D"whit=
e-space:pre"> </span>else</div>
<div>-<span class=3D"" style=3D"white-space:pre"> </span>os << (oid.=
scheme() =3D=3D Agent::pHTTP ? "http://" : "ftp://");</=
div><div>+<span class=3D"" style=3D"white-space:pre"> </span> =
os << (oid.scheme() =3D=3D Agent::pHTTP ? "/"=
: "ftp://");</div>
<div><br></div><div><span class=3D"" style=3D"white-space:pre"> </span>Oid2=
UrlHost(oid, false, os);</div><div><span class=3D"" style=3D"white-space:pr=
e"> </span>Oid2UrlPath(oid, os);</div><div><span class=3D"" style=3D"white-=
space:pre"> </span>return os;</div>
<div>}</div></div><div><br></div><div>I think that is not right way to use =
polygraph for test reverse proxy, so look forward to your reply!</div><div>=
<br></div><div><br></div><div><br></div><div><br></div><div> </div><di=
v>
<br></div><div><br></div><div> </div></div><div class=3D"gmail_extra">=
<br><br><div class=3D"gmail_quote">2014-03-11 0:05 GMT+08:00 Alex Rousskov =
<span dir=3D"ltr"><<a href=3D"mailto:[email protected]" t=
arget=3D"_blank">[email protected]</a>></span>:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div class=3D"">On 03/10/2014 02:02 AM, xu t=
ony wrote:<br>
> I've been using the "Simple tests" introdu=
ed by polygraph's User<br>
> manual to test my squid server. During testing, I found my squid=
can't<br>
> pull the resource from orign(that is polygraph-server) successfully.<b=
r>
<br>
</div><div class=3D"">> 3=A1=A2The command i used for starting polygraph=
-server and polygraph-client<br>
> is ,<br>
> /usr/local/polygraph/bin/polygraph-client --config<br>
</div><div class=3D"">> ../workdir/<a href=3D"http://my_simple.pg" targe=
t=3D"_blank">my_simple.pg</a> --verb_lvl 10 --proxy<br>
> <a href=3D"http://218.202.111.5:8101" target=3D"_blank">218.202.111.5:=
8101</a><br>
<br>
<br>
> During test, i found squid have got requests from<br>
> ploygraph-client,but using a wrong url to pull resource (that is<br>
> polygraph-server).Part of my squid access log is showed as below:<br>
> ----------------------------------------------------------------------=
----------------------------------------<br>
> 218.202.111.138 - - [10/Mar/2014:15:58:49 +0800] "GET<br>
> <a href=3D"http://w1cbc2601.7f1c694f:8/t03/_00000003" target=3D"_blank=
">http://w1cbc2601.7f1c694f:8/t03/_00000003</a> HTTP/1.1" 503 1513 &qu=
ot;-" "-"<br>
> TCP_MISS DIRECT 0 -<br>
<br>
</div>The wrong domain name in the above log implies that your Squid is<br>
confused as to where the requests are going. This would not happen with<br>
a regular Squid configuration. Starting with a simple forward proxy<br>
configuration for Squid may be a good idea before moving on to more<br>
complex stuff (squid.conf.default may work after adding http_access<br>
rules to allow Polygraph traffic).<br>
<div class=3D""><br>
<br>
> I snapped http-request packets from polygraph-client, an=
d found<br>
> request-line of http-request is not what squid expected,which caused t=
he<br>
> squid can't pull resource from polyserver.The request-line in<br>
> http-request sent by polyclient is like that :<br>
> "GET <a href=3D"http://218.202.111.138/w1cbc0feb.0fea1bae:0000000=
8/t03/_00000001" target=3D"_blank">http://218.202.111.138/w1cbc0feb.0fea1ba=
e:00000008/t03/_00000001</a><br>
> HTTP/1.1"<br>
<br>
</div>Yes, this is what you told Polygraph to send. In a forward proxy test=
<br>
that you are running (from Polygraph point of view), the URL should look<br=
>
like the one logged above.<br>
<div class=3D""><br>
<br>
> The right request-line which the squid expected ,i think, shoud be lik=
e:<br>
> "GET /<a href=3D"http://218.202.111.138/w1cbc02ee.0c726ba0:=
00000008/t03/_00000015" target=3D"_blank">218.202.111.138/w1cbc02ee.0c726ba=
0:00000008/t03/_00000015</a> HTTP/1.0"<br>
<br>
</div>No, but if you are testing a reverse proxy, the Request URI that Squi=
d<br>
expects should look like this:<br>
<br>
GET /w1cbc02ee.0c726ba0:00000008/t03/_00000015 HTTP/...<br>
<br>
It would still be logged as a full URL (with a host name) by Squid IIRC,<br=
>
but the packet capture should show from-Polygraph Request URIs without<br>
the protocol scheme and host name components.<br>
<br>
Please note that some packet capture GUIs such as Wireshark sometimes<br>
"restore" the full URL when rendering high-level details of the c=
aptured<br>
packet, even when a full URL is not present in the packet. Make sure you<br=
>
look at what has been actually sent and what what was rendered by GUI.<br>
<div class=3D""><br>
<br>
> So, i want to know which part is wrong in my test, if i h=
ave made<br>
> some mistakes in test, or polyclient sent http-request with wrong<br>
> request-line, or my squid had some problems ,or anything else? How can=
i<br>
> fix it?<br>
<br>
</div>You have configured Polygraph for a forward proxy test (the --proxy<b=
r>
option does that). One possibility is that you have also configured your<br=
>
Squid to be a reverse proxy. If the two configurations are out of sync,<br>
bad things like the one you have described may happen.<br>
<br>
If you want to test a reverse proxy, you have to change Polygraph<br>
configuration. Do not use --proxy at all (reverse proxies are origin<br>
servers from the HTTP protocol point of view). Add a PGL AddrMap that<br>
maps proxy address (<a href=3D"http://218.202.111.5:8101" target=3D"_blank"=
>218.202.111.5:8101</a>) to the Polygraph server address<br>
(S.addresses). And use map.names for Robot's origins field (R.origins)<=
br>
instead of S.addresses you are using there now. The WebAxe workload<br>
configuration distributed with Polygraph has a [more complex] example on<br=
>
how to do that.<br>
<br>
If you do not want to test a reverse proxy, reconfigure your Squid to be<br=
>
a regular forward proxy (starting with squid.conf.default may be a good<br>
idea).<br>
<br>
<br>
HTH,<br>
<br>
Alex.<br>
<br>
</blockquote></div><br></div>
--047d7b6d8f167f22ca04f44ec04a--
--===============1452000669==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Users mailing list
Users-mN9fGWdlm5pku/f+YMZH/di2O/[email protected]
http://www.web-polygraph.org/mailman/listinfo/users
--===============1452000669==--