Re: Is a client robot an OS thread?
Alberto Klocker <alberto.klocker-A5pq3itJMno0n/[email protected]> Mon, 9 Sep 2013 11:40:16 +1000
| Newsgroups | gmane.comp.web.web-polygraph.user |
|---|---|
| Message-ID | <CAP7CNSXaMMTXFRoDGLntZGMZSafCB-vrtvpsv8NACTLEMouB=Q@mail.gmail.com> |
--===============0827971852== Content-Type: multipart/alternative; boundary=089e01184a9295dd9e04e5e979d8 --089e01184a9295dd9e04e5e979d8 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, We run tests on a 10 Gb network environment with Polygraph as one of the benchmarking tools but have found we are hitting 100% CPU usage on both the poly-client and poly-server which would indicate we need to spread the processes and systems out. How well does the reporting engine handle log files from multiple sources? Say I were to script a system that launches two or three instances of the client and server, will the reporting engine be capable of combining the results into one report? Would it be better I write something that "merges" the results together? I'm curious to see what others have done to ramp Polygraph up above its single core limits. We can get 10 Gb throughput with large files but my goal is to find maximum, stable concurrent connections. Any help is much appreciated! Kevin Xie1 <Kevin_Xie1 at symantec.com <http://www.web-polygraph.org/mailman/listinfo/users>> writes: >* Thanks for your quick response, Dmitry.*>**>* However, I don't understan= d how a single thread can simulate multiple*>* concurrent "users" submittin= g request to web servers. Is it that the*>* single thread iterates all open= connections one by one (submitting a*>* request or processing response one= at a time)?* Sort of, though details are more complex than that. If you are interested, search for I/O multiplexing. >* bound by CPU since just 1 core is used at a time, or by slow*>* connecti= ons ..., do I miss something here, or my understanding is*>* fundamentally = wrong?*>** The fact that a single polygraph process can use only a single core may be a limitation indeed. But moving each simulated agent to a separate OS thread is not the solution. You can run multiple Polygraph client or server processes on a single system. But that makes it harder to manage the test and usually requires some auxiliary scripts to start all processes. It would be nice to have SMP support in Polygraph and we have some ideas about it. But so far there has not been enough interest in such project. Patches or sponsorships are welcome. In any case, please do not make conclusions about Polygraph performance from the fact that it is single-threaded. Polygraph is successfully used for high performance tests in 10Gbit networks (though it may require multiple Polygraph processes and/or systems). Regards, Dmitry >* Appreciated for any light shed!*>**>* Kevin*>**>* -----Original Message-= ----*>* From: Dmitry Kurochkin [mailto:dmitry.kurochkin at measurement-fact= ory.com <http://www.web-polygraph.org/mailman/listinfo/users>] *>* Sent: Ma= y-03-12 3:19 PM*>* To: Kevin Xie1*>* Cc: users at web-polygraph.org <http:/= /www.web-polygraph.org/mailman/listinfo/users>*>* Subject: Re: Is a client = robot an OS thread?*>**>* Hi Kevin.*>**>* Kevin Xie1 <Kevin_Xie1 at symante= c.com <http://www.web-polygraph.org/mailman/listinfo/users>> writes:*>**>>*= Hi,*>>**>>* Is a client robot a real OS thread? I'm using web polygrah in = Linux,*>>* but I doesn't see multiple threads in the OS during a test with = 500*>>* robots.*>>**>**>* Polygraph client and server processes are single-= threaded. All Robot*>* and Server agents are running in a single thread.*>= **>* Regards,*>* Dmitry*>**>>* Thanks,*>>* Kevin* *Alberto Klocker* | Software Developer | www.bluereef.com.au *_________________________________________________________________* *D: *+61 3 9895 8006 | *T*: +61 3 9898 8000 | @AlbertoKlocker Please consider the environment before printing this email --089e01184a9295dd9e04e5e979d8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><pre style=3D"color:rgb(0,0,0)">Hi,</pre><pre style=3D"col= or:rgb(0,0,0)">We run tests on a 10 Gb network environment with Polygraph a= s one of the benchmarking tools but have found we are hitting 100% CPU usag= e on both the poly-client and poly-server which would indicate we need to s= pread the processes and systems out.</pre> <pre style=3D"color:rgb(0,0,0)">How well does the reporting engine handle l= og files from multiple sources?=A0</pre><pre style=3D"color:rgb(0,0,0)"><sp= an style=3D"font-family:arial">Say I were to script a system that launches = two or three instances of the client and server, will the reporting engine = be capable of combining the results into one report?</span><br> </pre><pre style=3D"color:rgb(0,0,0)"><span style=3D"font-family:arial">Wou= ld it be better I write something that "merges" the results toget= her?</span></pre><pre style=3D"color:rgb(0,0,0)">I'm curious to see wha= t others have done to ramp Polygraph up above its single core limits.</pre> <pre style=3D"color:rgb(0,0,0)">We can get 10 Gb throughput with large file= s but my goal is to find maximum, stable concurrent connections.</pre><pre = style=3D"color:rgb(0,0,0)">Any help is much appreciated!</pre><pre style=3D= "color:rgb(0,0,0)"> <br></pre><pre style=3D"color:rgb(0,0,0)">Kevin Xie1 <<a href=3D"http://= www.web-polygraph.org/mailman/listinfo/users">Kevin_Xie1 at symantec.com</a= >> writes: ><i> Thanks for your quick response, Dmitry. </i>><i> </i>><i> However, I don't understand how a single thread can simulat= e multiple </i>><i> concurrent "users" submitting request to web servers.= Is it that the </i>><i> single thread iterates all open connections one by one (submitt= ing a </i>><i> request or processing response one at a time)? </i> Sort of, though details are more complex than that. If you are interested, search for I/O multiplexing. ><i> bound by CPU since just 1 core is used at a time, or by slow </i>><i> connections ..., do I miss something here, or my understanding = is </i>><i> fundamentally wrong? </i>><i> </i> The fact that a single polygraph process can use only a single core may be a limitation indeed. But moving each simulated agent to a separate OS thread is not the solution. You can run multiple Polygraph client or server processes on a single system. But that makes it harder to manage the test and usually requires some auxiliary scripts to start all processes. It would be nice to have SMP support in Polygraph and we have some ideas about it. But so far there has not been enough interest in such project. Patches or sponsorships are welcome. In any case, please do not make conclusions about Polygraph performance from the fact that it is single-threaded. Polygraph is successfully used for high performance tests in 10Gbit networks (though it may require multiple Polygraph processes and/or systems). Regards, Dmitry ><i> Appreciated for any light shed! </i>><i> </i>><i> Kevin </i>><i> </i>><i> -----Original Message----- </i>><i> From: Dmitry Kurochkin [mailto:<a href=3D"http://www.web-polygr= aph.org/mailman/listinfo/users">dmitry.kurochkin at measurement-factory.com= </a>]=20 </i>><i> Sent: May-03-12 3:19 PM </i>><i> To: Kevin Xie1 </i>><i> Cc: <a href=3D"http://www.web-polygraph.org/mailman/listinfo/us= ers">users at web-polygraph.org</a> </i>><i> Subject: Re: Is a client robot an OS thread? </i>><i> </i>><i> Hi Kevin. </i>><i> </i>><i> Kevin Xie1 <<a href=3D"http://www.web-polygraph.org/mailman/= listinfo/users">Kevin_Xie1 at symantec.com</a>> writes: </i>><i> </i>>><i> Hi, </i>>><i> </i>>><i> Is a client robot a real OS thread? I'm using web polyg= rah in Linux, </i>>><i> but I doesn't see multiple threads in the OS during a t= est with 500 </i>>><i> robots. </i>>><i> </i>><i> </i>><i> Polygraph client and server processes are single-threaded. All= Robot </i>><i> and Server agents are running in a single thread. </i>><i> </i>><i> Regards, </i>><i> Dmitry </i>><i> </i>>><i> Thanks, </i>>><i> Kevin</i></pre><div><div dir=3D"ltr"><span style=3D"backgro= und-color:rgb(255,255,255);border-collapse:separate;color:rgb(34,34,34);fon= t-size:medium;font-family:Helvetica"><div style=3D"word-wrap:break-word"><s= pan style=3D"border-collapse:separate"><div style=3D"word-wrap:break-word"> <span style=3D"border-collapse:separate"><div style=3D"margin:0cm 0cm 0.000= 1pt;font-size:11pt;font-family:Calibri,sans-serif"><span lang=3D"EN-US" sty= le=3D"font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)"><b= >Alberto Klocker</b>=A0| Software Developer</span><span style=3D"font-size:= 8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)">=A0|=A0</span><spa= n lang=3D"EN-US"><a href=3D"http://www.bluereef.com.au/" style=3D"color:blu= e" target=3D"_blank"><span lang=3D"EN-AU" style=3D"font-size:8pt;font-famil= y:Verdana,sans-serif;color:rgb(54,95,145);text-decoration:none">www.blueree= f.com.au</span></a></span><span lang=3D"EN-US" style=3D"font-size:8pt;color= :rgb(54,95,145)">=A0</span><span style=3D"font-size:8pt;color:rgb(54,95,145= )"></span></div> <div style=3D"margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sa= ns-serif"><u><span style=3D"font-size:8pt;color:rgb(54,95,145)">___________= ______________________________________________________</span></u></div></sp= an></div> </span></div></span><span style=3D"background-color:rgb(255,255,255);color:= rgb(34,34,34);font-size:15px;font-family:Calibri,sans-serif"><span lang=3D"= EN-US" style=3D"font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,9= 5,145)"><b>D:=A0</b><a value=3D"+61398958020" style=3D"color:rgb(17,85,204)= ">+61 3 9895 8006</a>=A0|=A0</span></span><span style=3D"background-color:r= gb(255,255,255);border-collapse:separate;color:rgb(34,34,34);font-size:medi= um;font-family:Helvetica"><span style=3D"font-size:15px;font-family:Calibri= ,sans-serif"><b><span lang=3D"EN-US" style=3D"font-size:8pt;font-family:Ver= dana,sans-serif;color:rgb(54,95,145)">T</span></b></span><span style=3D"fon= t-size:15px;font-family:Calibri,sans-serif"><span style=3D"font-size:8pt;fo= nt-family:Verdana,sans-serif;color:rgb(54,95,145)">:=A0<a value=3D"+6139898= 8000" style=3D"color:rgb(17,85,204)">+61 3 9898 8000</a>=A0</span></span><s= pan style=3D"font-size:15px;font-family:Calibri,sans-serif"><span style=3D"= font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)">|=A0</sp= an></span><a value=3D"+61398988000" style=3D"font-family:Verdana,sans-serif= ;font-size:11px;color:rgb(17,85,204)">@AlbertoKlocker</a><div style=3D"word= -wrap:break-word"> <span style=3D"border-collapse:separate"><div style=3D"word-wrap:break-word= "><span style=3D"border-collapse:separate"><span style=3D"border-collapse:s= eparate"><span style=3D"font-family:'Lucida Grande'"><div style=3D"= margin:0cm 0cm 0.0001pt;font-size:12pt;font-family:'Times New Roman'= ;,serif"> <span style=3D"border-collapse:separate;font-size:medium;font-family:Calibr= i"><span style=3D"font-family:'Times New Roman',serif;font-size:16p= x"><span style=3D"border-collapse:separate;font-size:medium;font-family:Cal= ibri"><div style=3D"margin:0cm 0cm 0.0001pt;font-size:12pt;font-family:'= ;Times New Roman',serif"> <span lang=3D"EN-US" style=3D"font-size:8pt;font-family:Verdana,sans-serif;= color:rgb(30,147,102)">Please consider the environment before printing this= email</span></div></span></span></span></div></span></span></span></div></= span></div> </span></div></div> </div> --089e01184a9295dd9e04e5e979d8-- --===============0827971852== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Users mailing list Users-mN9fGWdlm5pku/f+YMZH/di2O/[email protected] http://www.web-polygraph.org/mailman/listinfo/users --===============0827971852==--