MRTG check scheduling while in daemon mode
"Steve Shipway" <[email protected]> Mon, 4 Oct 2010 11:18:26 +1300
| Newsgroups | gmane.network.mrtg.devel |
|---|---|
| Organization | Cheshire Cat Computing |
| Message-ID | <000001cb6348$ee9b9580$cbd2c080$@org> |
This is a multi-part message in MIME format. --===============0748409271== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0001_01CB63B5.E594DD80" Content-Language: en-nz This is a multi-part message in MIME format. ------=_NextPart_000_0001_01CB63B5.E594DD80 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Has anyone on the list any thoughts on the MRTG check scheduler? Currently (we're considering Daemon mode only, here), every 5 mins it will run ALL the Target checks sequentially, running multiple threads according to the Forks: directive. After all checks are finished, it will sleep until the next 5-min cycle starts. This is sub-optimal because 1) You get a huge burst of CPU usage followed by a period of silence, which can make the frontend go slow and messes up monitoring of the system's own CPU 2) If the checks exceed the 5min window, then you miss a polling cycle and need to manually tune your forks upwards. I would propose an alternative method of scheduling. 1. Rather than specifying a number of forks, make it a MAXIMUM number (a bit like when defining threads in apache) 2. After the initial read of the CFG files, MRTG knows how many Targets there are. Divide the Interval by this to get the interleave. Then, start a new check every interleave, starting a new thread if necessary and if we've not hit the maximum threads. Benefits would be that it can expand to handle more targets, and spreads the load over the window. Disadvantages would be that it is hard to tell when you're reaching capacity, and (more importantly) it might be hard to do the optimisation that MRTG does where a single device is queried once for all interfaces. We coded up basically this system here, however it didn't use MRTG in daemon mode which negates a lot of the benefits you can gain from daemon mode and the new RRD memory-mapped IO. I've not yet looked at coding it directly into the MRTG code. Anyone have any thoughts? Steve _____ Steve Shipway [email protected] Routers2.cgi web frontend for MRTG/RRD; NagEventLog Nagios agent for Windows Event Log monitoring; check_vmware plugin for VMWare monitoring in Nagios and MRTG; and other Open Source projects. Web: http://www.steveshipway.org/software P Please consider the environment before printing this e-mail ------=_NextPart_000_0001_01CB63B5.E594DD80 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3DContent-Type content=3D"text/html; = charset=3Dus-ascii"> <meta name=3DGenerator content=3D"Microsoft Word 12 (filtered medium)"> <!--[if !mso]> <style> v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);} .shape {behavior:url(#default#VML);} </style> <![endif]--> <style> <!-- /* Font Definitions */ @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4;} @font-face {font-family:Webdings; panose-1:5 3 1 2 1 5 9 6 7 3;} @font-face {font-family:"Arial Narrow"; panose-1:2 11 6 6 2 2 2 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif";} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph {mso-style-priority:34; margin-top:0cm; margin-right:0cm; margin-bottom:0cm; margin-left:36.0pt; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif";} span.EmailStyle17 {mso-style-type:personal-compose; font-family:"Calibri","sans-serif"; color:windowtext;} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.WordSection1 {page:WordSection1;} /* List Definitions */ @list l0 {mso-list-id:734855438; mso-list-type:hybrid; mso-list-template-ids:-79908856 336134159 336134169 336134171 336134159 = 336134169 336134171 336134159 336134169 336134171;} @list l0:level1 {mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt;} @list l1 {mso-list-id:1611275509; mso-list-type:hybrid; mso-list-template-ids:-595166012 336134161 336134169 336134171 = 336134159 336134169 336134171 336134159 336134169 336134171;} @list l1:level1 {mso-level-text:"%1\)"; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt;} ol {margin-bottom:0cm;} ul {margin-bottom:0cm;} --> </style> <!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--> </head> <body lang=3DEN-NZ link=3Dblue vlink=3Dpurple> <div class=3DWordSection1> <p class=3DMsoNormal>Has anyone on the list any thoughts on the MRTG = check scheduler?<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>Currently (we’re considering Daemon mode = only, here), every 5 mins it will run ALL the Target checks sequentially, running = multiple threads according to the Forks: directive. After all checks are = finished, it will sleep until the next 5-min cycle starts.<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>This is sub-optimal because<o:p></o:p></p> <p class=3DMsoListParagraph style=3D'text-indent:-18.0pt;mso-list:l1 = level1 lfo1'><![if !supportLists]><span style=3D'mso-list:Ignore'>1)<span style=3D'font:7.0pt "Times New = Roman"'> </span></span><![endif]>You get a huge burst of CPU usage followed by a = period of silence, which can make the frontend go slow and messes up monitoring = of the system’s own CPU<o:p></o:p></p> <p class=3DMsoListParagraph style=3D'text-indent:-18.0pt;mso-list:l1 = level1 lfo1'><![if !supportLists]><span style=3D'mso-list:Ignore'>2)<span style=3D'font:7.0pt "Times New = Roman"'> </span></span><![endif]>If the checks exceed the 5min window, then you = miss a polling cycle and need to manually tune your forks = upwards.<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>I would propose an alternative method of = scheduling.<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoListParagraph style=3D'text-indent:-18.0pt;mso-list:l0 = level1 lfo2'><![if !supportLists]><span style=3D'mso-list:Ignore'>1.<span style=3D'font:7.0pt "Times New = Roman"'> </span></span><![endif]>Rather than specifying a number of forks, make = it a MAXIMUM number (a bit like when defining threads in = apache)<o:p></o:p></p> <p class=3DMsoListParagraph style=3D'text-indent:-18.0pt;mso-list:l0 = level1 lfo2'><![if !supportLists]><span style=3D'mso-list:Ignore'>2.<span style=3D'font:7.0pt "Times New = Roman"'> </span></span><![endif]>After the initial read of the CFG files, MRTG = knows how many Targets there are. Divide the Interval by this to get the interleave. Then, start a new check every interleave, starting a = new thread if necessary and if we’ve not hit the maximum = threads.<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>Benefits would be that it can expand to handle more = targets, and spreads the load over the window.<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>Disadvantages would be that it is hard to tell when = you’re reaching capacity, and (more importantly) it might be hard to do the optimisation that MRTG does where a single device is queried once for = all interfaces.<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>We coded up basically this system here, however it = didn’t use MRTG in daemon mode which negates a lot of the benefits you can gain = from daemon mode and the new RRD memory-mapped IO. I’ve not yet = looked at coding it directly into the MRTG code.<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>Anyone have any thoughts?<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <p class=3DMsoNormal>Steve<o:p></o:p></p> <p class=3DMsoNormal><o:p> </o:p></p> <div class=3DMsoNormal align=3Dcenter style=3D'text-align:center'><span = lang=3DEN-US> <hr size=3D2 width=3D"100%" align=3Dcenter> </span></div> <p class=3DMsoNormal><b>Steve Shipway<o:p></o:p></b></p> <p class=3DMsoNormal><span = style=3D'font-size:10.0pt'>[email protected]<o:p></o:p></span></p> <p class=3DMsoNormal><i><span = style=3D'font-size:10.0pt;color:#595959'>Routers2.cgi web frontend for MRTG/RRD; NagEventLog Nagios agent for Windows Event = Log monitoring; check_vmware plugin for VMWare monitoring in Nagios and = MRTG; and other Open Source projects.<o:p></o:p></span></i></p> <p class=3DMsoNormal><span style=3D'font-size:10.0pt'>Web: <a href=3D"http://www.steveshipway.org/software">http://www.steveshipway.org= /software</a><o:p></o:p></span></p> <p class=3DMsoNormal><span lang=3DEN-GB = style=3D'font-size:18.0pt;font-family:Webdings; color:green'>P</span><span lang=3DEN-GB style=3D'color:blue'> = </span><span lang=3DEN-GB style=3D'font-size:10.0pt;font-family:"Arial = Narrow","sans-serif"; color:green'>Please consider the environment before printing this = e-mail</span><span lang=3DEN-GB style=3D'color:blue'> </span><span lang=3DEN-GB = style=3D'font-size:7.5pt; font-family:"Verdana","sans-serif";color:navy'><o:p></o:p></span></p> <p class=3DMsoNormal><span = style=3D'font-size:10.0pt'><o:p> </o:p></span></p> <p class=3DMsoNormal><o:p> </o:p></p> </div> </body> </html> ------=_NextPart_000_0001_01CB63B5.E594DD80-- --===============0748409271== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ mrtg-developers mailing list [email protected] https://lists.oetiker.ch/cgi-bin/listinfo/mrtg-developers --===============0748409271==--