Re: Splunk Integration Question...
"Frost, Mark {BIS}" <[email protected]> Tue, 10 Sep 2013 19:12:43 +0000
| Newsgroups | gmane.network.nagios.user |
|---|---|
| Message-ID | <[email protected]> |
--===============4293996167959315872== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_3FC36F2C7B96D444A1138066E952D2B60DFB97PEPWMC00171corppe_" --_000_3FC36F2C7B96D444A1138066E952D2B60DFB97PEPWMC00171corppe_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Huh. Where did those new options come from? They weren't in the cgi.cfg = docs the last time I looked :). I agree, it's not terribly clear to me what that option does, but it does r= eference "Splunk IT" which is a special Splunk package that you can use for= Splunk benchmarking. That still doesn't make it clear what it's used for= . I see a second parameter, "splunk_url" that lets you specify the URL for yo= ur Splunk server. Maybe it just somehow says to pepper the logs with your Splunk URL in appro= priate places. Mark From: Sean Alderman [mailto:[email protected]] Sent: Tuesday, September 10, 2013 1:34 PM To: Nagios Users List Subject: Re: [Nagios-users] Splunk Integration Question... Just what's in the nagios doc on CGI.cfg. The doc is lacking about what it = does, so I guess I'm a little curious what that config is about. - Sean Alderman Senior Engineer, UDit Systems Integration This message had been brought to you by Android Bionic. On Sep 10, 2013 1:10 PM, "Frost, Mark {BIS}" <[email protected]<mailt= o:[email protected]>> wrote: Sean, Can you describe what you're doing for Splunk integration with Nagios? I'= ve used Splunk with Nagios in a couple different ways, but I'm not aware of= any single standard for doing so. Originally, I just had Splunk run a scheduled search, which would trigger a= script which sent a passive check result back to a Nagios service via NSCA= . That way - having Nagios process passive check results from Splunk - wa= s the only way I could see to do that. Recently, I played around a bit with writing scripts that made use of Splun= k's REST API so the checks could be run as active checks from Nagios. (I a= lways prefer active checks). I set this up for only one check, but once I= got it working it worked pretty well. As a side note, I'm still a little on the fence about whether or not I real= ly want to have Nagios find problems through Splunk and then alert on them = or have Splunk find an alert on them directly without using Nagios at all..= . Are you referring to another way of making Splunk and Nagios talk together? Mark From: Sean Alderman [mailto:[email protected]<mailto:salderman1@udayto= n.edu>] Sent: Monday, September 09, 2013 1:12 PM To: [email protected]<mailto:[email protected]= e.net> Subject: [Nagios-users] Splunk Integration Question... Greetings, I was hoping I might find someone who's got the splunk integration active= ly working. I'm running Nagios Core (via EPEL) and Splunk 5.0.3 on OracleL= inux 6.4. When I edit cgi.cfg and enable splunk integration, then set the splunk U= RL to https://<mysplunkserver>:8000/en-US/app/search/flastimeline<https://%= 3cmysplunkserver%3e:8000/en-US/app/search/flastimeline>, I notice the nagio= s URLs look like: https://<mysplunkserver>:8000/en-US/app/flashtimeline?q= =3Dsearch%20test1.udayton.edu<http://20test1.udayton.edu>%20<nagios plugin = check>. I have two questions... * Is there a way I can make nagios use the hostname only, not the F= QDN? We use short names in splunk so we don't a mix of fqdn and short name= s since we use both forwarders and syslog as input. * What data is this query looking for, is it expected that I should= have my nagios log in splunk? The <nagios plugin check> in the query does= n't seem useful to me, unless there's splunk data specifically tied to that= check, and I'm hoping someone could provide an example. Kind regards, -- Sean M. Alderman Senior Engineer, UDit Systems Integration and Engineering University of Dayton ---------------------------------------------------------------------------= --- How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=3D51271111&iu=3D/4140/ostg.cl= ktrk _______________________________________________ Nagios-users mailing list [email protected]<mailto:[email protected]= t> https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reportin= g any issue. ::: Messages without supporting info will risk being sent to /dev/null --_000_3FC36F2C7B96D444A1138066E952D2B60DFB97PEPWMC00171corppe_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= > <meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)"> <style><!-- /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0;} @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman","serif";} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} p {mso-style-priority:99; mso-margin-top-alt:auto; margin-right:0in; mso-margin-bottom-alt:auto; margin-left:0in; font-size:12.0pt; font-family:"Times New Roman","serif";} p.MsoAcetate, li.MsoAcetate, div.MsoAcetate {mso-style-priority:99; mso-style-link:"Balloon Text Char"; margin:0in; margin-bottom:.0001pt; font-size:8.0pt; font-family:"Tahoma","sans-serif";} span.EmailStyle18 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} span.BalloonTextChar {mso-style-name:"Balloon Text Char"; mso-style-priority:99; mso-style-link:"Balloon Text"; font-family:"Tahoma","sans-serif";} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri","sans-serif";} @page WordSection1 {size:8.5in 11.0in; margin:1.0in 1.0in 1.0in 1.0in;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--> </head> <body lang=3D"EN-US" link=3D"blue" vlink=3D"purple"> <div class=3D"WordSection1"> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">Huh. Where di= d those new options come from? They weren’t in the cgi.cfg docs= the last time I looked </span><span style=3D"font-size:11.0pt;font-family:Wingdings;color:#1F497D"= >J</span><span style=3D"font-size:11.0pt;font-family:"Calibri",&q= uot;sans-serif";color:#1F497D">.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D"><o:p> </o:p></span><= /p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">I agree, it’s not t= erribly clear to me what that option does, but it does reference “Spl= unk IT” which is a special Splunk package that you can use for Splunk benchmarking. That still doesn’t make it clear what it&#= 8217;s used for.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D"><o:p> </o:p></span><= /p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">I see a second parameter,= “splunk_url” that lets you specify the URL for your Splunk ser= ver.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D"><o:p> </o:p></span><= /p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">Maybe it just somehow say= s to pepper the logs with your Splunk URL in appropriate places.<o:p></o:p>= </span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D"><o:p> </o:p></span><= /p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">Mark<o:p></o:p></span></p= > <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D"><o:p> </o:p></span><= /p> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><b><span style=3D"font-si= ze:10.0pt;font-family:"Tahoma","sans-serif"">From:</spa= n></b><span style=3D"font-size:10.0pt;font-family:"Tahoma","= sans-serif""> Sean Alderman [mailto:[email protected]] <br> <b>Sent:</b> Tuesday, September 10, 2013 1:34 PM<br> <b>To:</b> Nagios Users List<br> <b>Subject:</b> Re: [Nagios-users] Splunk Integration Question...<o:p></o:p= ></span></p> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p> </o:p></p> <p style=3D"margin-left:.5in">Just what's in the nagios doc on CGI.cfg. The= doc is lacking about what it does, so I guess I'm a little curious what th= at config is about.<o:p></o:p></p> <p style=3D"margin-left:.5in">- Sean Alderman <br> Senior Engineer, UDit Systems Integration<o:p></o:p></p> <p style=3D"margin-left:.5in">This message had been brought to you by Andro= id Bionic.<o:p></o:p></p> <div> <p class=3D"MsoNormal" style=3D"margin-left:.5in">On Sep 10, 2013 1:10 PM, = "Frost, Mark {BIS}" <<a href=3D"mailto:[email protected]= ">[email protected]</a>> wrote:<o:p></o:p></p> <div> <div> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D">Sean,</span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D"> </span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D">Can you describe what you’re doing for Spl= unk integration with Nagios? I’ve used Splunk with Nagios= in a couple different ways, but I’m not aware of any single standard= for doing so.</span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D"> </span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D">Originally, I just had Splunk run a scheduled se= arch, which would trigger a script which sent a passive check result back t= o a Nagios service via NSCA. That way – having Nagios process passive check results from Splunk – was the only way I could= see to do that.</span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D"> </span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D">Recently, I played around a bit with writing scr= ipts that made use of Splunk’s REST API so the checks could be run as= active checks from Nagios. (I always prefer active checks). &nb= sp; I set this up for only one check, but once I got it working it worked pret= ty well.</span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D"> </span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D">As a side note, I’m still a little on the = fence about whether or not I really want to have Nagios find problems throu= gh Splunk and then alert on them or have Splunk find an alert on them directly without using Nagios at all…</span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D"> </span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D">Are you referring to another way of making Splun= k and Nagios talk together?</span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D"> </span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D">Mark</span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:.5in"> <span style=3D"font-size:11.0pt;font-family:"Calibri","sans-= serif";color:#1F497D"> </span><o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:1.0in"> <b><span style=3D"font-size:10.0pt;font-family:"Tahoma","san= s-serif"">From:</span></b><span style=3D"font-size:10.0pt;font-family:= "Tahoma","sans-serif""> Sean Alderman [mailto:<a href= =3D"mailto:[email protected]" target=3D"_blank">[email protected]= </a>] <br> <b>Sent:</b> Monday, September 09, 2013 1:12 PM<br> <b>To:</b> <a href=3D"mailto:[email protected]" target=3D"= _blank">[email protected]</a><br> <b>Subject:</b> [Nagios-users] Splunk Integration Question...</span><o:p></= o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:1.0in"> <o:p></o:p></p> <div> <div> <div> <div> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:1.0in"> Greetings,<o:p></o:p></p> </div> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;margin-bottom:12.0p= t;margin-left:1.0in"> I was hoping I might find someone who's got the splunk integration a= ctively working. I'm running Nagios Core (via EPEL) and Splunk 5.0.3 = on OracleLinux 6.4.<o:p></o:p></p> </div> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:1.0in"> When I edit cgi.cfg and enable splunk integration, then set th= e splunk URL to <a href=3D"https://%3cmysplunkserver%3e:8000/en-US/app/sear= ch/flastimeline" target=3D"_blank"> https://<mysplunkserver>:8000/en-US/app/search/flastimeline</a>, I no= tice the nagios URLs look like: https://<mysplunkserver>:8000/en-US/a= pp/flashtimeline?q=3Dsearch%<a href=3D"http://20test1.udayton.edu" target= =3D"_blank">20test1.udayton.edu</a>%20<nagios plugin check>. I have two questions...<o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:1.5in"> <span style=3D"font-size:10.0pt;font-family:Symbol">·</span><span st= yle=3D"font-size:7.0pt"> </span>Is there a way I can make nagios use the hostname only, not the FQDN= ? We use short names in splunk so we don't a mix of fqdn and short na= mes since we use both forwarders and syslog as input.<o:p></o:p></p> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:1.5in"> <span style=3D"font-size:10.0pt;font-family:Symbol">·</span><span st= yle=3D"font-size:7.0pt"> </span>What data is this query looking for, is it expected that I should ha= ve my nagios log in splunk? The <nagios plugin check> in the qu= ery doesn't seem useful to me, unless there's splunk data specifically tied= to that check, and I'm hoping someone could provide an example.<br clear=3D"all"> <o:p></o:p></p> </div> <div> <div> <div> <div> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:1.0in"> Kind regards,<o:p></o:p></p> </div> <div> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a= lt:auto;margin-left:1.0in"> -- <o:p></o:p></p> <div> <p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;margin-bottom:12.0p= t;margin-left:1.0in"> Sean M. Alderman<br> Senior Engineer, UDit Systems Integration and Engineering<br> University of Dayton<o:p></o:p></p> </div> </div> </div> </div> </div> </div> </div> </div> <p class=3D"MsoNormal" style=3D"margin-left:.5in"><br> ---------------------------------------------------------------------------= ---<br> How ServiceNow helps IT people transform IT departments:<br> 1. Consolidate legacy IT systems to a single system of record for IT<br> 2. Standardize and globalize service processes across IT<br> 3. Implement zero-touch automation to replace manual, redundant tasks<br> <a href=3D"http://pubads.g.doubleclick.net/gampad/clk?id=3D51271111&iu= =3D/4140/ostg.clktrk" target=3D"_blank">http://pubads.g.doubleclick.net/gam= pad/clk?id=3D51271111&iu=3D/4140/ostg.clktrk</a><br> _______________________________________________<br> Nagios-users mailing list<br> <a href=3D"mailto:[email protected]">[email protected]= urceforge.net</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/nagios-users" targe= t=3D"_blank">https://lists.sourceforge.net/lists/listinfo/nagios-users</a><= br> ::: Please include Nagios version, plugin version (-v) and OS when reportin= g any issue.<br> ::: Messages without supporting info will risk being sent to /dev/null<o:p>= </o:p></p> </div> </div> </body> </html> --_000_3FC36F2C7B96D444A1138066E952D2B60DFB97PEPWMC00171corppe_-- --===============4293996167959315872== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk --===============4293996167959315872== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Nagios-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nagios-users ::: Please include Nagios version, plugin version (-v) and OS when reporting any issue. ::: Messages without supporting info will risk being sent to /dev/null --===============4293996167959315872==--