Re: Something is killing my CPU

HI Dino, nothing making REST calls.  The only out-of-the-ordinary thing I can think of is that I automate 2 provisioning requisitions via an http request that takes quite a while to respond (8 minutes or so).  While I could better optimise that, I don’t see how that could permanently drive up CPU usage for this single thread.

I front the OpenNMS server with nginx for SSL offload – could that make a difference?

I posted the stack trace in response to David’s helpful reply, and if Definitely seems to be something related to the jetty server but I have no idea where to start looking for the root cause.

Gavin

From: Dino Yancey <[email protected]>
Sent: Wednesday, 10 October 2018 02:39
To: General OpenNMS Discussion <[email protected]>
Subject: Re: [opennms-discuss] Something is killing my CPU

Do you have anything that is making a whole bunch of REST calls?

-- 
Gavin Hill  |  Director
m: +27 83 601 8181
t: +27 10 020 5959
   

 
 
On Tue, Oct 9, 2018 at 7:52 AM Gavin Hill <[email protected]<mailto:[email protected]>> wrote:
OK, this looks like one of the culprits, but I don’t know what it means.


  1.  At 2pm you can see CPU clearly jump:

     *   [cid:[email protected]]

  1.  It looks to be this thread:

     *   [cid:[email protected]]

  1.  If I expand the thread all the way down (and boy does it go a long way) I finally get here:

     *   [cid:[email protected]]

Anyone know what this means?!

Gavin

--
Gavin Hill  |  Director
m: +27 83 601 8181<tel:+27%2083%20601%208181>
t: +27 10 020 5959<tel:+27%2010%20020%205959>
  [cid:8cjLIFIEkmMkb5MvDjHgHighpeak_email_jpg] <mailto:%28%21Email%21%29> [cid:JBQaheCdkCralVVZ5VwwHighpeak_linkedin_jpg] <https://www.linkedin.com/company/highpeak-pty-ltd/> [cid:ekXaOF3cEEiklSZaPFLQAHighpeak_skype_jpg] [cid:cYNqlH47C0KFeIXbGKtgawHighpeak_web_jpg] <https://www.highpeak.co.za/>

[cid:wUPJWB9UmUeU1J3UfmznTAHPLogoForSignature_jpg]
From: Gavin Hill <[email protected]<mailto:[email protected]>>
Sent: Tuesday, 09 October 2018 13:32
To: General OpenNMS Discussion <[email protected]<mailto:[email protected]>>
Subject: Re: [opennms-discuss] Something is killing my CPU

Thanks Dino, I have VisualVM up and running and a JMX connection to the OpenNMS box active.

Now what am I looking at/for?!  I have no real clue as to how to spot the culprit?

Gavin


--
Gavin Hill  |  Director
m: +27 83 601 8181<tel:+27%2083%20601%208181>
t: +27 10 020 5959<tel:+27%2010%20020%205959>
  [cid:[email protected]] <mailto:%28%21Email%21%29> [cid:[email protected]] <https://www.linkedin.com/company/highpeak-pty-ltd/> [cid:[email protected]] [cid:[email protected]] <https://www.highpeak.co.za/>


[cid:[email protected]]

From: Dino Yancey <[email protected]<mailto:[email protected]>>
Sent: Monday, 08 October 2018 20:28
To: General OpenNMS Discussion <[email protected]<mailto:[email protected]>>
Subject: Re: [opennms-discuss] Something is killing my CPU

I'd recommend watching the JVM with something like VisualVM ( https://visualvm.github.io/ ) to see which subsystem or thread is spinning out.  Probably easier to follow than a stack trace.

Dino

On Mon, Oct 8, 2018 at 12:01 PM Gavin Hill <[email protected]<mailto:[email protected]>> wrote:
How do I identify which OpenNMS (22.0.4) process is consuming all of my CPU time?  I have a 4-core machine that runs fine after a restart but over time, CPU just grows until it maxes out.



Any thoughts on how I identify what is doing this?  I look in htop and it shows it’s one of the OpenNMS Java threads, but I don’t know how to find out which.  Restarting OpenNMS every few days is not an option for me.

--
Gavin Hill  |  Director
m: +27 83 601 8181<tel:+27%2083%20601%208181>
t: +27 10 020 5959<tel:+27%2010%20020%205959>
  <mailto:%28%21Email%21%29>

<mailto:%28%21Email%21%29>
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss<mailto:%28%21Email%21%29>
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss<mailto:%28%21Email%21%29>

_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss