Re: How to find out which process cause a strange increasing cpu load on a certain core?

On Tue, 18 Jun 2019 18:40:13 +0700
Naruto Nguyen <[email protected]> wrote:

> Hello everyone,
> 
> I have a kvm hypervisor and setup some VMs running OpenSUSE kernel
> 4.0. After some reboot, I saw that sometime there is a VM that a
> certain core has higher %usr and %sys than other cores. The system is
> in ide state. I have some applications running on this VM, but there
> is no issue in other VMs.

How much higher? And is it always an issue with the same core on the
same VM?

> 
> If I enable ftrace in kernel or load any debug jprobe, the load is

I'm assuming by "ftrace" you mean the full function tracer?

 # echo function > /sys/kernel/tracing/current_tracer

> back to normal. The pidstat command output does not show the high load
> on one core like the "top" command. Could you please let me know if it
> is the problem of "top' report higher usage? Any commands that can
> help to troubleshoot this issue? I tried "perf" but most of the time
> the perf output does not show any function that cause the increase as
> well? Not sure, if the malfunction of top or anything else can add
> more work load?

I would suggest just tracing sched switches and interrupts. That
shouldn't affect the load, as it is very low weight.

 # cd /sys/kernel/tracing
 # echo 1 > events/sched/sched_waking/enable
 # echo 1 > events/sched/sched_switch/enable
 # echo 1 > events/irq/enable

And then monitor that.

-- Steve