9

man ksoftirqd indicates that:

If ksoftirqd is taking more than a tiny percentage of CPU time, this indicates the machine is under heavy soft interrupt load.

I'm working with a Debian Wheezy system under generally high system utilization in which ksoftirqd processes utilizes excessive cpu and disk resources for a short period of time. During that time, the system operates at a snails pace.

How can one begin to understand what the root cause is for this ksoftirqd resource utilization spikes?

mrP
  • 225
  • 1
  • 3
  • 8

2 Answers2

6

Check /proc/interrupts to find if one of or more interrupts occur excessively. Hint: Several thousand interrupts per second are no cause for alarm.

Excessive interrupts (aka interrupt storms) can have multiple reasons, one of them even being hardware issues (noisy interrupt line).

To further answer your question we need to know what OS on what hardware you use.

Jan
  • 7,600
  • 2
  • 34
  • 41
  • it looks like eth0 is the main culprit (not sure how to see on a per second basis though) -- CPU0 76739293 eth0-TxRx-0 – mrP Oct 11 '15 at 21:29
  • And 15 minutes later -- CPU0 77513350 eth0-TxRx-0. – mrP Oct 11 '15 at 21:35
  • ((77513350 - 76739293) / 15) / 60 = 860 interrupts / second - IMHO no cause for alarm, if you're using the network heavily. I don't think your machine has an issue at all, processing interrupts is a normal behaviour of your system... – Jan Oct 12 '15 at 06:40
  • what hardware info would be helpful here? – mrP Oct 12 '15 at 14:52
  • Holy cow Batman, how many cores are in there? – Jan Oct 12 '15 at 19:13
  • Ha, 48 cores (2x E5-2690v3) – mrP Oct 12 '15 at 20:36
0

Beside monitoring /proc/interrupts and /proc/softirqs - and checking the CPUx column that matches the affinity of the very active ksoftirqd/x kthread(s) you are seeing - you can look into it in more detail via running perf record for a short time.

Example:

perf record -C 6 -g -- sleep 30
perf report

Again, use -C 6 if ksoftirqd/6 has high CPU usage, otherwise specify another CPU core.

In theory, you could also use perf top, but sometimes I get better/more accurate results with perf record.

maxschlepzig
  • 56,316
  • 50
  • 205
  • 279