how to detect if isolcpus is activated?

Question

How to detect if isolcpus is activated and on which cpus, when for example you connect for the first time on a server. Conditions:

not spawning any process to see where it will be migrated.

The use case is that isolcpus=1-7 on a 6 cores i7, seems to not activate isolcpus at boot, and i would like to know if its possible from /proc/, /sys or any kernel internals which can be read in userspace, to provide a clear status of activation of isolcpus and which cpu are concerned. Or even read active setting of the scheduler which is the first concerned by isolcpus.

Consider the uptime is so big, that dmesg is no more displaying boot log to detect any error at startup. Basic answer like "look at kernel cmd line" will not be accepted :)

Can you use `pidstat -C isolcpus`. pidstat is from the `sysstat` package. — Timothy Pulliam, Jan 09 '17 at 15:46
May be im dumb, but i don't see any link between `pidstat` and `isolcpus`. Can you elaborate a little more ? — netmonk, Jan 09 '17 at 15:53
You say you need to know what CPUs the command is running on. I don't know what `isolcpus` is, but `pidstat` can tell you what CPU a process is running on if you pass the `-C process_name` flag. For example `pidstat -C top` produces the following. `10:56:52 AM PID %usr %system %guest %CPU CPU Command 10:56:52 AM 3457 0.00 0.00 0.00 0.00 6 top` — Timothy Pulliam, Jan 09 '17 at 15:57
Ok thank you for your reply. `isolcpus` is a kernel parameter wich forces the scheduler to not migrate any process by itself if they are isolated. For exemple on a 8 cpu servers, `isolcpus=1-7` in the kernel command line, will force all spawned process forked by init and any kind of command in shell to run only on CPU0. To run a task on an isolated cpu, you need to launch it with `taskset` for exemple. So in regard of what i am asking, it seems your answer is out of topic. Thank anyway for trying to answer me — netmonk, Jan 09 '17 at 15:58
You didnt read carefully the question: `Consider the uptime is so big, that dmesg is no more displaying boot log to detect any error at startup. Basic answer like "look at kernel cmd line" will not be accepted ` — netmonk, Sep 22 '19 at 17:26

score 26 · Accepted Answer · edited Jun 12 '19 at 09:50

What you look for should be found inside this virtual file:

/sys/devices/system/cpu/isolated

and the reverse in

/sys/devices/system/cpu/present    // Thanks to John Zwinck

From drivers/base/cpu.c we see that the source displayed is the kernel variable cpu_isolated_map:

static ssize_t print_cpus_isolated(struct device *dev,
    n = scnprintf(buf, len, "%*pbl\n", cpumask_pr_args(cpu_isolated_map));
...
static DEVICE_ATTR(isolated, 0444, print_cpus_isolated, NULL);

and cpu_isolated_map is exactly what gets set by kernel/sched/core.c at boot:

/* Setup the mask of cpus configured for isolated domains */
static int __init isolated_cpu_setup(char *str)
{
    int ret;

    alloc_bootmem_cpumask_var(&cpu_isolated_map);
    ret = cpulist_parse(str, cpu_isolated_map);
    if (ret) {
            pr_err("sched: Error, all isolcpus= values must be between 0 and %d\n", nr_cpu_ids);
            return 0;
    }
    return 1;
}

But as you observed, someone could have modified the affinity of processes, including daemon-spawned ones, cron, systemd and so on. If that happens, new processes will be spawned inheriting the modified affinity mask, not the one set by isolcpus.

So the above will give you isolcpus as you requested, but that might still not be helpful.

Supposing that you find out that isolcpus has been issued, but has not "taken", this unwanted behaviour could be derived by some process realizing that it is bound to only CPU=0, believing it is in monoprocessor mode by mistake, and helpfully attempting to "set things right" by resetting the affinity mask. If that was the case, you might try and isolate CPUS 0-5 instead of 1-6, and see whether this happens to work.

Tested on Centos 7.x, note that ```/sys/devices/system/cpu/possible``` doesn't appear to be the 'reverse' depending on how you interpret 'reverse'. E.g. if ```.../cpu/isolated``` returns ```2,4``` because they had been isolated, ```.../cpu/possible``` would return ```0-191```. — bgura, Jan 29 '18 at 22:54
I think most people would want `/sys/devices/system/cpu/present` which shows which CPUs exist, rather than `/sys/devices/system/cpu/possible` which shows which CPUs could possibly exist (but may not currently exist). On some systems they are the same, but even on a pretty basic desktop I checked, they are not. — John Zwinck, Mar 06 '19 at 07:20
With isolcpus=0 and 4 cores, I'm getting /isolated=0 and /present=0-3. — Stefan Reich, Sep 30 '19 at 21:20
`/present` doesn't work as intended - on my system (Ubuntu 20.04/Linux 5.8), I get the whole list of processors (0-31 on a 16C/32T CPU). — Marcus, Apr 05 '21 at 09:35

Rui F Ribeiro · Answer 2 · 2018-03-17T09:30:59.237

13

One of the easier ways to detect if isolcpus is consulting proc to see which parameters were passed to the kernel in runtime.

For that, you would use:

$cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-4.8.0-1-amd64 root=/dev/sda1 ro isolcpus=2,3 quiet

As you can see, in this particular example isolcpus=2,3was passed as an argument to the running kernel.

You can also use taskset pointed to PID 1. As PID 1 is the standard PID for the first task launched by the kernel, we can take as a pretty good indication that it will reflect whether we have isolcpus working. As in:

$taskset -cp 1
pid 1's current affinity list: 0,1

Comparing with the lscpu command in the same server:

$lscpu | grep CPU.s
CPU(s):                4
On-line CPU(s) list:   0-3
NUMA node0 CPU(s):     0-3

As it can be seen, lscpu is showing 4 CPU/cores, while taskset is only showing 0,1, so this shows isolcpus is working here.

Have a look to at: How to ensure exclusive CPU availability for a running process?

edited Mar 17 '18 at 09:30

answered Jan 09 '17 at 16:56

Rui F Ribeiro

55,929
26
146
227

Thank you sir for you answer, but i won't accept it. i specified that answer talking about the kernel command line won't be accepted. i obviously know about `/proc/cmdline` which is the exact copy of the content of `grub.conf` file. This is not what i asked ! Thank you anyway ! – netmonk Jan 09 '17 at 22:23
I have added to the answer. – Rui F Ribeiro Jan 10 '17 at 19:26
1

What is someone previously modified the affinity of processus with taskset ? what if someone sets init affinity to cpu 0 only, does it gives me any information about activation or not of isol cpu ? – netmonk Jan 13 '17 at 14:00
1

This helped me realise my isolcpus wasn't set. – user997112 Aug 01 '22 at 18:00

score 2 · Answer 3 · answered Jan 18 '17 at 17:00

2

You can check Cpus_allowed and Cpus_allowed_list for current shell process to see what cpus were reserved

cat /proc/$$/status|tail -6

for eg

Cpus_allowed_list:      0-1, 3-5

means that the cpu=2 was reserved by isolcpus on a 6 cpus server

answered Jan 18 '17 at 17:00

valentin

131
5

2

Actually... no. It shows what CPUs are in the affinity mask **inherited** by the current shell process. If it was, say, `ssh`, and one had manually limited the affinity of the parent `sshd` to CPU 1, you would see always CPU 1, whether `isolcpus` had "taken" or not. The `$$` branch gives you the process inherited values, not the original system ones. – LSerni Jan 18 '17 at 18:07
"useless use of cat" here – Vladimir Kunschikov May 18 '23 at 10:21

how to detect if isolcpus is activated?

3 Answers3