Which process (kernel thread) is doing the actual compression for zswap?

Question

I can imagine actually two locations:

1. In the kernel space belonging to the process whose ram is being swapped in/out
2. From [kswapd0]

However, digging into the kswapd source (mm/vmscan.c, init/main.c), I could find: kswapd is single-threaded, and is started on a single thread. (Except on NUMA systems, where all the memory regions have a different kswapd. But most ordinary PCs are not NUMA systems.)

However, from now we have a problem. We can assume, that the disk is far slower than the memory, this is why we don't need a multi-threaded kswapd to handle the disk I/O. But not this is the case if we need to utilize also the internal zswap layer. Particularly from stronger compression rates (deflate), cpu can, and likely will be a bottleneck.

But kswapd is single-threaded.

Is it true?

Is being any multi-threaded kswapd planned? Is it really needed?

---

P.s. I found this thread on the linux kernel mailing list. It is about a rejected patch suggestion, what could have enabled multi-threaded kswapd on non-NUMA systems. They are talking about everything, except this zswap problem. Maybe it is unrelated.

P.s.2. Context:

1. I have a highly ram-overcommitted Linux system (processes are using far more ram than physically is available).
2. The count of the simultanously running processes is far lower than the CPU cores.
3. I am using intensively zswap.
4. In this environment, it would highly useful to use all the available CPU cores for compress/decompress memory pages. My current best estimation is that the page compression/decompression is being done by [kswapd0], which is a single kernel thread. I am investigating the options to utilize all the CPU cores for compression/decompression. Essentially, it would be a way to transform the left-over CPU capacity to compensate the lack of the physical memory.

Answer 1

After investigating a lot, I think I found the answer.

The actual compression is being done in the [kswapd].

The writer of the rejection mail in the cited thread shows that the responsible person had at least some reason what he did not communicate, but more likely he simply did not know from zswap.

I installed the patch suggestion on my system. It makes kswapd multi-threaded, i.e. it can compress memory on all the CPU cores. The patch works like charm, and causes a significant improvement in zswap-ping environments.

The proof: I massively overloaded my system (qemu with large memory consumption + a pxz compressor), both its memory and cpu. After that, I see this in the top:

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  927 root      20   0   12,3g   3,6g   3612 R 141,8  23,1  36006:53 qemu-system-x86
 5017 root      20   0 7607428   4,2g   1580 R  78,4  26,8   2:32.85 pxz
  354 root      20   0       0      0      0 R  53,6   0,0  32:17.28 kswapd0:5
  128 root      20   0       0      0      0 R  45,8   0,0  32:57.42 kswapd0:0
  352 root      20   0       0      0      0 R  40,5   0,0  32:16.80 kswapd0:3
  356 root      20   0       0      0      0 R  36,6   0,0  32:53.58 kswapd0:7
  350 root      20   0       0      0      0 R  35,3   0,0  31:15.53 kswapd0:1
  353 root      20   0       0      0      0 R  35,3   0,0  30:48.00 kswapd0:4
  351 root      20   0       0      0      0 R  28,1   0,0  31:57.45 kswapd0:2
  355 root      20   0       0      0      0 R  27,5   0,0  31:44.12 kswapd0:6

Yes, it also means the following:

• multi-threaded kswapd hugely increases zswap performance,
• it is not included in the mainline kernel,
• yes it is likely because incompetence or in the fear of instability.

---

The zswap parameters I've used in my test environment were the following (can be set up in /sys/modules/zswap/parameters):

same_filled_pages_enabled:Y
enabled:Y
max_pool_percent:50
compressor:deflate
zpool:z3fold
accept_threshold_percent:90