20

This post from StackOverflow has this,

In some environments there there is a restriction on certain instructions or using certain registers. For example, in the Linux kernel, use of SSE/AVX or FP registers is generally disallowed. Therefore most of the optimized memcpy variants cannot be used as they rely on SSE or AVX registers, and a plain 64-bit mov-based copy is used on x86. For these platforms, using rep movsb allows most of the performance of an optimized memcpy without breaking the restriction on SIMD code.

Why is that, that the x86_64 Kernel can not use SSE/AVX? If it would make memcopy() faster it seems like it should be allowed. I'm just learning Intel Assembly, and specifically looking to learn SEE/AVX when I saw this comment.

Specifically interested in SSE/MME and AVX optimizations in the Linux Kernel.

Evan Carroll
  • 28,578
  • 45
  • 164
  • 290
  • 2
    One reason some OSes restrict the use of certain registers is that if they are used, then they must be saved on a context switch, which means that context switches take longer and thread contexts grow larger. I don't know whether this the/a reason why Linux puts restrictions on SIMD registers. – Gilles 'SO- stop being evil' Oct 17 '18 at 06:37
  • 1
    I'm in two minds about this question. It's off-topic here since it's about programming: a user or system administrator wouldn't care at all. Its rightful place is [so]. But I know that [so] has an epidermic reaction against “why”, even though this is a perfectly legitimate, answerable question that's on-topic on [so]. And [unix.se] probably has enough expertise to answer it, even though it's on the outer side of the topic border. – Gilles 'SO- stop being evil' Oct 17 '18 at 06:40
  • 1
    Answers will touch upon Linux-specific stuff when one considers things like CVE-2018-3665. – JdeBP Oct 17 '18 at 06:46
  • 1
    I don't think it's about programming, so I'd be inclined to agree with Stack Overflow. Though I don't think Stack Overflow is about programming either, so I'm inclined to agree. In reality, SO, imho, is a dumping site run by madmen and the question *would* be on topic there. **If this is a programming question, it's specific to the design of the Linux Kernel,** and lacking code that would seem on topic here? – Evan Carroll Oct 17 '18 at 06:51

1 Answers1

26

As Gilles mentions, anywhere the FPU is liable to be used, the kernel needs to support saving and restoring its state. Since user-space can use the FPU, this needs to be handled in any case on context switches (i.e., when the current CPU switches from one thread to another) — at least, when the previously-running thread used the FPU. So why not extend that to the kernel?

There are a couple of reasons to avoid using the FPU in the kernel:

  • from a portability perspective, some architectures don’t support using the FPU in the kernel at all, so generic code can’t rely on it;
  • saving and restoring the FPU state is expensive and introduces certain implementation-related constraints (on x86 Linux, pre-emption in particular needs careful consideration here).

Having the kernel avoid using the FPU means that the cost for user-space can be reduced: FPU state need only be restored after a context switch when returning to user-space (as opposed to immediately after a context switch), and not in all cases (only when the threads involved actually use the FPU).

It is however possible to use the FPU (and MMX/SSE/AVX) in the kernel, in x86-specific code where the benefits outweigh the costs: thus it ends up being used in the crypto code and RAID6. These emails from Linus provide some more details. If you want to use the FPU, you need to bracket all the FPU-using the code between kernel_fpu_begin and kernel_fpu_end, and make sure it can’t fault or sleep. See arch/x86/include/asm/fpu/api.h and arch/x86/kernel/fpu/core.c for details.

For memcpy, the performance gains don’t outweigh the cost of using the FPU.

(x86 has a rather complex FPU architecture, but it provides all the features needed to make it possible for an operating system to share the FPU: it can trap whenever an FPU instruction is emitted, which allows the kernel to optimise for processes which never use the FPU, and it can indicate when the CPU and FPU state are liable to diverge. It also provides instructions to save and restore the FPU state — FSAVE, FXSAVE, and XSAVE depending on the FPU’s vintage. FPU support is perhaps the aspect of the 8086 design where the designers had the most foresight.)

Stephen Kitt
  • 411,918
  • 54
  • 1,065
  • 1,164