7

I run Debian Jessie on a quad-core i7 processor, which results in 8 cores being shown thanks to hyperthreading. On disk-heavy loads, all eight cores are maxed out with io-wait, as shown by htop. This results in a very sluggish interface (I use Gnome), and the system sometimes freezes completely until the I/O is complete.

How should I go about diagnosing this? Which settings can I tune to prevent this from occurring?

I use a single HDD, and a S.M.A.R.T check shows that its healthy.

Edit: To clarify a few things, I'm not interested in speeding up the I/O. I'd like to just have it run in the background while I do something else. Upgrading to an SSD is currently not an option for me.

RandomBK
  • 91
  • 1
  • 5
  • That is interesting....how many threads are writing? – SailorCire Oct 25 '14 at 22:53
  • @SailorCire More than 8. This usually happens when I copy a large number of files in Gnome while other disk I/O is also going on (downloading files in chrome, compiling, etc). Running VMware also triggers this issue. I know that sluggishness is to be expected when doing so many things, but my system sometimes freezes for several minutes before I can do anything. Even switching to another tty using ctrl-alt-f1 takes about a minute or two to respond. I would have expected the i/o scheduler to at least leave a core free to keep the system functional... – RandomBK Oct 25 '14 at 23:08
  • I've never had this occur myself, but I am fairly sure that I/O bound processes have higher priority within the Linux scheduler. That point being that I/O bound processes have little CPU time requirements, so they quickly begin their I/O and yield the CPU. This is an interesting issue you are observing. Are you sure it's the I/O and perhaps not an overloaded system? – sherrellbc Oct 25 '14 at 23:20
  • @sherrellbc Fairly sure, though all the data comes off htop. During the unresponsive periods, the only major cpu tasks are io-bound, i.e. copying files. Memory doesn't look like an issue, and theres nothing in swap, as I have vm.swappiness set to 0. – RandomBK Oct 25 '14 at 23:39
  • If you've got more than 8 threads in disk-wait, the cores may report that they are idle, but the bus is jammed with the disk blitting blocks to core. Given any architecture I can bottleneck it with some arbitrarily large load; you found your limit. – msw Oct 26 '14 at 04:30
  • Sounds like the disk is failing since I've had this problem happen with failing disks.I can download files without any slowdown and compiling only slows due to cpu usage – Suici Doga Mar 05 '18 at 13:54

1 Answers1

2

There are several ways to tune I/O performance.

  • file system choice - choose a file system depending on what your typical data looks like - some file systems handle better lots of smaller files, some do well for large files, some don't scale well when accessing (especially creating) multiple files at once. Defragmenting might help on spinning plate drives (not so much on flash based devices, but the difference should still be there).

  • file system tweaking - modern file systems have loads of options (both create- and mount-time) which push performance of the file system in one direction or another. Prime example can be how journalling is done.

  • DMA setting - while this has been done automagically since ages, it never hurts to check it (hdparm -i is your friend here).

Linux specifically (but might be similar for other kernels as well):

  • THP - Transparent Huge Pages used to cause performance degradation. IIRC the problem has been fixed (at least to some extent) some time ago. If copying to/from USB is the issue, you probably should look in this direction. disable transparent hugepages might serve as a good starting point as for how to disable THP.

  • kernel I/O scheduler can influence the performance a big deal. Some people advocate the use of the Deadline scheduler over CFQ which is more complex and sometimes allegedly tends to overthink things.

Apart from that, in some scenarios heavy I/O just happens. When you start copying files from USB memory card reader to hard drive, from hard drive to a USB flash disk, browse the web (some browsers might be fsync() heavy to make sure things are recoverable in case of a crash), download emails, run a P2P client and play a movie all at the same time, the physical disk and the file system are just going to become the bottleneck. In some cases a carefully constructed (read with appropriate stripe size and number of constituent drives) RAID0 might help to alleviate the problem.

Community
  • 1
peterph
  • 30,520
  • 2
  • 69
  • 75
  • Thanks for your reply. I know that heavy disk loads will slow down the system - that isn't the issue. The issue is just _how_ slow the system gets. On even medium I/O tasks (like copying to an USB, as you pointed out), the system slows to an unusable state, with mouse movements taking half a minute to register. Besides, I thought the whole point of hyperthreading was to let the CPU do something else while waiting on I/O? I'll try your suggestion to change to another scheduler, and see if that changes anything. – RandomBK Oct 25 '14 at 23:47
  • @RandomBK THP - check updated answer for link. – peterph Oct 26 '14 at 09:12
  • @RandomBK Hyperthreading: no (or not just). The basic motivation behind HT is that multiplying register banks and some additional circuitry, while leaving most of the other things shared, is way cheaper then introducing additional core and the performance gain can be anything from around 0% to 30% (theoretically you should be able to craft a very special scenarios where you should get much higher, but that's more of an academic question than a serious use case). Hence: yes HT could help in heavy I/O, but so would any multicore (and would actually help more). Your problem is somewhere else. – peterph Oct 26 '14 at 09:13