14

Setup

I have USB enclosure (Buffalo DriveStation Quad) containing four drives connected to my nas server (ubuntu server 14.04). The enclosure is configured to JBOD mode, so I'll see all the disks in in Linux.

Two of the disks (sdb and sdc) are configured with software raid as /dev/md0 (raid1). And /dev/md0 is mounted as single partition (/mnt/part1) with ext4 filesystem without journalling.

The other two disks (sdd and sde) are set up with LVM as one volume group, from where I have mounted two logical partitions. One of which is 90% of whole volume group capacity (/mnt/part2), and one that is 10% (/mnt/part3). Both are also ext4 without journalling.

APM Issues

My problems started with default APM modes, as I noticed that the hard drives head parked quite aggressively every couple of minutes. After researching the topic for a bit, I ended up using hdparm -B198 /dev/sd[bcde]. This seems to allow some level of power saving, but without really doing any head parking.

Any sleep?

I'm sort of happy with current situation, but I'd still like the drives go to sleep if there is no activity. Especially the sdb and sdc (/mnt/part1) that doesn't really get any activity for 95% of time. Whatever I've tried, the problem seems to be that the drives don't sleep longer than a minute or two.

Unmounting all partitions, and issuing hdparm -y /dev/sd[bcde] will put the drives into sleep mode, but only for a few minutes. After that they will all wake up one by one. I've tried to debug the issue by enabling block_dump (echo 1 > /proc/sys/vm/block_dump), but don't see any access to the disks.

I also tried to disable APM with hdparm -B255 /dev/sd[bcde], and command them to sleep after that, but same thing. Still the drives wake up after couple of minutes.

I don't have mdadm running in daemon mode (just a single check once a day), nor should there be anything else probing the drives. So any ideas on what to try next? Is the Buffalo USB enclosure just crappy (and does this on its own)?

Update #1

I took time for how long does it take for the disks to wake up after issuing hdparm -y /dev/sd[bc]. Following timestamps illustrates the pattern: 

00:00 hdparm -y /dev/sd[bc]
00:40 disks start to wake up
00:59 disks fully awake
01:00 hdparm -y /dev/sd[bc]
03:40 disks start to wake up
03:59 disks fully awake
04:00 hdparm -y /dev/sd[bc]
06:40 disks start to wake up
06:59 disks fully awake

I.e. it seems that something checks/wakes the disks every 3 minutes. First command to go standby mode just happened to be 40 seconds from the checkpoint.

Update #2

Rebooted the machine with acpi=off apm=off. Did not help either. Btw, the machine is Lenovo L520 laptop. Just in case someone finds that relevant.

Toni
  • 401
  • 3
  • 9
  • 2
    my $.02: try to stop everything on your machine (overzealous daemons might be looking around to probe devices), use noatime mount option. – Laszlo Valko Jun 13 '15 at 16:33
  • @LaszloValko, managed to reduce processes to `upstart-{socket,file}-bridge, dhclient, getty and sshd` -- no luck :(. There is of course a lot of kernel processes running (ones listed in brackets). Haven't looked yet into if I could reduce those by some kernel parameters... and which ones would be good candidates. – Toni Jun 14 '15 at 14:38
  • 1
    Simple way to tell if it's the enclosure or your OS would be to spin down the drives then disconnect the USB. – Circus Cat Jun 15 '15 at 11:47
  • @qasdfdsaq, unfortunately this Buffalo Drivestation comes with some fancy powerdown feature. The enclosure shuts down itself down immediately when there the usb cable is unplugged. Even the power switch has only options "off" and "auto". – Toni Jun 15 '15 at 14:38
  • 1
    Just a shot in the dark: check updatedb.conf's pruned paths & bind mounts, so that these paths are explicitly skipped ('locate' service); it could easily be some other similar service, though. – michael Jun 16 '15 at 19:11
  • @michael_n, thanks for the effort, but /mnt is included in PRUNEPATHS, and problem exists even when the filesystems are not mounted. – Toni Jun 17 '15 at 16:43

1 Answers1

2

Might be a bit overkill, but SystemTap could help you identify what process is doing i/o on that disk.

Prepare SystemTap

[root@localhost ~]# stap-prep
snip

Install trace script

[root@localhost ~]# cat >/tmp/traceio2.stp
#! /usr/bin/env stap
global device_of_interest

probe begin {
  /* The following is not the most efficient way to do this.
      One could directly put the result of usrdev2kerndev()
      into device_of_interest.  However, want to test out
      the other device functions */
  dev = usrdev2kerndev($1)
  device_of_interest = MKDEV(MAJOR(dev), MINOR(dev))
}

probe vfs.write, vfs.read
{
  if (dev == device_of_interest)
        printf ("%s(%d) %s 0x%x\n",
            execname(), pid(), ppfunc(), dev)
}

Figure out the device id you want to monitor, in this case I'm going to monitor /dev/sda5

[root@localhost ~]#  df -k /
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/sda5       18141508 16293424    903496  95% /
[root@localhost ~]# ls -l /dev/sda5
brw-rw----. 1 root disk 8, 5 Jul  1 01:21 /dev/sda5
[root@localhost ~]#

Monitor, using the major+minor number (8,5) in hex. Find culprit. Rejoice

[root@localhost ~]# /tmp/traceio2.stp 0x805
accounts-daemon(434) vfs_read 0x800005
accounts-daemon(434) vfs_read 0x800005
accounts-daemon(434) vfs_read 0x800005
lightdm(503) vfs_write 0x800005
bash(3036) vfs_read 0x800005
bash(3036) vfs_read 0x800005
^C
[root@localhost ~]#
steve
  • 21,582
  • 5
  • 48
  • 75