1

On a panfs filesystem:

  • pan_quota says I'm using 8.67 GB.
  • du -hcsx says I'm using 8.1 GB.
  • baobab says I'm using 3.3 GB.

This is for my home directory, with a 8 GB soft and 10 GB hard quota. What might cause the modest discrepancy between duand pan_quota? And what may cause the huge discrepancy between baobab and either du or pan_quota?

The difference is not due to one or two specific files, but propagates down and is present for smaller and larger directories alike.

gerrit
  • 3,457
  • 6
  • 25
  • 41
  • Do you have a lot of tiny files? Could be a logical v. physical size difference; what does `du --apparent-size -hcsx` give? – derobert Apr 22 '16 at 17:34
  • 5
    Have you read [Why are there so many different ways to measure disk usage?](http://unix.stackexchange.com/questions/120311/why-are-there-so-many-different-ways-to-measure-disk-usage)? – Gilles 'SO- stop being evil' Apr 22 '16 at 22:17
  • @derobert 34301 files, blocksize 64 kB. The `du --apparent-size -hcsx` command you suggest gives the same answer as `baobab`. – gerrit Apr 22 '16 at 22:39
  • @derobert I have partly. I thought the differences I found are too large to be explained by it, but perhaps I was wrong. Block size is apparently 64k. – gerrit Apr 22 '16 at 22:41
  • @gerrit That's what's going on then, it's small files eating an entire block. A 64k block size is pretty big. The question Gilles linked explains in depth. – derobert Apr 22 '16 at 22:41

1 Answers1

1

The disk space used by a file includes more than the size of the contents. Most filesystems use fixed-size blocks; if a file's size isn't a multiple of the block size, then the last block is only partially filled. du counts the total number of blocks, which is usually a little more than the sum of the file sizes since it also includes the part of the last block that isn't filled.

I think the difference between pan_quota and du -h is due to the fact that the PANFS filesystem doesn't report its block size accurately. The JASMIN FAQ suggests using pand_du instead of du.

It seems that on your machine, Baobab is displaying the sum of the file sizes. That's the number you get with du --appparent-size. Baobab displays the actual disk usage on my machine (but with 1GB = 109 bytes, whereas du -h uses 1GB = 230 bytes); I think this is because you're using a version older than this change

commit dbcbe27e0452eeacbc4a253f1b0b1a06708834de
Author: Paolo Borelli <[email protected]>
Date:   2012-01-07

    Use the STANDARD_ALLOCATED_SIZE file attribute.

but I haven't traced it down to make sure.

For more information about disk usage discrepancies, see Why are there so many different ways to measure disk usage?

Community
  • 1
Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
  • I neglected to mention that I don't run baobab on the host machine (where it is unavailable), but through sshfs, so perhaps that prevents baobab from performing an accurate calculation. – gerrit Apr 23 '16 at 20:33