2

I have the following devices with Linux Mint 18.1 on laptops and GNU/Linux Debian 9 on the server.

(All are 64-bit and with Cinnamon desktop.)

All drive devices are formatted with ext4 filesystem; RAID 1 is done utilizing mdadm.

  1. Laptop with 1 SSHD (not to be confused with HDD).

  2. Laptop with 3 drives: 2 x consumer HDDs in RAID 1 and 1 x SSD.

  3. Server with 5 drives: 4 x enterprise HDDs in two times RAID 1 and 1 x SSD.

I have the system on those SSDs and I would never defragment an SSD.

The question is about HHDs and an SSHD.

I found an old PDF outlining a few more features to e4defrag.

  1. Why the filesystem must be mounted, as per this error message when trying to defragment an unmounted filesystem? I want to understand why that is:

    Filesystem is not mounted

  2. I would like to have implemented free space defragmentation. AFAIK it is now under review. Is it possible for me to e.g. compile e4defrag from source with these options available or anyhow?

    e4defrag -f /deviceOrDirectory

  3. I would also like to use relevant data feature:

    e4defrag -r /deviceOrDirectory

I have many relevant reasons to believe the fragmentation on these machines is slowing down the read speed, example:

  1. Taken from the server with RAID 1 HDDs:

    [2556/30987]/raid1a/bitcoind/blocks/rev00820.dat: 100%  extents: 16 -> 1  [ OK ]

  2. Taken from the laptop with RAID 1 HDDs:

    [29405/50810]/raid1/movies/SGA-HEVC/S04E01 - Adrift.mp4:  100%  extents: 31 -> 6  [ OK ]

As you can see, the defragmentation was not even able to put the 31 blocks file into 1 piece. Of course you might argue it is a movie file, so it does not matter. True, but only in this case.

The command I use to start the defragmentation:

  1. On the server:

    sudo e4defrag -v /dev/md1

  2. On the laptop:

    sudo e4defrag -v /raid1/

It does not seem to matter, whether I invoke the command using device name or a directory.

Can you point me to the right direction?

Vlastimil Burián
  • 27,586
  • 56
  • 179
  • 309
  • You have measurement results that point to defragmentation as the source of slow access? If yes, which ones? In the majority of cases, you don't need to do manual defragmentation on ext4 filesystems, as long as there's enough free space for the allocation algorithm to automatically defragment it during normal operation. – dirkt Apr 03 '17 at 09:53
  • @dirkt No measurements have been taken so far. I don't argue, whether it is or it is not effective to defragment `ext4` filesystem. The questions are clear: How to do `e4defrag -r` and `e4defrag -f`. – Vlastimil Burián Apr 03 '17 at 09:57

1 Answers1

7

e4defrag needs the file system to be mounted because it asks the kernel’s file system driver to perform the defragmentation, it doesn’t do it itself.

As for free space defragmentation and relevant file defragmentation, the patches were never completed; the last mention on the relevant mailing list dates back to 2014:

The e4defrag is in e2fsprogs, and the code is still getting maintained and improved. Dmitry Monakhov has in particular added a lot of “torture tests”, and found a number of race conditions in the underlying kernel code. He recently also sent a code refactor of the kernel code which significantly improved it and (shrank the size of ext4 by 550 lines of code).

That being said, there hasn't been any real feature development for e4defrag in quite some time. There has been some discussion about what the kernel APIs might be to support this feature, but there has never been a finalized API proposal, let alone an implementation.

So I doubt there’s anything worth testing currently.

Stephen Kitt
  • 411,918
  • 54
  • 1,065
  • 1,164
  • Is there any information on the current (2020) best defrag practice on ext4 - or a recommendation not to use any existing tool because of unreliability? My `e4defrag --version` says `e4defrag 1.45.5 (07-Jan-2020)`. – Ned64 Mar 11 '20 at 17:06
  • 1
    The situation hasn’t changed much. There was a thread not long after I wrote the above answer, which includes [a summary of the relevance of `e4defrag`](https://www.spinics.net/lists/linux-ext4/msg56865.html) at the time. It’s still a fully-supported part of Ext4, but it’s not usable in all cases (file systems with bigalloc can’t be defragged, data journaling is incompatible with defragging, and DAX inodes and encrypted files can’t be moved). – Stephen Kitt Mar 11 '20 at 17:37
  • Thanks, Stephen. Does it mean the command is safe in a sense that it will bail out before a possibly harmful action so that I can use it without risk to my data? (Basically, I just removed 750000 files and wondered whether the disk could now be fragmented (I suppose it is) and whether to clean it or leave it.) – Ned64 Mar 11 '20 at 17:39
  • Like any file system operation, it’s potentially dangerous, so only use it if you have backups (or the data is disposable). However the command is safe as far as its developers are aware; if it encounters situations it can’t handle it stops safely. – Stephen Kitt Mar 11 '20 at 17:42
  • Cheers. There are still 3 TB of data on the disk, no backup. Will wait until I have a day where I feel daring enough to try :-) – Ned64 Mar 11 '20 at 17:45