17

I have a drive (SD card) with a few ext4 partitions but also some unallocated space. The fstrim utility can only work within a filesystem. Before I reinvent the wheel and write one, is there another utility that can TRIM the unallocated space (or that can TRIM an explicitly specified range)?

I can verify that the majority of the unallocated space on the device is not currently known to be free by the controller, as I've observed that, on this particular card, reads to trimmed space return 0's, but a scan of the device shows plenty of garbage data left over.

Edit: I am having an issue using hdparm. The example below discards the first sector, but I am seeing the same results regardless of the range I specify. fstrim has no issues on the device:

root@ubuntu:~# hdparm --please-destroy-my-drive --trim-sector-ranges 0:1 --verbose /dev/mmcblk0 

/dev/mmcblk0:
trimming 1 sectors from 1 ranges
outgoing cdb:  85 0d 06 00 01 00 01 00 00 00 00 00 00 40 06 00
outgoing_data:  
00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

ioctl(fd,SG_IO): Invalid argument
FAILED: Invalid argument

I am investigating further but does anybody have any insight?

Jason C
  • 1,341
  • 3
  • 13
  • 29
  • 1
    If you map the unpartitioned space with one or more partition(s) (temporarily), you could use `blkdiscard` on them which trims the entire device. – frostschutz Oct 22 '13 at 19:03
  • Thanks! That looks like the right tool, but `blkdiscard` does not seem to be available in my standard package repositories (Ubuntu 12.04, doesn't seem to be present in `util-linux`). – Jason C Oct 22 '13 at 20:04
  • I was never able to get `hdparm` working for me for some reason, but I grabbed the util-linux source from https://github.com/karelzak/util-linux/tree/stable/v2.23, built it (`./autogen.sh ; ./configure ; make blkdiscard`) and it's working great. Still mystified about `hdparm` but `blkdiscard` does exactly what I want. Bonus: There's no need to make a temporary filesystem first, you can use `sfdisk -l` to figure out the space between partitions and have `blkdiscard` trim it. – Jason C Oct 30 '13 at 22:14
  • Oh, I wasn't even aware of `blkdiscard` being able to work with offset/length options. So the temporary partitions I suggested aren't even necessary. Nice! – frostschutz Oct 31 '13 at 10:45

3 Answers3

24

If you have a recent enough version of util-linux, it contains the tool blkdiscard which is able to TRIM entire devices, or ranges within a device using --offset and --length options.

Please note: blkdiscard is dangerous, if you let it TRIM the wrong regions, your data is gone!

So you can figure out the unpartitioned (free) regions of your partition table and then TRIM them using this tool. For msdos and gpt partitions, parted provides the free regions like so:

# parted -m /dev/sda unit b print free | grep ':free;'
1:17408B:1048575B:1031168B:free;
1:64022904832B:64023240191B:335360B:free;

Add a loop to it...

while IFS=: read -ra FREE
do
    echo blkdiscard --offset ${FREE[1]%%B} --length ${FREE[3]%%B} /dev/sda
done < <(parted -m /dev/sda unit b print free | grep ':free;')

which prints

blkdiscard --offset 17408 --length 1031168 /dev/sda
blkdiscard --offset 64022904832 --length 335360 /dev/sda

Verify that this output is correct for you, add additional options if you like (verbose?), and finally remove the echo so it will be actually executed, and you should be set.

The second command of that example actually fails because the length is too small - it may be worth checking inside the loop, ignore regions smaller than 1MB as they're unlikely to be successfully trimmed.

If you are using LVM instead of partitions, you can create a LV for the unoccupied space and trim that:

lvcreate -l100%FREE -n blkdiscard SSD-VG
blkdiscard -v /dev/SSD-VG/blkdiscard
lvremove SSD-VG/blkdiscard

If you set issue_discards = 1 in your lvm.conf, you can skip the blkdiscard call as LVM will issue the TRIM on lvremove by itself.

frostschutz
  • 47,228
  • 5
  • 112
  • 159
  • 1
    Thanks! I have remarked this as the answer because it's working for me, even though Michael's answer is also valid. Also I will add what I said in the question comments: If you *don't* have `blkdiscard` available; you can grab util-linux source from https://github.com/karelzak/util-linux/tree/stable/v2.23 then build `blkdiscard` (run `./autogen.sh ; ./configure ; make blkdiscard`) -- it built cleanly (although you may have to install some dependencies and tweak `configure` options) and as long as you don't install it, it does not conflict with existing util-linux. – Jason C Oct 31 '13 at 21:34
  • 1
    I was observing failures with certain values passed to `blkdiscard` as well, but noticed the values were based on the offset, not the length. It's possible there is an issue in `blkdiscard`, I am in the process of investigating. See http://unix.stackexchange.com/questions/98473/where-is-ioctl-0x1268-blksszget-actually-specified – Jason C Nov 01 '13 at 22:49
  • I have created a script that does discard free space on all mounted file systems as well as free LVM VGs: https://github.com/stefanct/discard-everything – stefanct Jan 10 '14 at 15:37
  • Thanks! I never knew that parted can just print the free ranges. – intelfx Jul 19 '20 at 02:25
7

hdparm --trim-sector-ranges can trim a range. The man page warns to use it, so you better be sure you got the right range and syntax.

I think sending a trim for all data outside a partition would be dangerous, as there is some hidden data there sometimes like bootloader code or second partition tables. You'd need to know exaclty, which areas outside of partitions are really unused.

Michael Suelmann
  • 1,055
  • 5
  • 7
  • Perfect, thanks! I can grab the device size with `hdparm`, unused ranges from `sfdisk`, and trim accordingly. I know what's on the drive. If there is a partioning scheme that hides data in accessible but unallocated areas, that would break any partition editor's basic functionality. I presume (hope) that any schemes like this have long been abandoned (I know of none off the top of my head though). All schemes I know of store the MBR in well-known unallocated space at the start of the device, and VBRs/EBRs within allocated partitions. There may be old obscure schemes that are different. :) – Jason C Oct 22 '13 at 20:18
  • I am having some difficulty with `hdparm`, I edited my question with details; wondering if you have any insight. – Jason C Oct 22 '13 at 20:35
  • 1
    Seems like at least some of hdparm doesn't work on /dev/mmcblk0. Sorry, I have no further idea. – Michael Suelmann Oct 22 '13 at 21:03
0

Note about blkdiscard - it is VERY VERY dangerous and can be messy.

I got accident on test system when issuing command:

root@test:/home/test# fdisk -l /dev/nvme1n1
Device               Start        End    Sectors   Size Type
/dev/nvme1n1p1        2048    1101823    1099776   537M EFI System
/dev/nvme1n1p2     1134592    1658879     524288   256M Linux RAID
/dev/nvme1n1p3     1658880 1025658879 1024000000 488.3G Linux RAID    
root@test:/home/test# 
root@test:/home/test# blkdiscard /dev/nvme0n1
root@test:/home/test# fdisk -l /dev/nvme0n1
bash: /usr/sbin/fdisk: cannot execute binary file: Exec format error
root@test:/home/test# 
root@test:/home/test# fdisk -l /dev/nvme1n1
bash: /usr/sbin/fdisk: cannot execute binary file: Exec format error
root@test:/home/test# fdisk -l /dev/nvme1n1p1 
bash: /usr/sbin/fdisk: cannot execute binary file: Exec format error
root@test:/home/test# 
root@test:/home/test# fdisk 
bash: /usr/sbin/fdisk: cannot execute binary file: Exec format error

It did TRIM another drive!!! OS was installed on nvme1n1p1..3 and nvme0n1 was test drive.

Fortunately I had complete loss of test system, not big deal, but I feel obligation to share this publicly to warn you.

I think better approach is fstrim and set issue_discards = 1 in lvm.conf as mentioned by frostschutz

Arunas Bartisius
  • 773
  • 5
  • 13
  • 1
    It is dangerous that's for sure. But your results don't seem like a blkdiscard bug; it doesn't really make sense (and also I've seen its source code). I strongly suspect that either the device names weren't mapped to the devices you thought they were mapped to, or your /usr/sbin wasn't on the device you thought it was on. – Jason C May 08 '20 at 23:27
  • I am sure root fs was not on the device to be trimmed. But OS level name mapping can be an issue. In this case it's even more dangerous, as there is uncertainty which device will be destroyed. I have thoughts that may be this is efi bios related, but it's weird that fdisk and smartctl showed identification in one way, while blkdiscard treated names other way. Will update title not to be pointing to a bug. – Arunas Bartisius May 12 '20 at 16:21