How to iterate through files and delete those older than x days but NOT in use

Question

So, I've been able to figure out bits of this myself but am having trouble piecing them together. I have a task I need to automate - I have folders filled with gigabytes of obsolete files, and I want to purge them if they meet two criteria.

1. Files must not have been modified in the past 14 days - for this, I'm using find -

find /dir/* -type f -mtime +14

2. And the files cannot be in use, which can be determined by

lsof /dir/*

I don't know bash quite well enough yet to figure out how to combine these commands. Help would be appreciated. I think I essentially want to loop through each line of output from find, check if it is present in output from lsof, and if not, rm -f -- however, I am open to alternative methodologies if they accomplish the goal!

Answer 1

The following should work:

for x in `find <dir> -type f -mtime +14`; do lsof "$x" >/dev/null && echo "$x in use" || echo "$x not in use" ; done

Instead of the echo "$x not in use" command, you can place your rm "$x" command.

How does it work:

• find files, last modified 14 days or longer ago:

find <dir> -type f -mtime +14

• loop over items in a list:

for x in <list>; do <command>; done

• execute command 2 if result of lsof is 0, else execute command 1:

lsof "$x" && <command 1> || <command 2>

This relies on the lazy evaluation of Bash to execute command 1 or command 2.

On my system (Ubuntu 14.04) this works with file names with spaces in them and even for file names with ? and * in them.
This is however no guarantee that it will work with every shell on any system. Please test before replacing the echo command with the rm command.

Answer 2

Use the -exec action in find to execute a command for each file. This executes a program with arguments; if you need a more complex command (with variable expansion, conditionals, etc.) then you need to invoke a shell explicitly:

find /dir/* -type f -mtime +14 -exec sh -c '
  if …; then
    rm "$0"
  fi
' {} \;

To test whether a file is currently open, the most straightforward way is to call fuser.

find /dir/* -type f -mtime +14 -exec sh -c '
  if ! fuser "$0" >/dev/null 2>/dev/null; then
    rm "$0"
  fi
' {} \;

Beware that just because a file hasn't been modified in a long time and isn't currently open doesn't mean that it isn't useful. I recommend at least testing that the file hasn't been read in a while; this can be tested with the access time, but beware that Linux systems don't update the access time reliably. (Whether they do, and how often, depends on the kernel version, on the mount options, and on how the access time compares with the modification time.)

I recommend reviewing the files before deleting them.

find /dir/* -type f -mtime +14 -atime +14 -exec sh -c '
  if ! fuser "$0" >/dev/null 2>/dev/null; then
    echo "$0"
  fi
' {} \; >files-to-delete-potentially.txt

Review the file names, and erase the ones you want to keep. Then to remove them all, assuming none of the file names contains a newline character, you can use

<files-to-delete-potentially.txt tr '\n' '\0' | xargs -0 rm

Answer 3

Instead of using lsof which has a nightmare tangle of options and interesting ouput to parse, I suggest using the -atime or -amin options to find. These let you specify the file access time in days or seconds, respectively.

Instead of using another process to find out if a file is currently "in use", you can check to see if was accessed within the last N seconds or days.

The following command lists all files which were modified more than 14 days a ago and accessed less than 60 seconds ago.

find "$dir" -type f -mtime +14 -amin -60

To remove the files that match this criteria, you can use find's -exec command ; command. The strange thing about this is that you specify each command argument separately to find and terminate with a semicolon (;). If {} appears in any argument, it is replaced by the name of the name of the file being processed.

This command removes all files created more than 14 days ago, and not accessed within the last 60 minutes:

find "$dir" -type f -mtime +14 -amin -60 -exec rm '{}' \;