5

I have a purge job that runs daily to clean up logs older than 30 days.

find /dir/app/logs -mtime +30 -exec rm -f {} \;

I am moving our jobs out of cron and into a 3rd party scheduling product, Automic. Since moving this job, I keep getting the error "No such file or directory" randomly. Running the find command at the prompt after receiving the error, without the -exec rm -f {} \;, always returns no results and runs successfully. Long story short, I'm unable to reproduce the error.

The job runs by executing the command:

ssh user@server "find /dir/app/logs -mtime +30 -exec rm -f {} \;"

against the remote server.

I have tested various solutions without any luck. Originally, the command ran without -f. Adding -f, I understand, is supposed to suppress errors, but I'm not seeing that happen. I tried replacing -exec rm {} \; with -delete, but that didn't help either.

Currently I'm testing changing \; to + as suggested here:

find - exec rm vs -delete

Thanks in advance for any insight into what's going on.

lightwing
  • 119
  • 2
  • 12
  • Ssh introduces another level of quoting; there’s another question on it here somewhere. – Jeff Schaller Jan 05 '18 at 18:19
  • Maybe this? https://unix.stackexchange.com/questions/212215/ssh-command-with-quotes `ssh user@server 'find /dir/app/logs -mtime +30 -exec "rm -f {}" \;'`? – lightwing Jan 05 '18 at 18:33
  • I have run into this problem periodically, and I wonder if it is actually caused by removing the directory while find is still traversing it. Try piping the output of find to xargs to delete the directories you want after find has finished traversing: `find . -type d -iname *.blah -print0 | xargs -0 -I {} rm -rf "{}"` – Bretton Wade Feb 02 '22 at 16:46
  • @BrettonWade: That doesn’t make much sense.  (1) The OP is using `rm -f` (without `-r`) and is not trying to remove directories.  (2) Your solution doesn’t make much sense.  (2a) It would create a race condition, where the `find` and the `xargs` are running concurrently.  It would still be possible for `xargs` to remove a directory while `find` is still traversing it.  (2b) You might believe that your suggestion is equivalent to  … (Cont’d) – G-Man Says 'Reinstate Monica' Aug 31 '22 at 21:51
  • (Cont’d) …  ```find . -type d -iname *.blah -print0 > tmpfile; xargs -0 -I {} rm -rf "{}" < tmpfile```.  It’s not, because, as I said, processes in a pipeline run concurrently (while **`;`** forces sequential execution).  But even if you did that, it would just move the “No such file or directory” errors from `find` to `xargs` / `rm`.  (3) Please don’t say `-iname *.blah`.  Deferred wildcards should be quoted: `-iname '*.blah'`. – G-Man Says 'Reinstate Monica' Aug 31 '22 at 21:51

1 Answers1

3

After some time off and more googling around, this solution may fix the issue:

find /usr/dir/logs/ -mindepth 1 -mtime +45 -delete

It appears find and rm were including the parent directory and trying to remove it first, which explains why it sometimes worked, then would throw missing file/directory errors. Using -delete worked fine, and adding -mindepth 1 caused it to skip the parent.

To retain subdirectories where they shouldn't be deleted, adding -type f to only look at files left those be. Additionally, there was one instance where a specific file needed ignored. This appears to work for that situation:

find /usr/dir/logs/ -mindepth 1 -type f \( ! -name "ignoreme*" \) -mtime +30 -delete
Karl Wilbur
  • 175
  • 1
  • 6
lightwing
  • 119
  • 2
  • 12
  • 4
    Also, with `-delete`, it will do a depth-first traversal of the directory structure. This means that if you _do_ delete directories (obviously you don't with `-type f`), then any directory that you delete will already have been processed previously (`find` won't try to access that directory again). You may force a depth-first traversal without `-delete` using `-depth`. – Kusalananda Jan 11 '19 at 15:08
  • This doesn’t make sense.  (1) `rm -f` (without `-r`) should not have removed any directories.  (2) If it was deleting `/usr/dir/logs`, how were you not noticing that?  (3) This does not explain why your job was working under cron and not Automic. – G-Man Says 'Reinstate Monica' Aug 31 '22 at 21:51