0

I have a strange issue with a cron-scheduled task using find which does two things:

  1. The first line should delete every night files older than 3 days starting with "read_" and end with ".gz".
  2. The second line should delete every minute files ending with ".gz.md5" older than 1 minute.
1   1 * * * find <path that defiantly does not match> -type f -iname "read_*.gz" -ctime +3 -delete
*/1 * * * * find <path that defiantly does not match> -type f -iname "*.gz.md5" -cmin +1 -delete

Typically this is working well, but every now and then at 01:01 I get the following error message by cron:

Cron <root@hostname> find <path that defiantly does not match> -type f -iname "read_*.gz" -mtime +3 -delete
find: ‘<path that defiantly does not match>/<hostname>/captured-syslog-1617836127-1617836351.gz.md5’: No such file or directory

But from my point of view, this makes no sense.

Jan D Hoffmann
  • 11
  • 2
  • Do you have a log-rotation job (`logrotate` or similar) running too, that may rotate the `captured-syslog-*` files at 01:00? – Kusalananda Apr 08 '21 at 07:13
  • The syslog files are copied via scp from another host, which aggregated the messages. This server as well creates the .gz.md5 files. But unfortunately this is not under my control. – Jan D Hoffmann Apr 08 '21 at 07:16
  • Do you have a local `logrotate` job running at 01:00 which manages the `captured-syslog-*` files? – Kusalananda Apr 08 '21 at 07:16
  • No. There is no logrotate or any other job in place for this time. – Jan D Hoffmann Apr 08 '21 at 07:30
  • 1
    please clarify that error is appearing "every minute" or just `01:01AM`? do you confirm that gz.md5 files deletion can finish in 1minute? if not then at next running time you will get that error as those are still under deletion by previous attempt by the cronjob and at the same time second attempt found them by when trying to delete they already deleted by previous same job – αғsнιη Apr 08 '21 at 08:28
  • `ctime` is NOT create time. It's inode change time. Use `mtime` instead – roaima Apr 08 '21 at 08:30
  • @αғsнιη: -> The issue ocures only once a day at 01:01AM. -> Yes the files are deleted within less than a minute. In fact less than a second. – Jan D Hoffmann Apr 08 '21 at 09:26
  • @roaima: You are right. But, in this specific case, the inode change time and creation time is the same. As mentioned before, the files are copied to the file system once and never touched again. – Jan D Hoffmann Apr 08 '21 at 09:30
  • what do you mean by ``, is this path contains both `.gz.md5` and `read_*.gz` files? if so then the error is come from because of these as both find comnad sees all the files in that path before passing to iname filter, so no worry about this prompt – αғsнιη Apr 08 '21 at 09:33
  • 1
    You should definitely look up the meaning of defiantly. – RalfFriedl Apr 08 '21 at 09:36
  • 1
    @RalfFriedl it works for me ;-) – roaima Apr 08 '21 at 09:44
  • @JanDHoffmann if you understand what `ctime` refers to, then that's great. Many people mistake `ctime` for _creation time_ and I was concerned that this group might have included you. – roaima Apr 08 '21 at 09:45

1 Answers1

2

Fact: at 01:01 the two jobs run concurrently.

Hypothesis: the filesystem in question does not store file types in directory entries.

Possible explanation

At some point find needs to know the type of the file under investigation. An explicit -type test can be a reason, but even without it (and in some cases before it) find needs to know if the file is a directory. If it is a directory then find will descend into it.

I think your filesystem does not store file type information in directory entries. In such case, to know the file type find needs to call lstat (although not always, some optimizations are possible, see this answer), this requires the file to exist.

If something else removes the file after find learns the file is in some directory, but before lstat, then there will be No such file or directory. In your case "something else" is the other job.

If the filesystem stored file type information in directory entries, this would be different. Only -ctime or -cmin could yield No such file or directory if the other job removed the file; but this cannot happen because -iname is tested earlier and filters all possible interference out.

I actually tested GNU find on ext4 filesystems created with and without the filetype flag. Without the flag the filesystem does not store file types in directory entries. I arranged a situation where files were created and deleted by another process. These files deliberately did not match -iname used with find.

In my tests on the filesystem without the filetype flag find often complained about No such file or directory for files that didn't even match -iname I used. In tests with filetype find never complained.

Kamil Maciorowski
  • 19,242
  • 1
  • 50
  • 94