Compare if the newest file in the directory is different from the previous one

Question

I have the following script:

#!/bin/bash
y=$(ls -t ./pics/0/*.png | head -n 2 | tail -n 1)
new=$(ls -t ./pics/0/*.png | head -n 1)
while true
do
 if cmp --silent "$y" "$new" ; then
y=$(ls -t ./pics/0/*.png | head -n 1)
    base64 $y | tr -d '\n' | sed '$ a \'
new=$(ls -t ./pics/0/*.png | head -n 1)
 fi
done

What am I doing wrong? To be clear my goal is to compare whether the newest file is different from the previous newest file and only if it is generate a unique BASE64 to STDOUT (meaning it should be printed only ONCE).

You are missing spaces in your `if`. It is `if [ cmp --silent $y $new != null ]`. — schrodingerscatcuriosity, Aug 19 '19 at 19:21
@guillermochamorro Now it says that I've got too many arguments.. — Sir Muffington, Aug 19 '19 at 19:24
@steeldriver this code works, but the script prints out multiple duplicate base64 strings, I probably made a logical error in doing the script then? any clue how to fix it so that prints unique base64 every time a file is changed? — Sir Muffington, Aug 19 '19 at 19:32
Removing the --silent flag seems to reduce the number of occurrences of base64 strings to only 3-4 per second, when a new file is added every second. Is this a bug??? — Sir Muffington, Aug 19 '19 at 20:03
It also spams my STDERR with argument list too long on my cmp line... So I guess the errors are making it slow down.. — Sir Muffington, Aug 19 '19 at 20:11
`$x -le 5` You don't have in your code an update of the `x` value. The expression is always true, which makes it an infinite loop. — schrodingerscatcuriosity, Aug 19 '19 at 21:44
to start with, you are comparing the value 0 (`y=0`) with a list of files ($new). this won't work. `cmp` takes **exactly** two filename arguments. 0 is probably not a filename, and $new can contain 0, 1, or any number of filenames. — cas, Aug 20 '19 at 01:01
try `y="$(ls -t ./pics/0/*.png | head -n 2 | tail -n 1)"` and `new="$(ls -t ./pics/0/*.png | head -n 1)"`. i have no idea what the rest of your script is meant to do because it makes little or no sense. — cas, Aug 20 '19 at 01:03
it's supposed to be an infinite loop. @cas I edited the code accordingly now it doesn't print anything.. — Sir Muffington, Aug 20 '19 at 10:12
why `[ -z "$(cmp ...)" ]`? You want to test the **exit code** from `cmp`, not its output (which is typically empty). Use `if cmp "$y" "$new" ; then ` instead. — cas, Aug 20 '19 at 10:15
and if you want an endless loop, use `while true; do ........ ; done`. Testing for `[ $x -le 5 ]` just makes it look like you forgot to increment $x. — cas, Aug 20 '19 at 10:17
@cas now it just spams me with standard output ./pics/0/out12:17:02.747683.png ./pics/0/out12:17:03.765651.png differ: char 36, line 3. If I add the --silent flag back it doesn't do anything at all.. — Sir Muffington, Aug 20 '19 at 10:18
For people wondering what `| tr -d '\n' | sed '$ a \'` does: it's for formatting the message, it first removes all newlines if there are any and puts one at the end. — Sir Muffington, Aug 20 '19 at 12:07

markgraf · Accepted Answer · 2019-08-20T15:28:26.550

0

Please note that the following snippet will NOT work with filenames containing spaces, tabs or newlines.

To find out more about the find-command used here, please reference https://unix.stackexchange.com/a/240424/364705

#!/bin/bash
lastrun=''
while true; do
  read -r newer older <<< \
    $(find . -type f -exec stat -c '%Y %n' {} \; \
    | sort -nr \
    | awk 'NR==1,NR==2 {print $2}'\
    | xargs )

  if [ "$newer" == "$lastrun" ]; then
    :
  else
    if ! cmp "$newer" "$older" > /dev/null ; then
      base64 "$newer"| tr -d '\n' | sed '$ a \'
      lastrun="$newer"
    fi
  fi
done

And a solution using inotify-wait:

#!/bin/bash
lastfile=''
inotifywait -m /path/to/somewhere -e close -e moved_to |
while read path action file; do
  if [ "$lastfile" != '' ];then
    if ! cmp "${path}$file" "${path}${lastfile}" > /dev/null  ; then
    base64 "${path}${file}"
    fi
  fi
    lastfile="$file"
done

edited Aug 20 '19 at 15:28

answered Aug 20 '19 at 13:10

markgraf

2,849
1
8
20

This works. However, when the file amount in the folder goes around over 1000 it appears to skip 1-3 files per second. Is this a performance limitation of find? – Sir Muffington Aug 20 '19 at 14:00
Could be a shell-script isn't fast enough if the files are coming quickly. If you do not have any subdirectories in the directory where you are running this, then you can drop the `find..-exec` and just do `stat -c '%Y %n' *` instead. That should be faster. – markgraf Aug 20 '19 at 14:06
If that still isn't fast enough, maybe look into `inotifywait` to trigger an action whenever a file is created/moved_to your directory. – markgraf Aug 20 '19 at 14:22
It doesn't appear to be working without the find part. I already wrote a prototype for inotifywait, but still need to format it properly. – Sir Muffington Aug 20 '19 at 14:23
```$(stat -c '%Y %n' * | sort -nr ...``` instead of ```$(find . -type f -exec stat -c '%Y %n' {} \; | sort -nr ...```. But only if there are no subdirs. – markgraf Aug 20 '19 at 14:26
Yes, it appears to print one line only for some reason... – Sir Muffington Aug 20 '19 at 14:29
Ok so I wrote a program called inotify, which prints out the newest added file. How do I integrate it now? – Sir Muffington Aug 20 '19 at 14:31
Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/97629/discussion-between-markgraf-and-sir-muffington). – markgraf Aug 20 '19 at 14:41
I tried out the script with close again and it works! – Sir Muffington Aug 22 '19 at 16:39

Compare if the newest file in the directory is different from the previous one

1 Answers1