Using rsync to create file versions only when changes are detected?

Question

I've recently been experimenting with using rsync as a means of version control/backup for certain files on my system.

My goal is to essentially have rsync set to monitor some specific files, and if/when those files change for rysnc to then make a copy of said files to another destination, as a separate file (rather than overwriting the initial backed-up file).

So currently, what I do have is this:

rsync ~/Database/original.sqlite ~/Backup/backup`date +'.%F_%H-%M-%S'`.sqlite

I have this running as an Agent in launchd on a 1 minute timer. So with how it is now, essentially rsync will just keep making new copies of this file every minute no matter what.

If I had a means for rsync to check the latest version in the Backup folder, compare to original file in Database folder, then if the files are different make a new backup version, that would be perfection.

Am I really stretching the limits of rsync with what I'm trying to do here?

Thank you in advance for any suggestions.

Answer 1

You can probably use:

rsync -au ~/Database/original.sqlite ~/Backup/

The -u option means

--update, -u skip files that are newer on the receiver

So, rsync will only overwrite files in the destination folder when it's newer on the source.

Then, you can monitor changes of your specific files in the destination folder, and make copies if necessary.

-- Update --

To monitor the change of the file, you can do something along the lines of the following script (adapted from the linked question). Note that this requires inotify-tools (sudo apt install inotify-tools in Ubuntu)

#! /bin/bash

f=~/Backup/original.sqlite
touch $f

inotifywait -m -e modify "$f" --format "%e" | while read -r event; do
    if [ "$event" == "MODIFY" ]; then
        prev="$curr"
        curr=$(<"$f")
        [ "$curr" == "$prev" ] || echo "$f changed: do something"
    fi
done

If you run the script, it will monitor change. The script just does echo "$f changed: do something" as an example, but you can adapt it to do anything you like. e.g. cp $f destination_file.

Answer 2

If this is always running, the simplest solution to me would be just to check if the file has been edited in the last minute, and if so, do the backup.

#!/bin/bash

# get current time
now="$(date +'%s')"

# get modification time of fime
modified="$(stat -c '%Y' ~/Database/original.sqlite)"

# check if modified in the last minute
if (( modified > (now - 60) )) ; then
    rsync ~/Database/original.sqlite ~/Backup/backup`date +'.%F_%H-%M-%S'`.sqlite
fi

If you want it to be really robust, you could either write the name of the last backup, or its timestamp, to a file, and then check against that. Not sure if you're trying to keep it simpler than that, however.

I suppose another option would be to have the script itself be in charge of firing every minute and have launchd just run it once in the background at boot. Then the script could keep track the last modification time itself:

#!/bin/bash

# initialize last backup at 0
lastbackup=0
while true ; do

    # get modification time
    modified="$(stat -c '%Y' ~/Database/original.sqlite)"

    # check to see if it's been changed
    if (( modified > lastbackup )) ; then

        # do backup
        rsync ~/Database/original.sqlite ~/Backup/backup`date +'.%F_%H-%M-%S'`.sqlite

        # set the variable to its current modification time
        lastbackup="$modified"

    fi
done