Diff directories using modification time (mtime) and size instead of contents

Question

Is there an option to have diff (-q) not look at file contents and instead just look at size and mtime? If not, is there a tool similar to this that has the option?

score 4 · Accepted Answer · answered Apr 22 '15 at 01:07

4

Use rsync, but tell it not to copy or remove any files.

rsync -a -nv --delete a/ b/

answered Apr 22 '15 at 01:07

Gilles 'SO- stop being evil'

807,993
194
1,674
2,175

score 1 · Answer 2 · 2021-03-11T18:12:14.803

The next script is an improvement of the answer from here:

#!/bin/sh

# diffm.sh

# DIFF with Modification date - a .sh (dash; bash; zsh - compatible) 
# "diff utility"-like script that can compare files in two directory 
# trees by path, size and modification date

GetOS () {
    
    OS_kernel_name=$(uname -s)
    
    case "$OS_kernel_name" in
        "Linux")
            eval $1="Linux"
        ;;
        "Darwin")
            eval $1="Mac"
        ;;
        "CYGWIN"*|"MSYS"*|"MINGW"*)
            eval $1="Windows"
        ;;
        "")
            eval $1="unknown"
        ;;
        *)
            eval $1="other"
        ;;
    esac
    
}

DetectShell () {
    eval $1=\"\";
    if [ -n "$BASH_VERSION" ]; then
        eval $1=\"bash\";
    elif [ -n "$ZSH_VERSION" ]; then
        eval $1=\"zsh\";
    elif [ "$PS1" = '$ ' ]; then
        eval $1=\"dash\";
    else
        eval $1=\"undetermined\";
    fi
}

PrintInTitle () {
    printf "\033]0;%s\007" "$1"
}

PrintJustInTitle () {
    PrintInTitle "$1">/dev/tty
}

trap1 () {
    CleanUp
    printf "\nAborted.\n">/dev/tty
}

CleanUp () {
    
    #Restore "INTERRUPT" (CTRL-C) and "TERMINAL STOP" (CTRL-Z) signals:
    trap - INT
    trap - TSTP
    
    #Restore Initial Directory:
    cd "$initial_dir"
    
    #Clear the title:
    PrintJustInTitle ""
    
    #Restore initial IFS:
    #IFS=$old_IFS
    unset IFS
    
    #Set shell flags (enable globbing):
    set +f
}

DisplayHelp () {
    printf "\n"
    printf "diffm - DIFF by Modification date\n"
    printf "\n"
    printf "    What it does:\n"
    printf "        - compares the files in the two provided directory tree paths (<dir_tree1> and <dir_tree2>) by:\n"
    printf "            1. Path\n"
    printf "            2. Size\n"
    printf "            3. Modification date\n"
    printf "    Syntax:\n"
    printf "        <caller_shell> '/path/to/diffm.sh' <dir_tree1> <dir_tree2> [flags]\n"
    printf "        - where:\n"
    printf "            - <caller_shell> can be any of the shells: dash, bash, zsh, or any other shell compatible with the \"dash\" shell syntax\n"
    printf "            - '/path/to/diffm.sh' represents the path of this script\n"
    printf "            - <dir_tree1> and <dir_tree2> represent the directory trees to be compared\n"
    printf "            - [flags] can be:\n"
    printf "                --help or -h\n"
    printf "                    Displays this help information\n"
    printf "    Output:\n"
    printf "        - lines starting with '<' signify files from <dir_tree1>\n"
    printf "        - lines starting with '>' signify files from <dir_tree2>\n"
    printf "    Notes:\n"
    printf "        - only the files in the two provided directory tree paths are compared, not also the folders\n"
    printf "\n"
}

Proc1 () {
    {\
        {\
            cd "$initial_dir"
            [ -n "$dir1" ] && { cd "$dir1"; PrintJustInTitle "Loading files in directory 1, please wait..."; }
            eval $command1
            cd "$initial_dir"
            [ -n "$dir2" ] && { cd "$dir2"; PrintJustInTitle "Loading files in directory 2, please wait..."; }
            eval $command2
            cd "$initial_dir"
        }|eval $sort_command;
    }|eval $uniq_command;
}

Proc2 () {
    cd "$initial_dir"
    [ -n "$dir1" ] && { cd "$dir1"; PrintJustInTitle "Loading files in directory 1, please wait..."; }
    eval $command1
    cd "$initial_dir"
    [ -n "$dir2" ] && { cd "$dir2"; PrintJustInTitle "Loading files in directory 2, please wait..."; }
    eval $command2
    cd "$initial_dir"
}

GetOS OS

DetectShell current_shell

OS_CASE=""
if [ "$OS" = "Linux" ]; then
    OS_CASE="1"; # = use Linux OS commands
elif [ "$OS" = "Mac" ]; then
    OS_CASE="2"; # = use Mac OS commands
else
    #################################################################################
    ##                  IN CASE YOUR OS IS NOT LINUX OR MAC:                       ##
    ##    MODIFY THE NEXT VARIABLE ACCORDING TO YOUR SYSTEM REQUIREMENTS (e.g.:    ##
    ##    "1" (use Linux OS commands) or "2" (use Mac OS commands)):               ##
    #################################################################################
    OS_CASE="3"
fi

if [ "$current_shell" = "undetermined" ]; then
    printf "\nWarning: This script was designed to work with dash, bash and zsh shells.\n\n">/dev/tty
fi

#Get the program parameters into the array "params":
params_count=0
for i; do
    params_count=$((params_count+1))
    eval params_$params_count=\"\$i\"
done
params_0=$((params_count))

if [ "$params_0" = "0" ]; then #if no parameters are provided: display help
    DisplayHelp
    CleanUp && exit 0
fi


#Create a flags array. A flag denotes special parameters:
help_flag="0"
i=1;
j=0;
while [ "$i" -le "$((params_0))" ]; do
    eval params_i=\"\$\{params_$i\}\"
    case "${params_i}" in
    "--help" | "-h" )
        help_flag="1"
    ;;
    * )
        j=$((j+1))
        eval selected_params_$j=\"\$params_i\"
    ;;
    esac
    
    i=$((i+1))
done
selected_params_0=$j

#Rebuild params array:
for i in $(seq 1 $selected_params_0); do
    eval params_$i=\"\$\{selected_params_$i\}\"
done
params_0=$selected_params_0

if [ "$help_flag" = "1" ]; then
    DisplayHelp
else #Run program:
    
    error1="false"
    error2="false"
    error3="false"
    error4="false"
    
    { sort --help >/dev/null 2>/dev/null; } || { error1="true"; }
    { stat --help >/dev/null 2>/dev/null; } || { error2="true"; }
    { find --help >/dev/null 2>/dev/null; } || { error3="true"; }
    { uniq --help >/dev/null 2>/dev/null; } || { error4="true"; }
    if [ "$error1" = "true" -o "$error2" = "true" -o "$error3" = "true" -o "$error4" = "true" ]; then
        {
            printf "\n"
            if [ "$error1" = "true" ]; then printf '%s' "ERROR: Could not run \"sort\" (necessary in order for this script to function correctly)!"; fi
            if [ "$error2" = "true" ]; then printf '%s' "ERROR: Could not run \"stat\" (necessary in order for this script to function correctly)!"; fi
            if [ "$error3" = "true" ]; then printf '%s' "ERROR: Could not run \"find\" (necessary in order for this script to function correctly)!"; fi
            if [ "$error4" = "true" ]; then printf '%s' "ERROR: Could not run \"uniq\" (necessary in order for this script to function correctly)!"; fi
            printf "\n"
        }>/dev/stderr
        exit
    fi
    
    #Check program arguments:
    if [ "$params_0" -lt "2" ]; then
        printf '\n%s\n' "ERROR: To few program parameters provided (expected two: <dir_tree1> and <dir_tree2>)!">/dev/stderr
        exit 1
    elif [ "$params_0" -gt "2" ]; then
        printf '\n%s\n' "ERROR: To many program parameters provided (expected two: <dir_tree1> and <dir_tree2>)!">/dev/stderr
        exit 1
    fi
    
    initial_dir="$PWD" #Store initial dir
        
    #If two program arguments are provided (<dir_tree1> and <dir_tree2>) proceed to checking them:
    
    initial_dir="$PWD" #Store initial dir
    dir1=""
    dir2=""
    file1=""
    file2=""
    error_encountered="false"
    
    error1="false"
    error2="false"
    [ -e "$params_1" ] && {
        if [ -d "$params_1" ]; then
            cd "$params_1" >/dev/null 2>/dev/null && {
                dir1="$PWD"
                cd "$initial_dir"
            } || {
                error1="true"
            }
        elif [ ! -d "$params_1" ]; then
            file1="$params_1"
        fi
    }||{
        error2="true"
    }
    if [ "$error1" = "true" -o "$error2" = "true" ]; then
        printf '\n%s\n' "ERROR: PARAMETER1: \"$params_1\" does not exist as a directory/file or is not accessible!">/dev/stderr
        error_encountered="true"
    fi
    
    printf "\n">/dev/tty
    
    error1="false"
    error2="false"
    [ -e "$params_2" ] && {
        if [ -d "$params_2" ]; then
            cd "$params_2" >/dev/null 2>/dev/null && {
                dir2="$PWD"
                cd "$initial_dir"
            }||{
                error1="true"
            }
        elif [ ! -d "$params_2" ]; then
            file2="$params_2"
        fi
    }||{
        error2="true"
    }
    if [ "$error1" = "true" -o "$error2" = "true" ]; then
        printf '%s\n' "ERROR: PARAMETER2: \"$params_2\" does not exist as a directory/file or is not accessible!">/dev/stderr
        error_encountered="true"
    fi
    
    if [ "$error_encountered" = "true" ]; then
        printf "\n">/dev/stderr
        exit
    fi
    
    ## TYPE ///// PATH ///// SIZE ///// LAST TIME WRITE IN SECONDS ##
    
    if [ "$OS_CASE" = "1" ]; then #Linux OS
        if [ -n "$dir1" ]; then
            command1='find . -not -type d -exec stat -c "< ///// %n ///// %s ///// %Y" {} \;'
        else
            command1_string="$(stat -c "< ///// %n ///// %s ///// %Y" "$file1")"
            command1="printf '%s\n' \"\$command1_string\""
        fi
        if [ -n "$dir2" ]; then
            command2='find . -not -type d -exec stat -c "> ///// %n ///// %s ///// %Y" {} \;'
        else
            command2_string="$(stat -c "> ///// %n ///// %s ///// %Y" "$file2")"
            command2="printf '%s\n' \"\$command2_string\""
        fi
        command3='date -d @'
        cd "$initial_dir"
        
        sort_command="sort -k 3"
        
        uniq_command="uniq -u -f 2"
    elif [ "$OS_CASE" = "2" ]; then #Mac OS
        if [ -n "$dir1" ]; then
            command1='find . -not -type d -exec stat -f "< ///// %N ///// %z ///// %m" {} \;'
        else
            command1_string="$(stat -f "< ///// %N ///// %z ///// %m" "$file1")"
            command1="printf '%s\n' \"\$command1_string\""
        fi
        if [ -n "$dir2" ]; then
            command2='find . -not -type d -exec stat -f "> ///// %N ///// %z ///// %m" {} \;'
        else
            command2_string="$(stat -f "> ///// %N ///// %z ///// %m" "$file2")"
            command2="printf '%s\n' \"\$command2_string\""
        fi
        command3='date -j -f %s '
        cd "$initial_dir"
        
        sort_command="sort -k 3"
        
        uniq_command="uniq -u -f 2"
    else
        printf '\n%s\n\n' "Error: Unsupported OS!">/dev/stderr
        exit 1
    fi
    
    #Trap "INTERRUPT" (CTRL-C) and "TERMINAL STOP" (CTRL-Z) signals:
    trap 'trap1' INT
    trap 'trap1' TSTP
    
    old_IFS="$IFS" #Store initial IFS value
    IFS="
    "
    set -f #Set shell flags (disable globbing):
    found_previous="false"
    count=0
    skip=0
    if [ -n "$dir1" -o -n "$dir2" ]; then
        for line in $(\
            if [ -n "$dir1" -a -n "$dir2" ]; then\
                Proc1;\
            elif [ -z "$dir1" -o -z "$dir2" ]; then\
                Proc2;\
            fi;\
        ); do
            count=$((count+1))
            PrintJustInTitle "Analyzing file $count..."
            if [ -z "$current_line_file_type" ]; then
                current_line="$line"
                current_line_file_type="${line%%" ///// "*}"
                current_line_file_mtime_in_seconds="${line##*" ///// "}"
                current_line_file_type_path_and_size="${line%" ///// "*}"
                current_line_file_size="${current_line_file_type_path_and_size##*" ///// "}"
                current_line_file_type_path="${line%" ///// "*" ///// "*}"
                current_line_file_path="${current_line_file_type_path#*" ///// "}"
            else
                previous_line="$current_line"
                previous_line_file_type="$current_line_file_type"
                previous_line_file_mtime_in_seconds="$current_line_file_mtime_in_seconds"
                previous_line_file_type_path_and_size="$current_line_file_type_path_and_size"
                previous_line_file_size="$current_line_file_size"
                previous_line_file_type_path="$current_line_file_size"
                previous_line_file_path="$current_line_file_path"
                
                current_line="$line"
                current_line_file_type="${line%%" ///// "*}"
                current_line_file_mtime_in_seconds="${line##*" ///// "}"
                current_line_file_type_path_and_size="${line%" ///// "*}"
                current_line_file_size="${current_line_file_type_path_and_size##*" ///// "}"
                current_line_file_type_path="${line%" ///// "*" ///// "*}"
                current_line_file_path="${current_line_file_type_path#*" ///// "}"
                
                if [ ! "$skip" = "$count"  ]; then
                    if [ "$found_previous" = "false" ]; then
                        seconds_difference=$(($current_line_file_mtime_in_seconds - $previous_line_file_mtime_in_seconds))
                        if [ \
                            \( "$current_line" = "$previous_line" \) -o \
                            \( \
                                \( "$current_line_file_path" = "$previous_line_file_path" \) -a \
                                \( "$current_line_file_size" = "$previous_line_file_size" \) -a \
                                \( "$seconds_difference" = "1" -o "$seconds_difference" = "-1" \) \
                            \) \
                        ]; then
                            found_previous="true"
                            skip=$((count+1))
                        else
                            printf '%s\n' "$previous_line_file_type $previous_line_file_path - ""Size: ""$previous_line_file_size"" Bytes"" - ""Modified Date: ""$(eval $command3$previous_line_file_mtime_in_seconds)"
                            found_previous="false"
                        fi
                    else
                        printf '%s\n' "$previous_line_file_type $previous_line_file_path - ""Size: ""$previous_line_file_size"" Bytes"" - ""Modified Date: ""$(eval $command3$previous_line_file_mtime_in_seconds)"
                        found_previous="false"
                    fi
                else
                    found_previous="false"
                fi
            fi
        done
        #Treat last case separately:
        if [ "$count" -gt "0" ]; then
            if [ "$found_previous" = "false" ]; then
                printf '%s\n' "$current_line_file_type $current_line_file_path - ""Size: ""$current_line_file_size"" Bytes"" - ""Modified Date: ""$(eval $command3$current_line_file_mtime_in_seconds)"
            fi
        fi
    else
        line1=""
        line2=""
        for line in $(\
            if [ -n "$dir1" -a -n "$dir2" ]; then\
                Proc1;\
            elif [ -z "$dir1" -o -z "$dir2" ]; then\
                Proc2;\
            fi;\
        ); do
            if [ -z "$line1" ]; then
                line1="$line"
                line1_file_type="${line%%" ///// "*}"
                line1_file_mtime_in_seconds="${line##*" ///// "}"
                line1_file_type_path_and_size="${line%" ///// "*}"
                line1_file_size="${line1_file_type_path_and_size##*" ///// "}"
                line1_file_type_path="${line%" ///// "*" ///// "*}"
                line1_file_path="${line1_file_type_path#*" ///// "}"
            else
                line2="$line"
                line2_file_type="${line%%" ///// "*}"
                line2_file_mtime_in_seconds="${line##*" ///// "}"
                line2_file_type_path_and_size="${line%" ///// "*}"
                line2_file_size="${line2_file_type_path_and_size##*" ///// "}"
                line2_file_type_path="${line%" ///// "*" ///// "*}"
                line2_file_path="${line2_file_type_path#*" ///// "}"
                
                seconds_difference=$(($line2_file_mtime_in_seconds - $line1_file_mtime_in_seconds))
                if [ \
                    \( "$line2_file_size" = "$line1_file_size" \) -a \
                    \( \
                        \( "$line2_file_mtime_in_seconds" = "$line1_file_mtime_in_seconds" \) -o \
                        \( "$seconds_difference" = "1" -o "$seconds_difference" = "-1" \) \
                    \) \
                ]; then
                    :;
                else
                    printf '%s\n' "$line1_file_type $line1_file_path - ""Size: ""$line1_file_size"" Bytes"" - ""Modified Date: ""$(eval $command3$line1_file_mtime_in_seconds)"
                    printf '%s\n' "$line2_file_type $line2_file_path - ""Size: ""$line2_file_size"" Bytes"" - ""Modified Date: ""$(eval $command3$line2_file_mtime_in_seconds)"
                fi
            fi
        done
    fi
    
    CleanUp
fi

What it does:

Compares the files (recursively) in the two directory tree paths provided as parameters (we denote them as: <dir_tree1> and <dir_tree2>) by:
1. relative path
2. size
3. modification date
if it finds differences: it lists the relative paths of the files that are different and their details (size and date modified):
- relative file paths for files in <dir_tree1> are prefixed with '<'
- relative file paths for files in <dir_tree2> are prefixed with '>'

Notes:
- Only files are compared, not also directories
- Should work on Linux OS, Mac OS - without installing any additional tools

I realize that you aren’t the original author of this code. But the original author seems to have left the building, and you’ve taken over, so I’ll treat you as if you were the author, and put these comments here. (1) You clearly understand what functions are. They are great for reducing redundancy. But you don’t use them well. Over and over we see commands repeated, once for ``dir1`` and again for ``dir2``, or for slightly different situations. When you’re doing the same thing(s) repeatedly, you should consider putting them into a function. … (Cont’d) — G-Man Says 'Reinstate Monica', Mar 13 '21 at 02:02
(Cont’d) … (1b) As far as I can tell, `Proc1` is basically just `Proc2 | sort | uniq` — so why write the code twice? (2) Similarly, the strings `///// %n ///// %s ///// %Y` and `///// %N ///// %z ///// %m` are present four times each; that redundancy should be eliminated. (3) Similarly, sometimes you have the same code in both branches of an `if`-`then`-`else`. (4) Why do you use `/dev/stderr`? I believe that ```>&2``` is more portable. (5) Unix/Linux programs & scripts should not be verbose by default. IMO, the display of status should be optional (opt-in). … (Cont’d) — G-Man Says 'Reinstate Monica', Mar 13 '21 at 02:02
(Cont’d) … You should definitely not do it if the script isn’t running on a tty, or if `$TERM` is not `xterm`. (5b) And why do you have the functionality in two layers? `PrintInTitle` isn’t called anywhere but `PrintJustInTitle`; why make them two separate functions? (5c) And why are you writing anything other than the status line to `/dev/tty`? Error messages should be written to stderr. (6) There are comments in the code for the obvious stuff. You should add explanations for the tricky stuff. What field is `sort_command` sorting? What is `uniq_command` doing? … (Cont’d) — G-Man Says 'Reinstate Monica', Mar 13 '21 at 02:02
(Cont’d) … Why does `Proc1` call `sort_command` and `uniq_command` when `Proc2` doesn’t? What are `line1` and `line2`? What can you ever get `"$current_line" = "$previous_line"`? What are `skip` and `found_previous`? Why do you test `seconds_difference` for `1` or `-1` (but not `0`)? And what, exactly, does the script do when you run it with one directory and one file (or two files) as arguments? (6b) And some of the conditions seem unnecessarily complex. If `dir1` is non-null or `dir2` is non-null, do something; otherwise, if `dir1` is non-null and `dir2` is non-null, do something; … (Cont’d) — G-Man Says 'Reinstate Monica', Mar 13 '21 at 02:03
(Cont’d) … otherwise, if ``dir1`` is null or `dir2` is null, do something. Under what (real-world) conditions does the second thing get done? Under what conditions do we fall through and not do anything? (Note that there are only three possible combinations of two Boolean values, where order doesn’t matter.) (7) Does your script handle filenames with spaces? (I suspect that it doesn’t; I’m not going to run it on my system until I understand it better.) (8) Have you considered processing the argument list once, and not copying it into an array-that-isn’t-an-array? … (Cont’d) — G-Man Says 'Reinstate Monica', Mar 13 '21 at 02:03
(Cont’d) … (9) You have a lot of unneeded backslashes; for example, in `Proc1`. (10) You should try to use `eval` a ***lot*** less. (11) You broke the `IFS` assignment — your code now sets it to . (12) The only thing in `CleanUp` that needs to be there is ``PrintJustInTitle`` — everything else is shell-local settings that go away when the shell exits. (13) You should avoid using `[`…`-a`…`]` and `[`…`-o`…`]`. (14) You shouldn’t use `-not` in `find` unless you’re sure that it’s GNU find. (15) This answer would be better if it showed some example output. — G-Man Says 'Reinstate Monica', Mar 13 '21 at 02:03
Hello, I am the original author of both: https://unix.stackexchange.com/a/621962/456983 and https://unix.stackexchange.com/a/638761/456983 (this), but I was just trying to help, if the code doesn't fit you simply don't use it... — , Mar 15 '21 at 13:11

score 0 · Answer 3 · answered Apr 21 '15 at 16:59

0

Not with diff - you don't need to look into the files for that - but just compare that information with stat in shell, as in

if [[ $(stat -c%s_%Y file1) == $(stat -c%s_%Y file2) ]]
then echo equal
else echo different
fi

The stat command provides information from the file's inode, -c allows you to select the desired attributes (%s and %Y in your case).

answered Apr 21 '15 at 16:59

Janis

14,014
3
25
42

I'm looking to get a picture of the way the directory trees differ including missing files/folders at any point in the structure (similar to what diff show). – Jack Apr 21 '15 at 17:07
2

Then please adjust your question accordingly, any make also clear why `diff -r` (recursive diff) is not helpful in your case. Also provide input and an desired output sample, so that it's clear what you expect. – Janis Apr 21 '15 at 17:11
diff -q -r does not work because I want to save time if the files have the same place in the path, name and mtime. – Jack Apr 21 '15 at 17:23

Diff directories using modification time (mtime) and size instead of contents

3 Answers3

Linked