4

I have been running the following rsync command as a cronjob for the past month. Today I discovered that this has been getting stuck on a *.png file and none of my other data has been backing up! In the meantime I've excluded *.png files and rsync has been working again, but I'd like a comprehensive solution to back up ALL of my files if possible.

This is getting stuck on *.png files

rsync -rcvl --delete --progress /home/user/target /media/user/backup/source

This is my temporary fix

rsync -rcvl --delete --progress --exclude='*.png' /home/user/target /media/user/backup/source

Each image is only ~300k.

The filesystems of both the target and source directories are fuseblk. The source dir is a Western Digital 4TB external hard-drive that is connected via USB 3.0.

EDIT: I was able to pin-down the problem after piping my rsync command into a grep failed. This revealed several files that had : characters in the file name, which rsync was unable to process. After renaming each of these files (replacing each : with _), and removing the --exclude='*.png' flag, the above rsync command successfully backed up my system!

philiporlando
  • 153
  • 6
  • 2
    How large are the large files and what filesystem type does the destination directory live on? – Kusalananda Nov 26 '18 at 23:53
  • Yes it still gets stuck outside of crontab. The png starts to copy, but stalls at about 11% in. I'm still trying working on the strace results. – philiporlando Nov 27 '18 at 00:32
  • 1
    I suspect concurrency/botteneck issues writing into USB disks and fuse to boot. I would stop using fuse for starters, and use a filesystem supported by Linux for starters. The ideal situation would be them not being USB. We also do not know either you work limitation, the USB version and whether you are using USB for something else. – Rui F Ribeiro Nov 27 '18 at 08:52
  • Thanks for the info. I am using USB 3.0. I don't think I'll be able to reformat either drive at this point. There is too much data and no where else for me to store it while I change filesystems. – philiporlando Nov 27 '18 at 21:49
  • 1
    Is this to an NTFS filesystem? – roaima Nov 27 '18 at 23:08
  • I'm running Xubuntu on a 1TB SSD that is using the fuseblk filesystem. Both the source and target directories are using fuseblk unfortunately. I can't afford to switch filesystems at the moment. I had no idea that fuseblk would cause problems in the future. Lesson learned! – philiporlando Nov 27 '18 at 23:32

0 Answers0