3

I am trying to backup bunch of desktops running RedHat 6.4 (i86_64) LVM on the top of ext4 to our file server running RedHat using rsnapshot and ssh. I am on 1GbE. I am finding the speed of the first back up abysmal. It takes me about 1 hour to push 1Gb of data. This is my rsnapshot.conf file. What am I doing wrong? Could people suggest some alternatives to rsnapshot.

config_version  1.2

snapshot_root   /mnt/backup/desktops/test/

cmd_cp          /bin/cp
cmd_rm          /bin/rm
cmd_rsync       /usr/bin/rsync 
cmd_ssh /usr/bin/ssh
cmd_logger      /usr/bin/logger
cmd_du          /usr/bin/du
#cmd_preexec    /path/to/preexec/script
#cmd_postexec   /path/to/postexec/script

linux_lvm_cmd_lvcreate  /sbin/lvcreate
linux_lvm_cmd_lvremove  /sbin/lvremove
linux_lvm_cmd_mount     /bin/mount
linux_lvm_cmd_umount    /bin/umount


linux_lvm_snapshotsize  2G
linux_lvm_snapshotname  rsnapshot
linux_lvm_vgpath        /dev
linux_lvm_mountpath     /mnt/lvm-snapshot


retain  hourly  6


#              GLOBAL OPTIONS              #
verbose         2
loglevel        3
logfile /var/log/rsnapshot
lockfile        /var/run/rsnapshot.pid

# Default rsync args. All rsync commands have at least these options set.
rsync_short_args        -az
rsync_long_args --delete --numeric-ids --relative --delete-excluded --rsync-path
="sudo /usr/bin/rsync"

# ssh has no args passed by default, but you can specify some here.
ssh_args        -p 22 -i /root/.ssh/backup_key_dsa


exclude '*.core'

[email protected]:/home/me/     land/  +rsync_long_args=--bwlimit=16
Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
Predrag Punosevac
  • 329
  • 1
  • 12
  • 2
    You've got --bwlimit set to 16KB/sec but the rsync throttling mechanism is crude. What happens if you double that number or remove the argument altogether? – msw Oct 06 '13 at 03:20
  • @msw Removing --bwlimit did a trick. I should have been more careful accepting default values from a sample file. Additionally I am doing -c blowfish ssh which furthers speeds the thing. Our NFSv4 has a problem with large files over 4GB otherwise I would mount a damn thing. – Predrag Punosevac Oct 06 '13 at 04:50

1 Answers1

3

SSH is pretty slow as a protocol for massive transmission of data, I hit a fraction of my network/disk speeds when doing transfers like this too. One thing you can do to improve performance - tho I have no idea how you would implement it - switch the "cipher" to blowfish, which I believe is the fastest of the SSH cyphers. I'm not really qualified to comment on the overall security of that tho, but I'm imagining you're pushing over LAN and not overly worried about the fairly dedicated hacking that would be needed to break into the stream. Of course switching to a non encrypted protocol provides the best gain.

(ssh -c blowfish / scp -c blowfish is the standard ssh arguments for this cipher)

(if rsnapshot is based around rsync it used to have its own rsync:// protocol which doesn't have the overheads of ssh, that might be another path for you too)

iain
  • 788
  • 4
  • 6
  • 1
    `arcfour` is typically the fastest ssh cipher. – Martin Schröder Oct 06 '13 at 23:01
  • @Martin Schröder I am getting OK speed with the blowfish cipher but your comment is very valuable as usual. Thank you. – Predrag Punosevac Oct 07 '13 at 02:46
  • This seemed relevant http://blog.famzah.net/2010/06/11/openssh-ciphers-performance-benchmark/ and confirms arcfour as the fastest set with arcfour128/arcfour256 as the fastest overall. Looks like my blowfish thing dates back to ssh1, or bad reading of a man page once upon a time ;) – iain Oct 07 '13 at 13:06