48

I was copying hundreds of files to another computer using the scp command that I got the stalled error. Now I am going to copy the files again. Is there any way to avoid copying the already copied files?

Community
  • 1
B Faley
  • 4,213
  • 11
  • 37
  • 48

3 Answers3

76

You can use rsync for it. rsync is really designed for this type of operation.

Syntax:

rsync -avh /source/path/ host:/destination/path

or

rsync -a --ignore-existing /local/directory/ host:/remote/directory/

When you run it first time it will copy all content then it will copy only new files.

If you need to tunnel the traffic through a SSH connection (for example, for confidentiality purposes), as indicated by you originally asking for a SCP-based solution, simply add -e ssh to the parameters to rsync. For example:

rsync -avh -e ssh /source/path/ host:/destination/path
user
  • 28,161
  • 13
  • 75
  • 138
  • Is `--ignore-existing` really required? – B Faley Dec 14 '13 at 12:41
  • Because in this example of manual: `rsync -t *.c foo:src/` -> "This would transfer all files matching the pattern *.c from the current directory to the directory src on the machine foo. **If any of the files already exist on the remote system then the rsync remote-update protocol is used to update the file by sending only the differences**" – B Faley Dec 14 '13 at 12:46
  • 2
    I typically use `rsync -avuzh /source/path/ host:/destination/path`. That does the whole job in one command, and on subsequent invocations, only transfers the files that need it. – Edward Falk Jun 24 '16 at 21:07
  • I don't get why the first one (`-avh`) would do the requested thing. (the second one I get) – KansaiRobot May 09 '22 at 05:48
13

If you want to stick with scp for any reason, you might remove w permissions on your local data (if you have the right to do it) and scp won't touch it. To be more precise:

  1. chmod a-w local/*.tar.gz (in your local dir)
  2. scp remote/*.tar.gz local

This is not very efficient, nice but might help if you need a fast temporary solution without changing to something else than scp. (Kudos: scp without replacing existing files in the destination )

Community
  • 1
xhudik
  • 231
  • 2
  • 4
9

There was no exact answer on how to do this with SCP, since that was what the original question was asking.

sudo find /files/ -type f -exec chmod a-w {} \;

scp -r [email protected]:/file/location/* /files

When it's done copying the files change the permissions back:

sudo chmod a+w -R /files

Geoffrey Booth
  • 103
  • 4
Rick
  • 1,973
  • 1
  • 11
  • 5