How do I copy all files and directories, except certain ones over ssh?

Question

Using the Linux command line, I use the scp command, to copy all the files and folders from a certain directory. However, I don't like to consume bandwidth, for copying things I rarely change like my tiny_mce folder. What's the trick to copy everything, but skip a short list of folders?

Answer 1

rsync works fine, and in most cases, uses SSH automatically as it's transport protocol. It will compare files and only upload those that have changed - but you can also use an exclude list to specify files in the tree that shouldn't be rsynced anyhow.

Answer 2

You could try rsync which only copies files that have changed, also works over ssh.

Answer 3

Using rsync --exclude is the more obvious choice here, but if you really want to just send a few files or folders and have something specific to exclude, you can use shell globing with scp. First make sure you have the right globing options set in your shell. For bash run shopt -s extglob and for zsh use setopt ksh_glob. Then something like this:

scp /path/to/folder/!(tiny_mce|other_folder|*.bak) user@host:target_path

...would copy everything in the source folder except for things matching the given pattern. Obviously you can get creative with that part.

Answer 4

rsync is a good solution, but if you're looking for an alternative:

Let's say, we have a directory "test" contain the directories "foo, bar, baz". In these dirs are a bunch of different file types:

test
|____bar
| |____1.jpg
| |____1.png
| |____1.txt
| |____2.jpg
| |____2.png
| |____2.txt
|____baz
| |____1.avi
| |____2.avi
| |____3.png
|____foo
| |____test.blah
|____test.txt

We want to copy everything except the PNGs

scp $(find /location/of/test -type f ! -name "*.png") # -> Note the logical NOT!!

In this example, the command will put all of the files into the same destination directory - this may not be the behavior you want.

Answer 5

A great tool you may want to try out is "lftp".

lftp sftp://etc.etc/ 
lftp> ls 
    --- remote listing ---
lftp> mirror -R -n local/ remote/

You can also use RSync over ssh

rsync -avzp -e ssh /this/dir/  remoteuser@remotehost:/remote/dir/

Should work.

Answer 6

Using Secure Copy - scp

scp -r file user@host:

To copy many file

scp /directory/* user@host:destinationPath

To copy some files

scp /directory/!(*.doc) user@host:destinationPath

It copies content of directory except .doc files

Answer 7

I would certainly recommend you rsync.

rsync -vra --exclude="what you want to exclude" -e ssh folder user@remotehost:/folder

Answer 8

I just finished writing how I prefer unison to rsync any day, since it

• doesn't need a daemon, other than ssh for transport
• lets me modify files on either side any time--multiple masters easily, while I only need to push a sync request from one side
• I am a stickler when it comes to modtimes, attributes/permissions, softlinks etc. No problems with that; for one project I even use 4 mirrors, one being a cygwin host. See my example crontab setup.
• supports exclusions like *.bak. Samples in my config file

Answer 9

This is what worked for me when I ran it from destination server.

rsync -av --progress user@servername:/sourcefolder /destinationfolder --exclude thefoldertoexclude

Answer 10

We can do it in two steps to ignore tiny_mce directory. Assuming all the directories start by lower case letters.

scp -r USER@HOSTNAME:~/FOLDER/[a-s]* .
scp -r USER@HOSTNAME:~/FOLDER/[u-z]* .

Also, change USER, HOSTNAME, and FOLDER to the real values.