rsync exclude directory not working

Question

I am running Fedora 17 64-bit and the rsync --exclude=/home/ben/<dir> is not working as expected.

I am trying to rsync my home directory to a thumb drive, but I want to exclude certainly directories that hold cache files and build files. This is the command I'm using:

rsync --exclude=/home/ben/build/ --exclude=/home/ben/.ccache -arv /home/ben home-ben/

However, content from the ~/build and ~/.ccache is being copied by rsync. What am I doing wrong?

FYI my issue was using the list syntax `--exclude={'landing','studio'}` and I had a space after the `,`. The list must not contain any spaces. — owencm, Apr 21 '22 at 07:22

jw013 · Accepted Answer · 2013-07-18T19:39:11.970

Global rsync filter rules beginning with a leading / are anchored to the root of transfer. Quoting from the "INCLUDE/EXCLUDE PATTERN RULES" section of the man page:

if the pattern starts with a / then it is anchored to a particular spot in the hierarchy of files, otherwise it is matched against the end of the pathname. This is similar to a leading ^ in regular expressions. Thus "/foo" would match a name of "foo" at either the "root of the transfer" (for a global rule) or in the merge-file's directory (for a per-directory rule).

In your command (rsync ... -arv /home/ben home-ben/), the file /home/ben/foo would be transferred to home-ben/ben/foo. The root of transfer is home-ben and the correct filter path is /ben/foo. Thus,

to match /home/ben/.ccache you need a filter path of /ben/.ccache
to match /home/ben/build/ you need a filter path of /ben/build/

A more detailed explanation can be found in the "ANCHORING INCLUDE/EXCLUDE PATTERNS" section of the rsync(1) man page.

Note that simply leaving out the leading / is not necessarily what you want. Quoting again from the same man page section:

An unqualified "foo" would match a name of "foo" anywhere in the tree because the algorithm is applied recursively from the top down; it behaves as if each path component gets a turn at being the end of the filename. Even the unanchored "sub/foo" would match at any point in the hierarchy where a "foo" was found within a directory named "sub". See the section on ANCHORING INCLUDE/EXCLUDE PATTERNS for a full discussion of how to specify a pattern that matches at the root of the transfer.

Thus a filter pattern of build/ would match a build directory anywhere in /home/ben, even /home/ben/many/sub/directories/build/.

The key detail is that slashes matter. If you append a slash to the end of the source directory (rsync -a source/ destination) it will copy the contents of source, otherwise `rsync -a source destination` copies the source directory itself to the destination. This determines what the root of the transfer will be in order to inform the exclude expression. The proper use of / in the exclude expression is then explained above. Note in particular that `rsync --exclude='/dir1' source destination` won't exclude anything. — pgoetz, Jun 19 '15 at 13:38

score 17 · Answer 2 · edited Dec 14 '20 at 09:14

17

WARNING: Per @rubo77's comment below, this answer should only be used if you're willing to exclude these items everywhere in your source tree, not just at the root. See @jw013's answer above for a more thorough explanation.

I think your exclude paths need to be relative to the source path. Can you try this instead?

rsync --exclude=build/ --exclude=.ccache -arv /home/ben home-ben/

Here's a test, for comparison:

/tmp$ tree src dest
src
├── a
└── b
dest
└── foo/

/tmp$ rsync -nav --exclude /tmp/src/b /tmp/src /tmp/dest
sending incremental file list
src/
src/a
src/b

$ rsync -nav --exclude b /tmp/src /tmp/dest
sending incremental file list
src/
src/a

edited Dec 14 '20 at 09:14

lumbric

339
5
15

answered Jul 17 '13 at 17:41

HorsePunchKid

424
4
8

6

This answer is wrong. With a relative exclude path you would exclude the pattern everywhere! – rubo77 Feb 04 '18 at 05:07
1

This post is correct. The matching is text-based, not file inode-based. Whatever path the output of the `--verbose` command shows is the format you have to use. – Sridhar Sarnobat May 01 '18 at 21:34
This worked for me. – Zhenyu May 28 '18 at 11:57
What if the source path is '/' then absolute are the relative paths, but it still doesn't work. – Soerendip Sep 27 '18 at 18:51
1

This is the only thing that worked for me. – Jonathan Aug 30 '19 at 09:46
From the rsync docu: "if the pattern starts with a / then it is anchored to a particular spot in the hierarchy of files, otherwise it is matched against the end of the pathname." So if I understand correctly then @rubo77 s answer is correct instead of this one. – Darius Duesentrieb Sep 12 '20 at 14:21
This solution worked to match EVERYWHERE, and was what i was looking for. – Louis Loudog Trottier Apr 29 '22 at 00:27

rubo77 · Answer 3 · 2018-02-04T05:18:35.130

Your exclude pattern would work, if you rsync the whole /home path i.e. without a trailing slash:

rsync --exclude=/home/ben/build/ --exclude=/home/ben/.ccache -arv /home home-all/

But better only rsync the content of the folder with a trailing slash:

rsync --exclude=/build --exclude=/.ccache -arv /home/ben/ home-ben/

You can use an ignore-list too, see: https://askubuntu.com/questions/545655/backup-your-home-directory-with-rsync-and-skip-useless-folders

score 0 · Answer 4 · edited Oct 08 '21 at 12:04

0

In order to exclude a specific folder with rsync you need to consider the relative path in the command. In you case:

rsync --exclude=/ben/build/ --exclude=/ben/.ccache -arv /home/ben home-ben/

where "ben" is the folder you are syncing.

edited Oct 08 '21 at 12:04

slm

363,520
117
767
871

answered Dec 13 '18 at 16:24

Samoht Rusconi

1

score 0 · Answer 5 · answered Mar 24 '23 at 21:57

I haven't tested this, but this is the first time I've seen someone share this syntax, which was apparently introduced in rsync version 3.1.3.

rsync [other opts...] --exclude='*/exclude_dirname/' /src/ /dst/

Ref: https://serverfault.com/a/954813/144798

More discussion: https://github.com/littlebizzy/slickstack/issues/150

rsync exclude directory not working

5 Answers5

Linked

Related