[`--delay-updates`](#opt) (which doesn't affect the sorted transfer order, but
does make the final file-updating phase happen much more rapidly).
+## MULTI-HOST SECURITY
+
+Rsync takes steps to ensure that the file requests that are shared in a
+transfer are protected against various security issues. Most of the potential
+problems arise on the receiving side where rsync takes steps to ensure that the
+list of files being transferred remains within the bounds of what was
+requested.
+
+Toward this end, rsync 3.1.2 and later have aborted when a file list contains
+an absolute or relative path that tries to escape out of the top of the
+transfer. Also, beginning with version 3.2.5, rsync does two more safety
+checks of the file list to (1) ensure that no extra source arguments were added
+into the transfer other than those that the client requested and (2) ensure
+that the file list obeys the exclude rules that were sent to the sender.
+
+For those that don't yet have a 3.2.5 client rsync (or those that want to be
+extra careful), it is safest to do a copy into a dedicated destination
+directory for the remote files when you don't trust the remote host. For
+example, instead of doing an rsync copy into your home directory:
+
+> rsync -aiv host1:dir1 ~
+
+Dedicate a "host1-files" dir to the remote content:
+
+> rsync -aiv host1:dir1 ~/host1-files
+
+See the [`--trust-sender`](#opt) option for additional details.
+
## ADVANCED USAGE
The syntax for requesting multiple files from a remote host is done by
preserve the characters you give it. Thus, if you ask for a file with spaces
in the name, that's what the remote rsync looks for:
-> rsync -aiv host:'a simple file.pdf' /dest/
+> rsync -aiv host:'a simple file.pdf' /dest/
If you use scripts that have been written to manually apply extra quoting to
the remote rsync args (or to require remote arg splitting), you can ask rsync
--from0, -0 all *-from/filter files are delimited by 0s
--old-args disable the modern arg-protection idiom
--protect-args, -s no space-splitting; wildcard chars only
+--trust-sender trust the remote sender's file list
--copy-as=USER[:GROUP] specify user & optional group for the copy
--address=ADDRESS bind address for outgoing socket to daemon
--port=PORT specify double-colon alternate port number
The parameter may need to be quoted in some manner for it to survive the
shell's command-line parsing. Also keep in mind that a leading tilde (`~`) in
a pathname is substituted by your shell, so make sure that you separate the
-option name from the pathname using a space if you want the shell to expand it.
+option name from the pathname using a space if you want the local shell to
+expand it.
[comment]: # (Some markup below uses a literal non-breakable space when a backtick string)
[comment]: # (needs to contain a space since markdown strips spaces from the start/end)
Starting with 3.1.0, rsync will skip the sender-side removal (and output an
error) if the file's size or modify time has not stayed unchanged.
+ Starting with 3.2.6, a local rsync copy will ensure that the sender does
+ not remove a file the receiver just verified, such as when the user
+ accidentally makes the source and destination directory the same path.
+
0. `--delete`
This tells rsync to delete extraneous files from the receiving side (ones
By default, an exclude or include has both a server-side effect (to "hide"
and "show" files when building the server's file list) and a receiver-side
- effect (to "protect" and "risk" files when deletions are occuring). Any
+ effect (to "protect" and "risk" files when deletions are occurring). Any
rule that has no modifier to specify what sides it is executed on will be
instead treated as if it were a server-side rule only, avoiding any
"protect" effects of the rules.
A rule can still apply to both sides even with this option specified if the
- rule is given both the sender & receiver modifer letters (e.g., `-f'-sr
+ rule is given both the sender & receiver modifier letters (e.g., `-f'-sr
foo'`). Receiver-side protect/risk rules can also be explicitly specified
- to limit the deletions. This is saves you from having to edit a bunch of
- `-f'- foo'` rules into `-f'-s foo'` or `-f'H foo'` rules (not to mention
+ to limit the deletions. This saves you from having to edit a bunch of
+ `-f'- foo'` rules into `-f'-s foo'` (aka `-f'H foo'`) rules (not to mention
the corresponding includes).
See the [FILTER RULES](#) section for more information. See
behavior. The environment is always overridden by manually specified
positive or negative options (the negative is `--no-old-args`).
+ Note that this option also disables the extra safety check added in 3.2.5
+ that ensures that a remote sender isn't including extra top-level items in
+ the file-list that you didn't request. This side-effect is necessary
+ because we can't know for sure what names to expect when the remote shell
+ is interpreting the args.
+
This option conflicts with the [`--protect-args`](#opt) option.
0. `--protect-args`, `-s`
Note that this option is incompatible with the use of the restricted rsync
script (`rrsync`) since it hides options from the script's inspection.
+0. `--trust-sender`
+
+ This option disables two extra validation checks that a local client
+ performs on the file list generated by a remote sender. This option should
+ only be used if you trust the sender to not put something malicious in the
+ file list (something that could possibly be done via a modified rsync, a
+ modified shell, or some other similar manipulation).
+
+ Normally, the rsync client (as of version 3.2.5) runs two extra validation
+ checks when pulling files from a remote rsync:
+
+ - It verifies that additional arg items didn't get added at the top of the
+ transfer.
+ - It verifies that none of the items in the file list are names that should
+ have been excluded (if filter rules were specified).
+
+ Note that various options can turn off one or both of these checks if the
+ option interferes with the validation. For instance:
+
+ - Using a per-directory filter file reads filter rules that only the server
+ knows about, so the filter checking is disabled.
+ - Using the [`--old-args`](#opt) option allows the sender to manipulate the
+ requested args, so the arg checking is disabled.
+ - Reading the files-from list from the server side means that the client
+ doesn't know the arg list, so the arg checking is disabled.
+ - Using [`--read-batch`](#opt) disables both checks since the batch file's
+ contents will have been verified when it was created.
+
+ This option may help an under-powered client server if the extra pattern
+ matching is slowing things down on a huge transfer. It can also be used to
+ work around a currently-unknown bug in the verification logic for a transfer
+ from a trusted sender.
+
+ When using this option it is a good idea to specify a dedicated destination
+ directory, as discussed in the [MULTI-HOST SECURITY](#) section.
+
0. `--copy-as=USER[:GROUP]`
This option instructs rsync to use the USER and (if specified after a
include the destination.
CAUTION: keep in mind that a source arg with a wild-card is expanded by the
- shell into multiple args, so it is never safe to try to list such an arg
- without using this option. For example:
+ shell into multiple args, so it is never safe to try to specify a single
+ wild-card arg to try to infer this option. A safe example is:
> rsync -av --list-only foo* dest/
We will first cover the basics of how include & exclude rules affect what files
are transferred, ignoring any deletion side-effects. Filter rules mainly
affect the contents of directories that rsync is "recursing" into, but they can
-also affect a top-level item in the transfer that were specified as a argument.
+also affect a top-level item in the transfer that was specified as a argument.
The default for any unmatched file/dir is for it to be included in the
transfer, which puts the file/dir into the sender's file list. The use of an
(with a trailing space) will not exclude a file named "foo".
Exclude and include rules can specify wildcard [PATTERN MATCHING RULES](#)
-(similar to shell wilcards) that allow you to match things like a file suffix
+(similar to shell wildcards) that allow you to match things like a file suffix
or a portion of a filename.
A rule can be limited to only affecting a directory by putting a trailing slash
Aside: this copy could also have been accomplished using the [`-R`](#opt)
option (though the 2 commands behave differently if deletions are enabled):
-> rsync -aR x/y/file.txt host:/tmp/
+> rsync -aiR x/y/file.txt host:/tmp/
The following command does not need an include of the "x" directory because it
is not a part of the transfer (note the traililng slash). Running this command
This command would omit the zzz.txt file while copying "x" and everything else
it contains:
-> rsync -aiv -f'- zzz.txt' x host:/tmp/
+> rsync -ai -f'- zzz.txt' x host:/tmp/
### FILTER RULES WHEN DELETING
-By default a filter rule affects both the sender (as it creates its file list)
+By default the include & exclude filter rules affect both the sender
+(as it creates its file list)
and the receiver (as it creates its file lists for calculating deletions). If
no delete option is in effect, the receiver skips creating the delete-related
file lists. This two-sided default can be manually overridden so that you are
When deleting, an exclude protects a file from being removed on the receiving
side while an include overrides that protection (putting the file at risk of
-deletion). The default is for a file to be at risk (its safety depends on it
-matching a corresponding file from the sender).
+deletion). The default is for a file to be at risk -- its safety depends on it
+matching a corresponding file from the sender.
An example of the two-sided exclude effect can be illustrated by the copying of
a C development directory between 2 systems. When doing a touch-up copy, you
older rules are specified using [`--include`](#opt) and [`--exclude`](#opt) as
well as the [`--include-from`](#opt) and [`--exclude-from`](#opt). These are
limited in behavior but they don't require a "-" or "+" prefix. An old-style
-exclude rule is turned into a `+ name` filter rule (with no modifiers) and an
-old-style include rule is turned into a `- name` filter rule (with no
+exclude rule is turned into a "`- name`" filter rule (with no modifiers) and an
+old-style include rule is turned into a "`+ name`" filter rule (with no
modifiers).
Rsync builds an ordered list of filter rules as specified on the command-line
below. If you use a short-named rule, the ',' separating the RULE from the
MODIFIERS is optional. The PATTERN or FILENAME that follows (when present)
must come after either a single space or an underscore (\_). Any additional
-spaces and/or undeerscore are considered to be a part of the pattern name.
+spaces and/or underscores are considered to be a part of the pattern name.
Here are the available rule prefixes:
0. `exclude, '-'` specifies an exclude pattern that (by default) is both a
`hide` and a `protect`.
0. `include, '+'` specifies an include pattern that (by default) is both a
`show` and a `risk`.
-0. `merge, '.'` specifies a merge-file to read for more rules.
-0. `dir-merge, ':'` specifies a per-directory merge-file.
+0. `merge, '.'` specifies a merge-file on the client side to read for more
+ rules.
+0. `dir-merge, ':'` specifies a per-directory merge-file. Using this kind of
+ filter rule requires that you trust the sending side's filter checking, so
+ it has the side-effect mentioned under the [`--trust-sender`](#opt) option.
0. `hide, 'H'` specifies a pattern for hiding files from the transfer.
Equivalent to a sender-only exclude, so `-f'H foo'` could also be specified
as `-f'-s foo'`.
Equivalent to a receiver-only exclude, so `-f'P foo'` could also be
specified as `-f'-r foo'`.
0. `risk, 'R'` files that match the pattern are not protected. Equivalent to a
- receiver-only include, so `-f'P foo'` could also be specified as `-f'+r
+ receiver-only include, so `-f'R foo'` could also be specified as `-f'+r
foo'`.
0. `clear, '!'` clears the current include/exclude list (takes no arg)
- If a pattern contains a `/` (not counting a trailing slash) or a "`**`"
(which can match a slash), then the pattern is matched against the full
pathname, including any leading directories within the transfer. If the
- pattern doesn't contain a `/` or a "`**`", then it is matched only against
- the final component of the filename or pathname. For example, `foo` means
- that the final path component must be "foo" while `foo/bar` would match the
- last 2 elements of the path (as long as both elements are within the
- transfer).
+ pattern doesn't contain a (non-trailing) `/` or a "`**`", then it is matched
+ only against the final component of the filename or pathname. For example,
+ `foo` means that the final path component must be "foo" while `foo/bar` would
+ match the last 2 elements of the path (as long as both elements are within
+ the transfer).
- A pattern that ends with a `/` only matches a directory, not a regular file,
symlink, or device.
- A pattern that starts with a `/` is anchored to the start of the transfer
- path instead of the end. For example, `/foo` or `/foo/bar` match only
+ path instead of the end. For example, `/foo/**` or `/foo/bar/**` match only
leading elements in the path. If the rule is read from a per-directory
filter file, the transfer path being matched will begin at the level of the
filter file instead of the top of the transfer. See the section on
must match one character.
- a trailing `***` in the pattern is a shorthand that allows you to match a
directory and all its contents using a single rule. For example, specifying
- "`dir_name/***`" will match both the "dir_name" directory (as if "dir_name/"
+ "`dir_name/***`" will match both the "dir_name" directory (as if "`dir_name/`"
had been specified) and everything in the directory (as if "`dir_name/**`"
had been specified).
- a backslash can be used to escape a wildcard character, but it is only
- Option `-f'- /foo'` would exclude a file (or directory) named foo in the
transfer-root directory
- Option `-f'- foo/'` would exclude any directory named foo
-- Option `-f'- /foo/*/bar'` would exclude any file/dir named bar which is at
- two levels below a directory named foo, which must be at the root of the
- transfer
-- Option `-f'- /foo/**/bar'` would exclude any file/dir named bar two or more
- levels below a directory named foo, which must be at the root of the transfer
+- Option `-f'- foo/*/bar'` would exclude any file/dir named bar which is at two
+ levels below a directory named foo (if foo is in the transfer)
+- Option `-f'- /foo/**/bar'` would exclude any file/dir named bar that was two
+ or more levels below a top-level directory named foo (note that /foo/bar is
+ **not** excluded by this)
- Options `-f'+ */' -f'+ *.c' -f'- *'` would include all directories and .c
source files but nothing else
- Options `-f'+ foo/' -f'+ foo/bar.c' -f'- *'` would include only the foo
[`--prune-empty-dirs`](#opt) option will not judge a directory as being empty
even if it contains only files that the transfer rules omitted.
-Similarly, a transfer rule does not have any extra affect on which files are
+Similarly, a transfer rule does not have any extra effect on which files are
deleted on the receiving side, so setting a maximum file size for the transfer
does not prevent big files from being deleted.
git.samba.org / rsync.git / blobdiff