rsync command in Linux with Examples

rsync or remote synchronization is a software utility for Unix-Like systems that efficiently sync files and directories between two hosts or machines. One of them being the source or the local-host from which the files will be synced, the other one being the remote-host, on which synchronization will take place. There are basically two ways in which rsync can copy/sync data:

• Copying/syncing to/from another host over any remote shell like ssh, rsh.
• Copying/Syncing through rsync daemon using TCP.

Rsync is famous for its delta-transfer algorithm, in which it copies only the differences between the source files present in the local-host and the existing files in the destination or the remote host.

Example:

rsync local-file user@remote-host:remote-file

What Happens here: Rsync will first use SSH to connect as user to remote-host and will ask for user's password. Once connected, it will invoke the remote host’s rsync and then the two programs will determine what parts of the local-file needs to be copied so that the remote file matches the local one. Please note the following behavior of rsync:

• Files that do not exist on the remote-host are copied.
• Files that have been updated will be synced, rsync will copy only the changed parts of files to the remote host.
• File that is exactly the same are not copied to the remote host at all.

Syntax of rsync:

rsync [options] source [destination]

Options:

• -a, –archive: This is equivalent to using -rlptgoD. Archive mode includes all the necessary options like copying files recursively, preserving almost everything (like symbolic links, file permissions, user & group ownership and timestamps).

  -a, --archive
  

  Note: The archive mode does not preserve hard links, because finding multiply-linked files is expensive. A -H options must be explicitly specified for hard links.

• -v, –verbose: By default, rsync works silently. A single -v will give us information about what files are being transferred and a brief summary about the data transferred at the end. Two -v options will give us information on the status of delta-transmission and on what files are up to date so as to be skipped and slightly more information at the end. More than two -v options are generally used for debugging rsync.

  -v, --verbose
  

• -h, –human-readable format: Outputs in a human readable format.

  -h, --human-readable format
  

• -z, –compress: Compress file data during the transfer

  -z, --compress
  

Examples:

Using rsync as a list command: If only the source path is specified, the contents of the source are listed in an output format similar to ls -l.

rsync foo/ 

The above command will list the files and directories present in the directory foo.

Output:

Copy/Sync files and directory locally: If neither the source or destination path specifies a remote host, the rsync commands behave as a copy command.

rsync -avh foo/ bar/ 

The above command will copy/sync all the files and directories present in directory foo to directory bar. If the destination directory is not present (here bar), rsync automatically creates one and copies all the data in it.

Output:

Rsync using ssh: There are two different ways for rsync to contact a remote system:

• Using a remote-shell program as the transport(such as ssh(Secure Shell) or rsh(Remote Shell)).
• Contacting an rsync daemon directly via TCP.

Here we will be discussing rsync over ssh.

rsync -avhze ssh /foo user@remote-host:/tmp/

To specify the type of protocol to be used, -e option is used.

Output:

Rsync with particular file permissions: If we want to sync files to the local or remote host with the permissions of the files being changed. The following command must be used.

rsync -avhe ssh --chown=USER:GROUP /foo user@remote-host:/tmp/

The above command will sync all the files present in directory /foo with the files present in directory /tmp in the remote-host with all the files owned by USER with group GROUP.

Output:

Note: The user and group must already be created in the remote-host.

Rsync with --ignore-existing-files: We can also skip the already existing files on the destination. This can generally be used when we are performing backups using the –link-dest option, while continuing a backup run that got interrupted.

rsync --ignore-existing -avhe /foo user@remote-host:/tmp/

So any files that do not exist on the destination will be copied over. Here I have deleted the geeksforgeeks folder from the directory foo, so it should copy only the geeksforgeeks directory.

Note: This does not ignore existing directories, or nothing would get done. Even if there are some changes in a file in the local host, it still would not be synced if its present on the remote host.

Output:

Show progress during transfer: To show the progress while transferring the data from local-host to remote-host, we can use -–progress option.

rsync -avhe ssh --progress /foo user@remote-host:/tmp/

When the transfer completes for a particular file, rsync outputs a summary line as shown below.

Output:

In the above image, if we look at the file /foo/file2, it tells us that

• The file was 44 bytes long in total.
• The average rate of transfer for the whole file was 42.97 kilobytes per second over the 0:00:00 seconds that it took to complete.
• It was the second transfer during the current rsync session.
• There are 10 more files for the remote-host to check (to see if they are up-to-date or not) remaining out of the 13 total files in the file-list.

Update the remote only if there is a newer version is on the local filesystem: If we want to copy files over the remote-host that have been updated more recently on the local filesystem. It is done with –update flag. The behavior is now like this:

• Files that do not exist on the remote-host are copied.
• Files that exist on both local and remote but have a newer timestamp on the local-host are copied to remote-host. (Conversely, files that have an older timestamp are not copied).

Here, I made some changes in file1 and file2, but the changes in file2 were done recently. So only, file2 will get synced.

rsync -avhe ssh --progress --update /foo root@remote-host:/tmp/

Output:

Automatically delete files from local-host after successful transferring: Now, suppose we have a web server and a backup server and we created a daily backup and synced it with our backup server and then we don’t want to keep the local copy of the backup in our web server. So, instead of deleting it manually after successful transfer, we can use the --remove-source-files flag to automatically delete the files from the web server.

rsync -avhe ssh --remove-source-files /foo user@backup-server:/tmp

Output:

Note: This will delete only the files and not the directories.

Delete the files that have been deleted on the local-host: If there are some files that are deleted on the local-host and we want that to be updated on the remote host as well, then we need to use the --delete option.

rsync -avhe ssh  /foo --delete user@remote-host:/tmp/

Output:

So, here file1, file2, file3 were deleted on the local-host, and as can be seen, are updated in the remote-host as well.

Note:rsync does not delete the files automatically to sync the directories present on both sides.

Performing a Dry run with rsync: A Dry run makes rsync perform a trial run that doesn’t make any changes and displays almost the same output as a real run would do. It is generally used with the -v, –verbose and/or -i, –itemize-changes options so as to see what an rsync command would do before one actually runs it.

rsync -avhe ssh --dry-run --chown=USER:GROUP /foo user@remote-host:/

Output: