Rsync (Remote Sync) | 20 Rsync command with Examples in Linux

Introduction

Remote Sync (Rsync) is common across all the Linux distribution, it can be RPM-based, Debian based Linux distribution or it can be BSD based operating system, Solaris and Mac OS as well. If you are good to work with anyone of the operating systems it’s more than enough to go through the below guide.

Syntax to run a Rsync Command

# rsync options  source                        destination
# rsync -avz     root@192.168.107.201:/root/   /root/

1. Install Remote Sync (Rsync) Package

In a minimal installed Operating system, rsync command will not be available. we need to install the package to start with Rsync.

# yum install rsync -y             # RHEL, CentOS, Oracle Linux
# sudo apt-get install rsync       # Ubuntu, Debian
# zypper install rsync             # OpenSuse/SLE Linux
# brew install rsync               # MacOS
# pkg install rsync                # FreeBSD

2. Copying from Source to Destination

Sync files from source to a remote server with archive, verbose and compress.

# rsync -avz /root/ root@192.168.107.101:/root
  • -a – archive mode
  • -v – Verbose mode
  • -z – Compress the files while in transfer

3. No action just Dry Run

Before starting with sync just do a dry run to check what are the files will be synced.

# rsync -avz --dry-run root@192.168.107.201:/root/* /root/
[root@corcls1 ~]# rsync -avz --dry-run root@192.168.107.201:/root/* /root/
 receiving incremental file list
 configfile1.conf
 configfile2.conf
 configfile3.conf
 configfile4.conf
 configfile5.conf
 largefile.iso
 largefile1.iso
 logfile1.log
 logfile2.log
 logfile3.log
 logfile4.log
 logfile5.log
 textfile1.txt
 textfile2.txt
 textfile3.txt
 textfile4.txt
 textfile5.txt
 sent 75 bytes  received 424 bytes  998.00 bytes/sec
 total size is 314,572,800  speedup is 630,406.41 (DRY RUN)
[root@corcls1 ~]#

Good for the small number of files, large filesystem dry run will consume the equal time of actual file sync.

4. Syncing multiple Directories

Copy multiple different directories from the remote server to the source.

# rsync -avc root@192.168.107.101:/root/somefiles :/root/linuxsysadmins .

Separate each directory with “:

  • -c – Skip based on checksum

5. Syncing only Specific files

Sync only specific files from a remote server to the destination.

# rsync -avz root@192.168.107.101:/root/somefiles/{test1.txt,test2.txt} .

Mention all your specific files inside the curly brace with comma-separated.

6. Files with White space

Copying a filename with whitespace using rsync from remote to source servers.

# rsync -av root@192.168.107.101:'/root/somefiles/file\ with\ space.txt' .

Use backslash in every white space.

7. Showing the Progress

Transfer and show the progress for each file. This will help while we transfer large size files.

# rsync -avz --progress /root/ root@192.168.107.101:/root

8. Read a file and Sync

Remote Syncing from source to a destination only required listed files from a file.

Create a file and list out all the files need to be copied on the remote server.

# vim list.txt
[root@docker ~]# cat list.txt 
 ./somefiles/somelarge_file.iso
 ./somefiles/test10.txt
 ./somefiles/test3.txt
 ./somefiles/test5.txt
 ./somefiles/test7.txt
 ./somefiles/test9.txt
 [root@docker ~]#
# rsync -avz --progress --files-from=/root/list.txt /root/ root@192.168.107.101:/root/

Output for Reference

[root@docker ~]# rsync -avz --progress --files-from=/root/list.txt /root/ root@192.168.107.101:/root/
 building file list … 
 8 files to consider
 ./
 somefiles/
 somefiles/somelarge_file.iso
   1,073,741,824 100%  114.47MB/s    0:00:08 (xfr#1, to-chk=5/8)
 somefiles/test10.txt
      10,485,760 100%    9.59MB/s    0:00:01 (xfr#2, to-chk=4/8)
 somefiles/test3.txt
      10,485,760 100%   70.42MB/s    0:00:00 (xfr#3, to-chk=3/8)
 somefiles/test5.txt
      10,485,760 100%   40.32MB/s    0:00:00 (xfr#4, to-chk=2/8)
 somefiles/test7.txt
      25,165,824 100%   52.40MB/s    0:00:00 (xfr#5, to-chk=1/8)
 somefiles/test9.txt
      10,485,760 100%   18.59MB/s    0:00:00 (xfr#6, to-chk=0/8)
 sent 1,109,971 bytes  received 136 bytes  105,724.48 bytes/sec
 total size is 1,140,850,688  speedup is 1,027.69
 [root@docker ~]# 

9. Remote Sync (Rsync) through different Port

Using Rsync from local server to remote server with a non-standard (22) SSH port.

# rsync -avz -e "ssh -p 5254" /root/ root@192.168.107.101:/root/
  • -e – Specify the remote shell to use
  • -p – SSH Port or remote server.

10. Copy with Bandwidth Limit

Sync by limiting the bandwidth, This will be helpful in case your server is a most critical production system and needs to free from chocking the network.

$ rsync -avz --progress --bwlimit=56KB /home/babin/ubuntu-18.04.2-live-server-amd64.iso root@192.168.107.200:/root
  • –bwlimit – To Limit the bandwidth
sending incremental file list
 ubuntu-18.04.2-live-server-amd64.iso
       3,768,320   0%   55.02kB/s    4:23:46

11. Remove from Source (Remote server)

Here, I’m trying to sync the files from my remote server to my local desktop machine. Once the files completed syncing it will remove from the remote server.

# rsync -rvz --remove-source-files root@192.168.107.200:/home/ansible/* /home/babin/backup/
  • –remove-source-files – Remove the files from the remote server.

12. Limiting the size of a single file size

While syncing we can limit each file size to a maximum or minimum size. If you set a maximum size it will not transfer any file above the specified size. The same thing applied for minimum size as well.

--min-size=SIZE
--max-size=SIZE 

List the files in the source server

[root@corcls1 ~]# ls -lthr
 total 1.4G
 -rw-r--r--. 1 root root 1.0G Aug  8 19:52 largefile.iso
 -rw-r--r--. 1 root root 300M Aug  8 19:53 largefile2.iso
 -rw-r--r--. 1 root root 100M Aug  8 19:54 largefile4.iso
[root@corcls1 ~]#

From the above list, we need to transfer only the 100 MB file. Let us use –max-size=150MB to transfer the file not larger than 150 MB.

# rsync -avz --progress --max-size=150M /root/* root@192.168.107.201:/root/

Here we can see only one file transferred from source to destination. The file “largefile4.iso” is 100 MB in size.

[root@corcls1 ~]# rsync -avz --progress --max-size=150M /root/* root@192.168.107.201:/root/
 sending incremental file list
 largefile4.iso
     104,857,600 100%   78.16MB/s    0:00:01 (xfr#1, to-chk=0/3)
 sent 102,115 bytes  received 35 bytes  40,860.00 bytes/sec
 total size is 1,493,172,224  speedup is 14,617.45
 [root@corcls1 ~]#

Verify the transferred file on the destination server.

[root@corcls2 ~]# ls -lthr
 total 101M
 -rw-------. 1 root root 1.8K Aug  2 02:00 anaconda-ks.cfg
 -rw-r--r--. 1 root root 100M Aug  8 19:54 largefile4.iso
[root@corcls2 ~]#

13. Ignore empty directories during Remote Sync

These are the file and directories on the remote server.

[root@corcls2 ~]# ls -lthr
 total 300M
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile5.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile4.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile3.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile2.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile1.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile1.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile5.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile4.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile3.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile2.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile5.conf
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile4.conf
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile3.conf
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile2.conf
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile1.conf
 -rw-r--r--. 1 root root 100M Aug  8 20:49 largefile.iso
 -rw-r--r--. 1 root root 200M Aug  8 20:50 largefile1.iso
 drwxr-xr-x. 2 root root    6 Aug  8 22:01 somedir1
 drwxr-xr-x. 2 root root    6 Aug  8 22:01 someemptydir
[root@corcls2 ~]#

Ignore all the empty directories

# rsync -avz --prune-empty-dirs root@192.168.107.201:/root/* /root/

Use –prune-empty-dirs or “-m”.

[root@corcls1 ~]# rsync -avz --prune-empty-dirs root@192.168.107.201:/root/* /root/
 receiving file list … done
 configfile1.conf
 configfile2.conf
 configfile3.conf
 configfile4.conf
 configfile5.conf
 largefile.iso
 largefile1.iso
 logfile1.log
 logfile2.log
 logfile3.log
 logfile4.log
 logfile5.log
 textfile1.txt
 textfile2.txt
 textfile3.txt
 textfile4.txt
 textfile5.txt
 sent 347 bytes  received 306,967 bytes  68,292.00 bytes/sec
 total size is 314,572,800  speedup is 1,023.62
[root@corcls1 ~]# 
[root@corcls1 ~]# ls -lthr
 total 300M
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile5.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile4.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile3.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile2.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 logfile1.log
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile1.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile5.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile4.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile3.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:44 textfile2.txt
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile5.conf
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile4.conf
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile3.conf
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile2.conf
 -rw-r--r--. 1 root root    0 Aug  8 20:48 configfile1.conf
 -rw-r--r--. 1 root root 100M Aug  8 20:49 largefile.iso
 -rw-r--r--. 1 root root 200M Aug  8 20:50 largefile1.iso
[root@corcls1 ~]# 

The somedir1 and someemptydir are not transferred to the destination.

14. Log everything in Destination server

While transferring files just we need to log all entries in the remote system.

# rsync -avz --remote-option=--log-file=/tmp/rsyslogstatus.log /root/ root@192.168.107.201:/root/

Logging will help to debug if any disconnectivity happened in the middle of any sync.

[root@corcls2 ~]# ls -lthr /tmp/rsyslogstatus.log 
 -rw-r--r--. 1 root root 1.1K Aug  9 01:09 /tmp/rsyslogstatus.log
 [root@corcls2 ~]# 
 [root@corcls2 ~]# cat /tmp/rsyslogstatus.log
 2019/08/09 01:09:01 [14336] receiving file list
 2019/08/09 01:09:01 [14336] .d..t…… ./
 2019/08/09 01:09:01 [14336] >f+++++++++ configfile1.conf
 2019/08/09 01:09:01 [14336] >f+++++++++ configfile2.conf
 2019/08/09 01:09:01 [14336] >f+++++++++ configfile3.conf
 2019/08/09 01:09:01 [14336] >f+++++++++ configfile4.conf
 2019/08/09 01:09:01 [14336] >f+++++++++ configfile5.conf
 2019/08/09 01:09:02 [14336] >f+++++++++ largefile.iso
 2019/08/09 01:09:04 [14336] >f+++++++++ largefile1.iso
 2019/08/09 01:09:04 [14336] >f+++++++++ logfile1.log
 2019/08/09 01:09:04 [14336] >f+++++++++ logfile2.log
 2019/08/09 01:09:04 [14336] >f+++++++++ logfile3.log
 2019/08/09 01:09:04 [14336] >f+++++++++ logfile4.log
 2019/08/09 01:09:04 [14336] >f+++++++++ logfile5.log
 2019/08/09 01:09:04 [14336] >f+++++++++ textfile1.txt
 2019/08/09 01:09:04 [14336] >f+++++++++ textfile2.txt
 2019/08/09 01:09:04 [14336] >f+++++++++ textfile3.txt
 2019/08/09 01:09:04 [14336] >f+++++++++ textfile4.txt
 2019/08/09 01:09:04 [14336] >f+++++++++ textfile5.txt
 2019/08/09 01:09:04 [14336] sent 350 bytes  received 307304 bytes  total size 314578933
[root@corcls2 ~]#

15. Append a file without replacing it

Appending to a file without overwriting the entire file. For instance, Copy a log file from source to destination and write the update from source to destination.

In below example, we have copied /var/log/messages from remote server 192.168.107.201 to our source server 192.168.107.200 under /root directory. Then to sync the content of the source and destination file we are using –append option.

# rsync --append root@192.168.107.201:/var/log/messages /root/

This will not overwrite the file again and again instead it will append the new contents.

16. Syncing with Include and exclude

We need to sync only the messages logs from one of the remote servers to source. In our example, we are syncing all messages log and excluding all other files.

# rsync -avz --include 'messages*' --exclude '*' root@192.168.107.201:/var/log/ /root/

17. Use Remote Shell while doing a Sync

For a secure synchronization, we can use a remote shell by forcing to use the SSH. This invokes a secure file transfer.

# rsync -avze ssh root@192.168.107.201:/home/linuxsysadmins/ /root/backup

18. Recursively back-up all changed files

In our example, we are syncing all log files “/var/log/” from source server 192.168.107.201 to destination server 192.168.107.200 under /root/log/. While performing the second sync if any changes happened on the log files on the source (192.168.107.201) a backup copy of existing files in the destination will be copied under /root/backup/. This help to keep our old files in a backup location.

# rsync -btr --backup-dir=/root/backup/ root@192.168.107.201:/var/log/ /root/logs/
  • -b – –backup – makes a backup of existing files
  • -t – –times – Preserve modification of times
  • -r – Copy the files recursively

19. Use as a Backup tool

Use Rsync as a Backup tool for large file system over the network

# rsync -avzPr --delete -e ssh root@192.168.107.201:/home/* /backup/users_files/

20. Getting Help of Remote Sync commands

To get to know more about Rsync run

# man rsync
# rsync --help

That’s it we have seen a few of Remote Sync (Rsync) examples which we are used in the production environment.

Conclusion

Remote Sync used to sync files securely on any nix servers, which can be used as a backup tool for the small environment as well. Subscribe for our newsletter and provide your feedback in the below comment section.

One thought on “Rsync (Remote Sync) | 20 Rsync command with Examples in Linux

Comments are closed.