The rsync command in Linux stands for remote sync.
Generally, we use rsync to perform the backup process in Linux/UNIX.
Linux utility rsync is very powerful to synchronize files and directories from one location to another.
Now, let’s discuss on 15 practical uses of rsync command:
Synchronize two directories in the same system
It is the primary use of rsync command, and to perform this operation just using rsync command with “-zvr” option like below:
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyDigJN6dnIgL2hvbWUvbGludXhjb25jZXB0L25ld3dlYi8gL2hvbWUvNTEwODQ4LmNsb3Vkd2F5c2FwcHMuY29tL2txYXdteXNuY2cvcHVibGljX2h0bWwvIApidWlsZGluZyBmaWxlIGxpc3Qg4oCmIGRvbmUgCmhvbWUuaHRtbCAKaW5kZXguaHRtbCAKLiAKc2VudCAyMzQ1MiBieXRlcyAJcmVjZWl2ZWQgMTAyMyBieXRlcyAJNTYzNDIzLjAwIGJ5dGVzL3NlYyAKdG90YWwgc2l6ZSBpcyAzNDIzMyAJc3BlZWR1cCBpcyAxLjMyCiQ="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
In the above example we use three options:
- -z we use to enable compression
- -v is for run command in verbose mode
- -r is used to recursive execution of the command
Now we can check the timestamp for both the files copied and source; you can see both file’s timestamp is different is not preserve timestamps during sync.
[et_pb_dmb_code_snippet 0=”code="JCBscyDigJNsIC9ob21lL2xpbnV4Y29uY2VwdC9uZXd3ZWIvaW5kZXguaHRtbCAvaG9tZS81MTA4NDguY2xvdWR3YXlzYXBwcy5jb20va3Fhd215c25jZy9wdWJsaWNfaHRtbC9pbmRleC5odG1sIArigJMgciDigJMg4oCTIHIg4oCTIOKAkyByIOKAkyDigJMgMSBiaW4gYmluIDk0OSBTZXAgMTggMjAxOCAvaG9tZS9saW51eGNvbmNlcHQvbmV3d2ViL2luZGV4Lmh0bWwgCuKAkyByIOKAkyDigJMgciDigJMg4oCTIHIg4oCTIOKAkyAxIGJpbiBiaW4gOTQ5IEFwciAxMyAyMDE5IC9ob21lLzUxMDg0OC5jbG91ZHdheXNhcHBzLmNvbS9rcWF3bXlzbmNnL3B1YmxpY19odG1sL2luZGV4Lmh0bWw="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="369a0a83-f997-4079-bcdc-749886c9e1c7"”][/et_pb_dmb_code_snippet]
Using rsync –a to preserve timestamp
The rsync command use with an option “-a” to run command in archive mode. This option does all the following
- Recursive mode operation
- Preserves timestamp during sync
- Preserves Permissions during sync
- Preserver Owner and Group settings
- Preserves symbolic links, if have
Now, we execute the same previous command with “-a” option, as shown below:
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyDigJN6dmEgL2hvbWUvbGludXhjb25jZXB0L25ld3dlYi8gL2hvbWUvNTEwODQ4LmNsb3Vkd2F5c2FwcHMuY29tL2txYXdteXNuY2cvcHVibGljX2h0bWwvIApidWlsZGluZyBmaWxlIGxpc3Qg4oCmIGRvbmUgCmhvbWUuaHRtbCAKaW5kZXguaHRtbCAKLiAKc2VudCAyMzQ1MiBieXRlcyAJcmVjZWl2ZWQgMTAyMyBieXRlcyAJNTYzNDIzLjAwIGJ5dGVzL3NlYyAKdG90YWwgc2l6ZSBpcyAzNDIzMyAJc3BlZWR1cCBpcyAxLjMyIAok"” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
Now we can see the timestamp for both the files
[et_pb_dmb_code_snippet 0=”code="4oCTIHIg4oCTIOKAkyByIOKAkyDigJMgciDigJMg4oCTIDEgYmluIGJpbiA5NDkgU2VwIDE4IDIwMTggL2hvbWUvbGludXhjb25jZXB0L25ld3dlYi9pbmRleC5odG1sIArigJMgciDigJMg4oCTIHIg4oCTIOKAkyByIOKAkyDigJMgMSBiaW4gYmluIDk0OSBTZXAgMTggMjAxOCAvaG9tZS81MTA4NDguY2xvdWR3YXlzYXBwcy5jb20va3Fhd215c25jZy9wdWJsaWNfaHRtbC9pbmRleC5odG1s"” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="369a0a83-f997-4079-bcdc-749886c9e1c7"”][/et_pb_dmb_code_snippet]
rsync use to synchronize only one file
We can use rsync command to copy one file also, as shown below:
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyDigJN2IC9ldGMvc3NsL2xpbnV4Y29uY2VwdC9wdWJrZXlzIC9ob21lL2xpbnV4Y29uY2VwdC9wdWJrZXlzIApwdWJrZXlzIApzZW50IDU2IGJ5dGVzIAlyZWNlaXZlZCAyNDMyMSBieXRlcyAJMzQ1Ni4xNCBieXRlcy9zZWMgCnRvdGFsIHNpemUgaXMgMTIyODggCXNwZWVkdXAgaXMgMC43Ng=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
rsync also use to synchronize files from local system to remote system
The rsync command allows us to synchronize files/directories between local and remote system.
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyAtYXZ6IC9yb290L3RlbXAvIGxpbnV4Y29uY2VwdEAxOTIuMTY4LjIuMTA6L2hvbWUvbGludXhjb25jZXB0L3RlbXAvIApQYXNzd29yZDogCmJ1aWxkaW5nIGZpbGUgbGlzdCDigKYgZG9uZSAKLi8gCnNhdGlzaC8gCnNhdGlzaC9hYmMudHh0IApzYXRpc2gvdGVzdC50eHQgCnNlbnQgMTU4MTU1NjEgYnl0ZXMgCXJlY2VpdmVkIDQ1MiBieXRlcyAJMjQyNTQxMS4yMyBieXRlcy9zZWMgCnRvdGFsIHNpemUgaXMgNDUwMjQ2NTggCXNwZWVkdXAgaXMgMi4xNw=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
We are doing file synchronization with remote server/system, we need to specify the remote system IP and username to login on the remote system and specify the location on the remote system to sync file in the particular location.
As you can see in the above example, it asks for a password to login on the remote system.
Sometimes we don’t want to enter a password and execute the same command from a script when we want to perform the backup operation and schedule it for automation.
To perform rsync without password, you need to set up ssh passwordless login.
Use rsync command to synchronize files from remote system to local system.
This is similar to previous one, here we specify the remote path into source and local system path into destination.
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyAtYXZ6IGxpbnV4Y29uY2VwdEAxOTIuMTY4LjIuMTA6L2hvbWUvbGludXhjb25jZXB0L3NhdGlzaCAvcm9vdC90ZW1wIApQYXNzd29yZDogCnJlY2VpdmluZyBmaWxlIGxpc3Qg4oCmIGRvbmUgCnNhdGlzaC8gCnNhdGlzaC9hYmMudHh0IAouIApzZW50IDQyNiBieXRlcyAJcmVjZWl2ZWQgMTU2NTIzMCBieXRlcyAJMjQzNzYzNS41NCBieXRlcy9zZWMgCnRvdGFsIHNpemUgaXMgNDUzNDU2NTggCXNwZWVkdXAgaXMgMi4zNw=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
Specify remote shell into rsync
The rsync utility allows specifying the remote shell into a command which you want to use. Like the below example we use ssh to enable the secure connection with a remote system.
You can use “-e” option to specify the shell in the command, as shown below:
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyAtYXZ6IC1lIHNzaCBsaW51eGNvbmNlcHRAMTkyLjE2OC4yLjEwOi9ob21lL2xpbnV4Y29uY2VwdC9zYXRpc2ggCi9yb290L3RlbXAgClBhc3N3b3JkOiByZWNlaXZpbmcgZmlsZSBsaXN0IOKApiBkb25lIApzYXRpc2gvIApzYXRpc2gvQmFzZW5hbWVzIApzZW50IDQxNiBieXRlcyAJcmVjZWl2ZWQgMTUzNDUyMzAgYnl0ZXMgCTIxNDU2MDUuNTQgYnl0ZXMvc2VjIAp0b3RhbCBzaXplIGlzIDQ1NjcyODU4IAlzcGVlZHVwIGlzIDIuMzc="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
Use rsync without overwriting in destination
In general, if the files are modified in the destination location, we don’t want to overwrite those files.
To prevent files from overwriting you can use the “-u” option. In the below example you can see the file abc.txt is already modified, So it will not be overwritten while using rsync –u command.
[et_pb_dmb_code_snippet 0=”code="JCBscyAtbCAvcm9vdC90ZW1wL2FiYy50eHQgCnRvdGFsIDM5MDg4IAotcnd4ci14ci14IDEgcm9vdCByb290IDQwOTYgT2N0IDIgMTE6MzUgYWJjLnR4dCAKJCByc3luYyAtYXZ6dSBsaW51eGNvbmNlcHRAMTkyLjE2OC4yLjEwOi9ob21lL2xpbnV4Y29uY2VwdC9zYXRpc2ggL3Jvb3QvdGVtcCAKUGFzc3dvcmQ6IApyZWNlaXZpbmcgZmlsZSBsaXN0IOKApiBkb25lIApzYXRpc2gvIApzZW50IDExMiBieXRlcyAJcmVjZWl2ZWQgNTM1IGJ5dGVzIAkxMDQuMDAgYnl0ZXMvc2VjIAp0b3RhbCBzaXplIGlzIDQ1NjAzNTU4IAlzcGVlZHVwIGlzIDM0MjU4LjMxIAokIGxzIC1scnQgCnRvdGFsIDM5MDg4IAotcnd4ci14ci14IDEgcm9vdCByb290IDQwOTYgT2N0IDIgMTE6MzUgYWJjLnR4dA=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
During rsync show progress
To show the progress on the terminal while transferring files from one system to another system using “- -progress” option. It will display the file name which is currently transfer and remaining time to transfer.
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyAtYXZ6aGUgc3NoIOKAk3Byb2dyZXNzIC9ob21lL2xpbnV4Y29uY2VwdCBsaW51eGNvbmNlcHRAMTkyLjE2OC4yLjEwOi9yb290L3JwbXBrZ3MgCnBhc3N3b3JkOiAKc2VuZGluZyBpbmNyZW1lbnRhbCBmaWxlIGxpc3QgCmNyZWF0ZWQgZGlyZWN0b3J5IC9yb290L3JwbXBrZ3MKcnBtcGtncy8gCnJwbXBrZ3MvaHR0cGQtMi4yLjMtODIuZWw1LmNlbnRvcy5pMzg2LnJwbSAxLjAyTSAxMDAlIDIuNzJNQi9zIDA6MDA6MDAgKHhmZXIjMSwgdG8tY2hlY2s9My81KSAKcnBtcGtncy9tb2Rfc3NsLTIuMi4zLTgyLmVsNS5jZW50b3MuaTM4Ni5ycG0gOTkuMDRLIDEwMCUgMjQxLjE5a0IvcyAwOjAwOjAwICh4ZmVyIzIsIHRvLWNoZWNrPTIvNSkgCnJwbXBrZ3MvbmFnaW9zLTMuMy4wLnRhci5neiAxLjcyTSAxMDAlIDEuMDZNQi9zIDA6MDA6MDEgKHhmZXIjMywgdG8tY2hlY2s9MS81KSAKcnBtcGtncy9uYWdpb3MtcGx1Z2lucy0xLjMuMTYudGFyLmd6IDIuMDlNIDEwMCUgMS4yN01CL3MgMDowMDowMSAoeGZlciM0LCB0by1jaGVjaz0wLzUpIApzZW50IDQuNzlNIGJ5dGVzIAlyZWNlaXZlZCA5NCBieXRlcyAJNDY1LjU2SyBieXRlcy9zZWMgCnRvdGFsIHNpemUgaXMgNC45OU0gCXNwZWVkdXAgaXMgMS4wMA=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
Delete files in the destination
Sometimes we use rsync command to create a replica of any application, and in this case, we want to make destination directory exact similar to the source directory.
In this case, if some of the files are in the destination directory but not in the source directory, we wanted to delete them automatically.
To perform this operation you can use “- – delete” option like shown below:
[et_pb_dmb_code_snippet 0=”code="JCB0b3VjaCB0ZXN0aW5nLnR4dCAKJCByc3luYyDigJNhdnog4oCTIC1kZWxldGUgbGludXhjb25jZXB0QDE5Mi4xNjguMi4xMDovaG9tZS9saW51eGNvbmNlcHQvIC4gClBhc3N3b3JkOiAKcmVjZWl2aW5nIGZpbGUgbGlzdCAuIC4gLiBkb25lIApkZWxldGluZyB0ZXN0aW5nLnR4dCAKLi8gCnNlbnQgMjQgYnl0ZXMgCXJlY2VpdmVkIDM0MCBieXRlcyAJNDguNjQgYnl0ZXMvc2VjIAp0b3RhbCBzaXplIGlzIDQ1Njc4MjM0IAlzcGVlZHVwIGlzIDEwODcyLjM0"” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
Specify max file size with rsync command
You can also specify the maximum file size to be transferred while using rsync utility. To determine the maximum file size you can use “- -max-size” option in rsync command.
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyDigJNhdnpoZSBzc2gg4oCTIC1tYXgtc2l6ZT3igJk0MDBr4oCZIC9ob21lL2xpbnV4Y29uY2VwdC9zYXRpc2gvIGxpbnV4Y29uY2VwdEAxOTIuMTY4LjIuMTA6L2hvbWUvbGludXhjb25jZXB0L3RtcCAKUGFzc3dvcmQ6IApzZW5kaW5nIGluY3JlbWVudGFsIGZpbGUgbGlzdCAKY3JlYXRlIGRpcmVjdG9yeSAvaG9tZS9saW51eGNvbmNlcHQvdG1wIAouLyAKYWJjLnR4dCAKdGVzdGluZy50eHQgCmluZGV4Lmh0bWwgCmhvbWUuaHRtbCAKc2VudCAxNjguNDVLIGJ5dGVzIAlyZWNlaXZlZCAyMzQgYnl0ZXMgCTE0LjIwSyBieXRlcy9zZWMgCnRvdGFsIHNpemUgaXMgMzQuMDZNIAlzcGVlZHVwIGlzIDEzNC40Mw=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
Delete source files after successful execution of rsync
Suppose you are taking backup your production data every day on your production server and transferring on the local system using rsync and don’t want to keep after that copying into the local system.
In this case, we can delete the source file automatically after copying data to the local system. To perform this operation, you can use “- -remove-source-files’ option.
[et_pb_dmb_code_snippet 0=”code="JHJzeW5jIOKAk3p2aCDigJMgLXJlbW92ZS1zb3VyYy1maWxlcyB3ZWItYmFja3VwLnRhciAvaG9tZS9saW51eGNvbmNlcHQvYmFja3Vwcy8gCndlYmJhY2t1cC50YXIgCnNlbnQgMTMuMjRNIGJ5dGVzIAlyZWNlaXZlZCAzNiBieXRlcyAJNC4wMk0gYnl0ZXMvc2VjIAp0b3RhbCBzaXplIGlzIDI0LjM2TSAJc3BlZWR1cCBpcyAxLjM0IAokIGxsIHdlYmJhY2t1cC50YXIgCmxzOiB3ZWJiYWNrdXAudGFyOiBObyBzdWNoIGZpbGUgb3IgZGlyZWN0b3J5"” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
Files include and exclude in rsync
The rsync command gives the option to include and exclude files or directories while doing synchronization.
In the below example, we include only files or directories starting with ‘S’ using “- -include” option and exclude all other files using “- -exclude” option.
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyDigJNhdnog4oCTIC1pbmNsdWRlIOKAmFMq4oCZIOKAkyAtZXhjbHVkZSDigJgq4oCZIGxpbnV4Y29uY2VwdEAxOTIuMTY4LjIuMTA6L2hvbWUvbGludXhjb25jZXB0LyAKUGFzc3dvcmQ6IApyZWNldmluZyBmaWxlIGxpc3QgLiAuIC4gZG9uZSAKLi8gCnNhdGlzaC50eHQgCnNhdGlzaDAxLnR4dCAKU1NILWNvbmZpZyAKc2VudCAyMzQgYnl0ZXMgCXJlY2VpdmVkIDEwNjI1NzgwIGJ5dGVzIAkyMzQ1NjcxOC43MyBieXRlcy9zZWMgCnRvdGFsIHNpemUgaXMgNDIzNTY3MTggCXNwZWVkdXAgaXMgMi40NQ=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"”][/et_pb_dmb_code_snippet]
Set bandwidth while synchronizing
You can use “- -bwlimit” option to set and utilize network bandwidth during file transfer using rsync command.
This option is used to limit the input and output network bandwidth over the network.
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyDigJNhdnog4oCTIC1id2xpbWl0PTEwMCAvaG9tZS9zYXRpc2gvdG1wLyBsaW51eGNvbmNlcHRAMTkyLjE2OC4yLjEwOi9ob21lL2xpbnV4Y29uY2VwdC8KUGFzc3dvcmQ6IApzZW5kaW5nIGluY3JlbWVudGFsIGZpbGUgbGlzdCAKc2VudCAzMTQgYnl0ZXMgCXJlY2VpdmVkIDE0IGJ5dGVzIAk0My4zNSBieXRlcy9zZWMgCnRvdGFsIHNpemUgaXMgMjQuMDNNIAlzcGVlZHVwIGlzIDUyMzQ1Ni4wNQ=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"” 4=”hover_enabled="0"” 5=”sticky_enabled="0"”][/et_pb_dmb_code_snippet]
Synchronize Whole files using
The rsync we use for its best feature is that it synchronizes only the changed block to the destination, instead of sending the whole file.
If you don’t have a network bandwidth issue and having enough CPU to do the process for a long time, you can use the “-W” option for transfer whole files.
[et_pb_dmb_code_snippet 0=”code="JHJzeW5jIOKAk2F2elcgIGxpbnV4Y29uY2VwdEAxOTIuMTY4LjIuMTA6L2hvbWUvbGludXhjb25jZXB0LyAgL2hvbWUvc2F0aXNoL2xpbnV4Y29uY2VwdApQYXNzd29yZDoKcmVjZWl2aW5nIGZpbGUgbGlzdCAuIC4gLiBkb25lCi4vCmFiYy50eHQKaG9tZS5odG1sCmluZGV4Lmh0bWwKc2VudCA0MDMgYnl0ZXMgICAgICAgcmVjZWl2ZWQgMTQ1MjY3MCBieXRlcyAgICAgOC42MiBieXRlcy9zZWMKdG90YWwgc2l6ZSBpcyA0MjU2NzgxMiAgICAgICBzcGVlZHVwIGlzIDIuMzg="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"” 4=”hover_enabled="0"” 5=”sticky_enabled="0"”][/et_pb_dmb_code_snippet]
After running of rsync job, doing an undo can be a tedious job. If anything gets mess up with rsync, it is difficult to move on the previous state.
So, there is an option to run rsync command with “- -dry-run” option where it executes to show what will happen if this command will execute.
Use of this option will not make any changes on source or destination and show the output similar to the actual execution of the command. You can test the command using the “- -dry-run” option and remove this option and execute the command when you satisfy with output.
[et_pb_dmb_code_snippet 0=”code="JCByc3luYyDigJNhdnog4oCTIC1kcnktcnVuIOKAkyAtcmVtb3ZlLXNvdXJjZS1maWxlcyB3ZWJiYWNrdXAudGFyIC9ob21lL2xpbnV4Y29uY2VwdC9iYWNrdXAgCndlYmJhY2t1cC50YXIgCnNlbnQgMzIgYnl0ZXMgCXJlY2VpdmVkIDE2IGJ5dGVzIAkxMDEuMDAgYnl0ZXMvc2VjIAp0b3RhbCBzaXplIGlzIDEyLjEyTSAJc3BlZWQgaXMgMzI0NTY3LjAwIChEUlkgUlVOKQ=="” 1=”copy_button="on"” 2=”_builder_version="4.9.7"” 3=”_module_preset="3a2d4e4b-f2ae-4571-a284-ca584312491f"” 4=”hover_enabled="0"” 5=”sticky_enabled="0"”][/et_pb_dmb_code_snippet]
0 Comments