How to sync the data between two servers without using SAN replication ? Do you have better than RSYNC tool for this job ? I don’t think so, you we will not get better than RSYNC. It uses ‘rsync algorithm’ which provides a very fast method for syncing the directories or filesystems. An important feature of rsync is that the mirroring takes place with only one transmission in each direction and which is not available in other similar programs.
Rsync’s default port is 873 and its an opensource software. Rsync is available for Unix,Linux and windows operating systems. You can freely download rsync source code from rsync.samba web portal.
Here we will see how to sync data between two servers using automated script.
Operating system:Red Hat Linux
Source Server IP: 192.168.10.20 (mylinz1)
Source Server Path:/db/oracle/
Source Server Path:/db/oracle/
Destination Server IP:192.168.10.25 (mylinz2)
Destination Server Path:/db/oracle-bck/
Destination Server Path:/db/oracle-bck/
Before proceeding to rsync part,you need to configure key-less authentication to ensure each can communicate using that.
Configuring key-less authentication
1.Verify whether your host will allow to perform RSA key-less authentication.If you didn’t get similar output,then you need to comment out the lines in sshd_config.
[root@mylinz1 ~]# cat /etc/ssh/sshd_config |egrep "RSA|Pubkey|Authorized" |grep -v "#" RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys [root@mylinz1 ~]# [root@mylinz2 ~]# cat /etc/ssh/sshd_config |egrep "RSA|Pubkey|Authorized" |grep -v "#" RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys
2.Generate the keygen if you didn’t have one already. Here the user is “root”.
[root@mylinz1 ~]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 94:e6:6f:66:bb:cd:a8:30:4d:90:94:31:ae:64:f6:5e root@mylinz1 The key's randomart image is: +--[ RSA 2048]----+ | +o | | o.o . | | + + + | | + o = | | . . E | | . + . | | + . = | | o + = | | ..+.o | +-----------------+ [root@mylinz1 ~]# cd .ssh/ [root@mylinz1 .ssh]# ls -lrt total 8 -rw-r--r--. 1 root root 394 Jun 19 00:43 id_rsa.pub -rw-------. 1 root root 1671 Jun 19 00:43 id_rsa [root@mylinz1 .ssh]# [root@mylinz2 ~]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: e7:a8:19:ac:dd:e3:28:b3:42:00:a0:84:4a:10:4e:fe root@mylinz2 The key's randomart image is: +--[ RSA 2048]----+ |B+ | |O. | |=o | |o . | | . E S . | | . . + | | . o . . | | . oo *. | | .o+=.o. | +-----------------+ [root@mylinz2 ~]# cd .ssh/ [root@mylinz2 .ssh]# ls -lrt total 12 -rw-r--r--. 1 root root 395 Jun 19 00:17 known_hosts -rw-------. 1 root root 1675 Jun 19 00:44 id_rsa -rw-r--r--. 1 root root 394 Jun 19 00:44 id_rsa.pub
3.Share the “id_rsc.pub” file across the servers to enable the ssh key-less authentication.
[root@mylinz1 .ssh]# scp -r id_rsa.pub 192.168.10.25:/root/.ssh/authorized_keys The authenticity of host '192.168.10.25 (192.168.10.25)' can't be established. RSA key fingerprint is 5a:56:fd:69:cf:f2:b8:78:b9:67:e8:d0:f2:a4:ef:cb. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.10.25' (RSA) to the list of known hosts. root@192.168.10.25's password: id_rsa.pub 100% 394 0.4KB/s 00:00 [root@mylinz1 .ssh]# [root@mylinz2 .ssh]# scp -r id_rsa.pub 192.168.10.20:/root/.ssh/authorized_keys root@192.168.10.20's password: id_rsa.pub 100% 394 0.4KB/s 00:00 [root@mylinz2 .ssh]#
4.Verify your work.
[root@mylinz2 ~]# ssh 192.168.10.20 Last login: Wed Jun 19 00:42:27 2013 from 192.168.10.101 [root@mylinz1 ~]# ssh 192.168.10.25 Last login: Tue Jun 18 23:59:19 2013 from 192.168.10.101 [root@mylinz2 ~]#
Let’s move to RSYNC part.
RSYNC SCRIPT
Here is the rsync script which will be used for syncing the data between servers mylinz1 & mylinz2.
Here I am running script from mylinz1 to sync the data.
[root@mylinz1 ~]# cat rsync_oracle.sh #!/bin/bash # RSYNC SCRIPT TO SYNC TWO SERVER'S SPECIFIC DIRECTORIES # Website:www.UnixArena.com SOURCE_PATH='/db/oracle/' SOURCE_SERVER='192.168.10.20' #Added for reference DESTINATION_PATH='/db/oracle-bck/' DESTINATION_HOST='192.168.10.25' DESTINATION_USER='root' LOGFILE='rsync_oralce.log' echo $'\n\n' >> $LOGFILE rsync -av --rsh=ssh $SOURCE_PATH $DESTINATION_USER@$DESTINATION_HOST:$DESTINATION_PATH 2>&1 >> $LOGFILE echo "Sync Completed at:`/bin/date`" >> $LOGFILE [root@mylinz1 ~]#pwd /root [root@mylinz1 ~]#chmod 700 rsync_oracle.sh
This script creates log as well with newly synchronized files information.
Testing rsync script:
1.Run the script manually.
[root@mylinz1 ~]#./rsync_oracle.sh
2.Verify the log file.
[root@mylinz1 ~]# tail -1rsync_oralce.log kshrc latrace.conf ld.so.cache ld.so.conf libaudit.conf libuser.conf sent 160321 bytes received 1155 bytes 107650.67 bytes/sec total size is 156728 speedup is 0.97 Completed at:Wed Jun 19 01:02:25 IST 2013 [root@mylinz1 oracle]#
3.Create a new file to sync the data to mylinz2 server.
[root@mylinz1 oracle]#cd /db/oracle/ [root@mylinz1 oracle]# touch verify_rsync [root@mylinz1 oracle]# ls -lrt verify_rsync -rw-r--r--. 1 root root 0 Jun 19 01:04 verify_rsync [root@mylinz1 oracle]# cd /root [root@mylinz1 ~]# ./rsync_oracle.sh [root@mylinz1 ~]# [root@mylinz1 ~]# tail -10 rsync_oralce.log sending incremental file list ./ verify_rsync sent 1093 bytes received 34 bytes 2254.00 bytes/sec total size is 156728 speedup is 139.07 Completed at:Wed Jun 19 01:05:03 IST 2013 [root@mylinz1 ~]#
4.Verify the whether the newly created file synced in mylinz2.
[root@mylinz2 ~]# cd /db/oracle-bck/ [root@mylinz2 oracle-bck]# ls -lrt verify_rsync -rw-r--r--. 1 root root 0 Jun 19 01:04 verify_rsync [root@mylinz2 oracle-bck]#
That’s it. Our rsync script is working fine.
Automating sync
If you want to sync the data between two servers automatically on preferred time interval,you can add the script in to crontab.
Add the below line in root’s crontab to sync the data for every 5 minutes.
Add the below line in root’s crontab to sync the data for every 5 minutes.
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /root/rsync.sh 2>&1 > /dev/null
Thank you for reading this article.Please leave a comment if you have any doubt.