e-mail   
 Menu
  Home
  Download
  Top 10 Downloads
  Last 15 New Files
  Web Links
  Tips
  Last 15 New Tips
  NLM Programming
  Admins Club





SUPLA System
Internet of Things




Installation and Administration






Polish Forum SUSE


 
Who's Online

 There are currently,
21 guest(s)
that is (are) online.
 


Technical Information

Back to List of Categories

Technical Information about
  A Little YaST Magic
  A Slideshow for Your KDE Wallpaper
  Add Novell Support to Your Firefox Search Engine
  Adding To The Panels In Gnome.
  Akregator - RSS News Reader
  An Introduction to WBEM and OpenWBEM in SUSE LINUX
  Authenticating SUSE LINUX to eDirectory via LDAP
  Basic Mounting Differences between Windows and Linux
  Beginner's Guide to Using Novell SUSE LINUX's Build Utility
  Browsing Files
  Card Readers on SUSE LINUX 9.2 Professional?
  Changing the Default Browser in Gnome
  Developing Perl CGI scripts on SUSE LINUX
  Do I Have Permission?
  Double SYNC -- Using RSYNC Simultaneously With and Without Branch Office for Data Backup
  Dumb Question: Mounting Linux CDs
  Easy Command-Line Introduction
  Easy Way to Debug SSL Connections
  Enabling an Interactive Startup in Novell/SUSE LINUX
  Exploring MySQL 4.1 and PHP 5 on SUSE LINUX
  Extend Your Panels Using Gnome Drawers.
  Firefox Speed Tip
  Fix Java Errors With ConsoleOne On Linux
  Fix Your Mouse Wheel...
  Gimp -- Closing In on Photoshop...
  Gimp Tips Part 1: Resizing Images
  Gimp Tips Part 2: Rounding Corners
  Gimp Tips Part 3: Fading Photography Edges
  Gimp Tips Part 4: Selective Colorization
  Gimp Tips Part 5: Plugins a Plenty - Adding Light
  Gimp Tips Part 6: Using The Clone Tool
  Google Auto-fill Replica for Linux
  How a Corrupted USB Drive Was Saved by GNU/Linux
  How to Implement Login Scripts into a Pure Linux Environment
  How to install and connect an External Modem on a Linux PC
  How to Install the Novell Client for Linux on SUSE Linux 10.0
  How to launch applications at login in Gnome
  How to Remote Control Linux
  How to setup a Linux application to automatically startup when you login.
  How to share Files and Printers Between Linux and MS Windows with Samba
  How to uninstall the Novell Client for Linux
  How to upgrade the Novell Client for Linux
  How to use SMB without turning off the firewall
  How's the Weather?
  HTML Development Tools in Linux
  Informative Bash Prompt
  Install and Use QEMU on SUSE 9.2
  Install Linux Frequently, Without the Hassle
  Installing Apache, PHP, and MySQL on SUSE LINUX Professional
  Installing RPM Files
  Integrating Novell Linux Desktop into a Novell NetWare network
  Introduction to Oracle 10g R1 (10.1.0.3) on SUSE LINUX Professional 9.3
  iPrint Printers on Novell Linux Desktop
  Linux Scheduling Priorities Explained
  Make a Misbehaving Program Quit
  Manually Mounting a USB Flash Drive in Linux
  Migrating To Linux, Advantages Over Windows 9.x And XP
  Minimize Your Maximizing with Multiple Desktops
  Mounting NetWare Volumes On NLD Illustrated
  Move Your /home to Another Computer
  Mozilla Mail Review
  Need a Backup?
  Novell Client Settings with Windows Terminal Server
  Novell Linux Desktop Equivalents of Windows Software - New and Improved
  Novell Linux Desktop System Upgrade How-To
  Partitioning Hard Drive for Linux Install
  Perform Remote SuSE Installations With Virtual Network Computing
  Quick Intro to Klipper
  Quick Shortcuts in The Gimp
  Reset Your Lost Root Password
  Resizing Your Gnome / KDE Panels
  Runlevels in SUSE LINUX
  Running Existing Win32 Applications on SUSE LINUX
  Service Command in /etc/init.d/
  Setting Shortcut Keys
  Setting up a Linux NFS Install Source for Your LAN
  Should I choose Novell Linux Desktop or SUSE LINUX Professional as my desktop?
  Starting Services From the Command Line On SuSE LINUX
  Sticky Notes for Gnome
  SUSE LINUX 9.3 Professional Review
  Sync Up The Taskbar and the Desktop in Gnome
  System Upgrade How-To
  Taking Screenshots in Linux
  The Basics of Messaging in the Cross-Platform GroupWise Client
  The Novell Client for Linux (beta).
  Transparent Panels in Gnome
  Trick Out Your KDE Desktop
  Understanding the YAST Software Module Options
  Updating Your Software with Red Carpet
  Using dsbk on Linux and Unix
  Using iFolder to Synchronize Firefox Bookmarks
  Using Multiple Swap Partitions In 2.4
  Using openSSH to Securely Access Remote Systems
  What FTP client should I use on Linux?
  What Is Logical Volume Management?
  What is so great about Firefox?
  Which Graphic Formats For What?
  Windows Shortcuts for Firefox Tabs on Linux
  Windows to Linux: A Beginner's Guide
  Windows Users Can Feel at Home Exploring Files in Linux
  WLAN interface as the Default
  Work With User Selection Icons In KDM
  YAST Online Update De-Mystified

Technical Information
 Double SYNC -- Using RSYNC Simultaneously With and Without Branch Office for Data Backup

Printer-friendly version

Posted: 22 Feb 2005

I have received a number of queries from my previous posts about RSYNC, so I thought it might be helpful to make a new post.

Some people might already be using (or considering) Nterprise Branch Office, in order to realize the savings on hardware and administrative costs. But they are also using full-fledged NetWare servers where the Branch Office appliance is not appropriate, and may want to leverage the same benefit of using RSYNC to replicate files from the full servers, at the same time. Can RSYNC do both?

You bet it can. Why would we want this? Nterprise Branch Office allows us to replicate data (using RSYNC) over public Internet connections using SSL to encrypt the data during transit, so there is a need to run the RSYNC daemon on the Central Office server using SSL. However, other NetWare servers (perhaps on the local LAN or over private WAN links) do not require the overhead of SSL encryption, and are better replicated using an RSYNC daemon running without SSL. This could be accomplished by running two separate RSYNC servers at the central office, or --better-- by running two instances of RSYNC on one Central Office server. Or --best-- we can run the two (SSL and non-SSL) instances of RSYNC on a NetWare cluster, to allow for server failover.

Let's examine a scenario where a central office NetWare cluster receives data replication from both Branch Office appliances and regular NetWare servers. Essentially, the central office cluster runs two instances of RSYNC, one using SSL for the NBO servers, and one without SSL for NetWare servers on the LAN. The data is copied daily using RSYNC from remote servers to a central office server, which is then centrally backed up to tape.

My Central Office (2-node) cluster servers run NetWare 6.5 with the RSYNC component installed. To provide for server failover, the RSYNC service is configured as a cluster resource, which can load on either physical server. Since I'm running the service twice (one for SSL, and one without), I have configured two separate cluster volumes, RSYNC1 and RSYNC2, as the cluster resources, so that they can independently failover to either server node.

Creating Clustered Instances of the RSYNC Daemon

I simply create the two cluster volumes using NSSMU, or iManager, and assign them their respective IP addresses and preferred node (they can start on the same server for an active-standby cluster, or on separate servers for an active-active cluster). I copy the RSYNC software to both volumes (RSYNC1 and RSYNC2) in RSYNC, and modify the cluster resource load and unload scripts to launch the RSYNC daemon for each instance:

Example Load Script for the SSL Instance of RSYNC:

nss /poolactivate=RSYNC1 
mount RSYNC1 VOLID=252
CLUSTER CVSBIND ADD RSYNC1_SERVER 10.10.10.207
NUDP ADD RSYNC1_SERVER 10.10.10.207
add secondary ipaddress 10.10.10.207

RSYNC1:/rsync/rsync --progress --address=10.10.10.207 --port=873 --ssl 
--daemon --config=RSYNC1:/rsync/rsyncd.conf

Example Load Script for the non-SSL Instance of RSYNC:

nss /poolactivate=RSYNC2 
mount RSYNC2 VOLID=251
CLUSTER CVSBIND ADD RSYNC2_SERVER 10.10.10.208
NUDP ADD RSYNC2_SERVER 10.10.10.208
add secondary ipaddress 10.10.10.208

RSYNC2:/rsync/rsync --progress --address=10.10.10.208 --port=873 --daemon
--config=RSYNC2:/rsync/rsyncd.conf

NOTE: Normally the rsyncd.conf file lives in the SYS:ETC folder, but to allow for multiple instances, as well as failover with the cluster volume, we specify with the "--config=" parameter to use the rsyncd.conf file in the RSYNC folder in the cluster volumes (RSYNC1 and RSYNC2).

The unload scripts similarly add a statement to shut down the particular RSYNC service, as in these examples:

RSYNC1:rsync/rsyncdn Rsync10.10.10.207:873

del secondary ipaddress 10.10.10.207
CLUSTER CVSBIND DEL RSYNC1_SERVER 10.10.10.207
NUDP DEL RSYNC1_SERVER 10.10.10.207
nss /pooldeactivate=RSYNC1 /overridetype=question
RSYNC2:rsync/rsyncdn Rsync10.10.10.208:873

del secondary ipaddress 10.10.10.208
CLUSTER CVSBIND DEL RSYNC2_SERVER 10.10.10.208
NUDP DEL RSYNC2_SERVER 10.10.10.208
nss /pooldeactivate=RSYNC2 /overridetype=question

NOTE: Use RSYNCDN as in the above examples, rather than "unload RSYNC" since this would kill both instances of the RSYNC daemon, if they were running on the same physical server. Also note that there is no space before the IP address in the RSYNCDN command.

Since the SSL instance of RSYNC is running on the cluster resource associated with the RSYNC1 volume, I can add a line in the HOSTS file of each Branch Office server to resolve RSYNC1 to that IP address (10.10.10.207), or point the replication service directly to that IP address. Likewise, since the non-SSL instance of RSYNC is running on the cluster resource associated with the RSYNC2 volume, I can edit the HOSTS file of each full NetWare server to resolve RSYNC2 to the non-SSL resource IP address (10.10.10.208).

Performing Data Replication from a Normal NetWare Server

Thus on a non-SSL NetWare server, my RSYNC command might look like this:

rsync -rav --volume=DATA: /USERS RSYNC2::SITE1 --delete --exclude-from=
SYS:rsync/excludes.txt

The -rav means -r for recursive, -a for archive, -v for verbose. The source string is "--volume=DATA: users/" (note the trailing "/"). The destination string is "RSYNC1::SITE1" (the server name can be resolved via the HOSTS file, and the SITE1 corresponds to the section specified in the rsyncd.conf file on the RSYNC server). The double-colon "::" specifies that rsync communication will occur via TCP, default port 873. The -- delete switch will delete files on the target that have been deleted on the source (use carefully). The --exclude-from switch will exclude copying files that are listed in the specified text file (like, *.mp3 - see below).

See the rsync man pages for a full list of rsync options (available at http://samba.anu.edu.au/ftp/rsync/rsync.html.

The sending server then resolves RSYNC2 as the receiving host from its own hosts file, and then resolves SITE1 as the file destination from the rsyncd.conf file on the receiving server (the server running the daemon). Remember, each instance of RSYNC running on the server will have its own rsyncd.conf file. That's why it is placed on the corresponding cluster volume, rather than in the SYS:ETC location. For example:

The Central Office Server RSYNC Daemon Configuration File

My RSYNC2 cluster resource has this section in its rsyncd.conf file, which allows the command above to work:

[SITE1]
   path = RSYNC2:/SITE1/USERS
   comment = USERS backup area for SITE1 server
   read only = no
   use chroot = no 
   timeout = 3600
   transfer logging = yes
   use lfs=yes
   hosts allow=10.1.1.203
   hosts deny=*

NOTE: The file location /SITE1/USERS on the volume RSYNC2 must already exist for the previous RSYNC command to work. Be sure to create it ahead of time.

Preserving NetWare File Trustee Information

Since rsync does not save the NetWare attributes on the files, I run this trustusers.ncf one minute before the rsync:

TRUSTEE SAVE DATA:USERS DATA:USERS	rustees.txt

Then the trustees.txt file gets synched with the data. Thus if I have to restore, I can also later restore all the NetWare permissions. TRUSTEE.nlm can be downloaded at http://support.novell.com/servlet/downloadfile?file=/uns/ftf/trust110e.exe.

Restoring Data Using RSYNC

In the event that I want to perform a full data restore of the entire USERS directory, I can pull data back from the RSYNC server to the source server with a restusrs.ncf file:

rsync -rav RSYNC2::SITE1 --volume=DATA: users --update

This will restore any files deleted since the last backup, by reversing the source and target strings. The --update switch will ensure that any files modified since the last backup will not be overwritten.

This is a "brute force" restore, I realize. If used, it would also restore deleted files in subdirectories that weren't meant to be restored.

If I need to perform a more selective restore of data, say, down to the directory level, I can execute the following command on the REMOTE server:

rsync -rav RSYNC2::SITE1/admin/foo/bar/ --volume=DATA: users/admin/foo/bar
--update

This will restore all files in the data:/users/admin/foo/bar directory, while preserving any files that were updated since the last backup (rsync). The trailing "/" in the source string is important -- meaning all files/folders in that directory, rather than the directory itself.

Replication from the Nterprise Branch Office Server

When I am replicating data from a Branch Office Appliance, the configuration is done using the Nterprise Branch Office Web Management portal, as is normal for this product. I merely enable replication, point the Central Office Server IP address to the SSL-enabled instance of RSYNC running on the NetWare cluster, and import the SSL certificate that I exported from the Central Office server. See "Setting Up Data Replication" on the Novell Documentation site for Nterprise Branch Office.

Excluding Certain Files Types in the Data Replication

Of course, we don't necessarily want to replicate all file types. Some users (perish the thought) may download and save undesirable file types to their home directories, which we don't want to copy to the Central Office server. Thus the sending servers will have the parameter "--exclude-from=SYS:rsync/excludes.txt" in their RSYNC command, which specifies the file types that we wish to exclude from replication.

Example: My SITE1 server has this excludes.txt file to exclude certain
file types from the transfer:

#EXCLUDES.TXT
*.mp3
*.wma
*.rma
*.mov
*.vid
*.avi

Thus we see that RSYNC works happily for both SSL and non-SSL purposes, replicating for Nterprise Branch Office, or for regular NetWare servers. It can operate in both modes on the same Central Office server, or --best-- on a NetWare cluster to provide failover and load balancing. For your data replication needs, RSYNC does it all.

More Information on RSYNC

For more tips on how to use RSYNC instead of tape backup for remote site servers, see: "Using RSYNC in Data Backup Solution"

For ideas on how to use RSYNC to distribute service packs or standard corporate documents, see: Distribute Service Packs and Corporate Documents using RSYNC: "Distribute Service Packs and Corporate Documents using RSYNC"

For an example of how to replicate data from a Windows XP workstation to a NetWare server using RSYNC, see: "Backup Windows XP Computers using RSYNC and NetWare 6.5"






Since 2003

Portal posiada akceptację firmy Novell Polska
Wszystkie materiały dotyczące produktów firmy Novell umieszczono za zgodą Novell Polska
Portal has been accepted by the Novell Polska
All materials concerning products of Novell firm are placed with Novell Polska consent.
NetWare is a registered trademark of Novell Inc. in the United States and other countries.
Windows is a trademark or a registered trademark of Microsoft Corporation in the United States and other countries.
Sybase is a registered trademark of Sybase Inc. in the United States of America.
Other company and product names are trademarks or registered trademarks of their respective owners.