Data Pool Rsync Administration

General

This page is under construction.

Overview

The rsync daemon has access to one or more sets of data pools. Each set of pools is associated with an rsync module name. For example, you could have a set of pools associated with the module name PV and another set with the module name IST. You can also use module names with subsets of larger pools. For example, if you have a root directory for the pools created during each week of PV, you could have module names PV1, PV2, and so on. The generic PV module could then be associated with the directory above all the weekly pool directories. The installation would look something like this:

pools
  |
  |-IST
  |
  |-PV
  |
  |-PV1
  |
  |-PV2
  |
  |-PV...

When a client makes a request of the daemon, it uses a module name, so it could request all PV pools, or just those for a specific week, for example, requesting module PV2.

As pools are created and updated, clients continually make requests of the daemon. These requests make a mirror of the ICC module at the client site.

Security

The daemon allows an authorized client to get lists of items and to copy directories, files and symbolic links. It does not allow the client to write into the module area.

Clients are authorized in two ways:

  1. The client must supply a password to the daemon. This password is associated with a user name and password in the daemon's rsyncd.secrets file. Authorized users can be allowed access to all modules or just to some.

  2. The machines and subnets that can connect to the daemon are listed in the rsyncd.config file by IP address or domain name. As with user names, you can limit module access to only selected machines or subnets.

Currently, traffic between the daemon and a client is not encrypted; but that can be added.

Anchor(install)

Software Installation

The [http://samba.anu.edu.au/rsync/documentation.html rsync documentation] describes how to set-up an rsync daemon. In this section, we give a basic example. Our example does not run as root, so we explicitly start it and give it it's own TCP port number.

Use a machine that has rynsc installed, or install it yourself. Check for installation with the command

> rsync --version
rsync  version 2.6.3  protocol version 28
Copyright (C) 1996-2004 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, 
              inplace, IPv6, 64-bit system inums, 64-bit internal inums
}}
Once you have the software installed, you'll need to create some configuration files:
 *'''rsyncd''' The script used to start the daemon manually.
 *'''rsyncd.config''' The configuration file read by the rsync daemon.
 *'''rsyncd.secrets''' The file containing the list of authorized users.

When you run the daemon, it will create some additional files:
 *'''rsyncd.lock''' This supports the ''maximum connections'' option specified in the {{{rsyncd.config

file.

Daemon Startup

Here's a simple shell script to start the rsync daemon as a non-root process. The command takes three parameters:

  1. Run rsync as a daemon, or server, process.
  2. Listen on TCP port 44520.
  3. Read the daemon's configuration from file /home/poolman/rsync/rsyncd.config

Start the process in the background and then wait 5 seconds. After that, check that the server is indeed running.

The script's file protection is set such that the owner can execute the file.

rsyncd

# File rsyncd

echo "Starting rsyncd `date`"
rsync --daemon --port=44520 --config=/home/poolman/rsync/rsyncd.config &
sleep 5
ps -elf | grep 'rsync --daemon' | grep -v grep

The Configuration File

Here's an example configuration file.

# File: configd.rsync

# rsync daemon command
# rsync --daemon --port=44520 --config=/home/poolman/rsync/configd.rsync

# rsync client
# rsync -avz --password-file=/local/home/versant/rsync/password --port=445200 nhsc@pacs1.mpe.mpg.de::pools/ /local/home/versant/PacsProductPools

secrets file = /home/poolman/rsync/rsyncd.secrets
# If you want something to appear when someone logs on, put it in the
# rsyncd.motd file
#motd file = /home/poolman/rsync/rsyncd.motd
lock file = /home/dppool/rsync/rsyncd.lock
pid file = /home/dppool/rsync/rsyncd.pid
#use chroot = yes
read only = yes
list = yes
uid = poolman
gid = users
max connections = 2
log file = /home/dppool/rsync/rsyncd.log
#log format = %t %a %m %f %b
timeout = 300
auth users = mpe, nhsc, leuven
hosts allow = herschel6.kluvent.be hcss1.ipac.caltech.edu 139.8.289/8
hosts deny = * 

[pv]
   comment = All PV obsid pools
   path = /pools/pv

[pv1]
   comment = Week 1 pools
   path = /pools/pv/pv1
[pv2]
   comment = Week 2 pools
   path = /pools/pv/pv2

[pvcal]
   comment = Calibration files contributed by PACS calibration scientists.
   path = /cal/pv/special

Security

The user names and passwords of agents allowed to access the server are kept in the rsyncd.secrets file. The file protections are set so only the owner can read and write the file. rsyncd.secrets

mpe:password1
leuven:password2
nhsc:password3