Data Pool Rsync Administration
General
This page is under construction.
- [#install Software installation]
[wiki:PACS/Pools PACS Pools] Documentation
[wiki:PACS/Rsync Data Pool Rsync Client] Documentation
Overview
The rsync daemon has access to one or more sets of data pools. Each set of pools is associated with an rsync module name. For example, you could have a set of pools associated with the module name PV and another set with the module name IST. You can also use module names with subsets of larger pools. For example, if you have a root directory for the pools created during each week of PV, you could have module names PV1, PV2, and so on. The generic PV module could then be associated with the directory above all the weekly pool directories. The installation would look something like this:
pools | |-IST | |-PV | |-PV1 | |-PV2 | |-PV...
When a client makes a request of the daemon, it uses a module name, so it could request all PV pools, or just those for a specific week, for example, requesting module PV2.
As pools are created and updated, clients continually make requests of the daemon. These requests make a mirror of the ICC module at the client site.
Security
The daemon allows an authorized client to get lists of items and to copy directories, files and symbolic links. It does not allow the client to write into the module area.
Clients are authorized in two ways:
The client must supply a password to the daemon. This password is associated with a user name and password in the daemon's rsyncd.secrets file. Authorized users can be allowed access to all modules or just to some.
The machines and subnets that can connect to the daemon are listed in the rsyncd.config file by IP address or domain name. As with user names, you can limit module access to only selected machines or subnets.
Currently, traffic between the daemon and a client is not encrypted; but that can be added.
Software Installation
The [http://samba.anu.edu.au/rsync/documentation.html rsync documentation] describes how to set-up an rsync daemon. In this section, we give a basic example. Our example does not run as root, so we explicitly start it and give it it's own TCP port number.
Use a machine that has rynsc installed, or install it yourself. Check for installation with the command
> rsync --version rsync version 2.6.3 protocol version 28 Copyright (C) 1996-2004 by Andrew Tridgell and others <http://rsync.samba.org/> Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, inplace, IPv6, 64-bit system inums, 64-bit internal inums }} Once you have the software installed, you'll need to create some configuration files: *'''rsyncd''' The script used to start the daemon manually. *'''rsyncd.config''' The configuration file read by the rsync daemon. *'''rsyncd.secrets''' The file containing the list of authorized users. When you run the daemon, it will create some additional files: *'''rsyncd.lock''' This supports the ''maximum connections'' option specified in the {{{rsyncd.config
file.
rsyncd.log Errors and authorization failures are written to this file.
rsyncd.pid This file contains the daemons process ID.
Daemon Startup
Here's a simple shell script to start the rsync daemon as a non-root process. The command takes three parameters:
- Run rsync as a daemon, or server, process.
- Listen on TCP port 44520.
Read the daemon's configuration from file /home/poolman/rsync/rsyncd.config
Start the process in the background and then wait 5 seconds. After that, check that the server is indeed running.
The script's file protection is set such that the owner can execute the file.
rsyncd
# File rsyncd echo "Starting rsyncd `date`" rsync --daemon --port=44520 --config=/home/poolman/rsync/rsyncd.config & sleep 5 ps -elf | grep 'rsync --daemon' | grep -v grep
The Configuration File
Here's an example configuration file.
# File: configd.rsync # rsync daemon command # rsync --daemon --port=44520 --config=/home/poolman/rsync/configd.rsync # rsync client # rsync -avz --password-file=/local/home/versant/rsync/password --port=445200 nhsc@pacs1.mpe.mpg.de::pools/ /local/home/versant/PacsProductPools secrets file = /home/poolman/rsync/rsyncd.secrets # If you want something to appear when someone logs on, put it in the # rsyncd.motd file #motd file = /home/poolman/rsync/rsyncd.motd lock file = /home/dppool/rsync/rsyncd.lock pid file = /home/dppool/rsync/rsyncd.pid #use chroot = yes read only = yes list = yes uid = poolman gid = users max connections = 2 log file = /home/dppool/rsync/rsyncd.log #log format = %t %a %m %f %b timeout = 300 auth users = mpe, nhsc, leuven hosts allow = herschel6.kluvent.be hcss1.ipac.caltech.edu 139.8.289/8 hosts deny = * [pv] comment = All PV obsid pools path = /pools/pv [pv1] comment = Week 1 pools path = /pools/pv/pv1 [pv2] comment = Week 2 pools path = /pools/pv/pv2 [pvcal] comment = Calibration files contributed by PACS calibration scientists. path = /cal/pv/special
Security
The user names and passwords of agents allowed to access the server are kept in the rsyncd.secrets file. The file protections are set so only the owner can read and write the file. rsyncd.secrets
mpe:password1 leuven:password2 nhsc:password3