= Data Pool Rsync Administration = = General = This page is under construction. * [[#install|Software installation]] * [[PACS/Pools|PACS Pools]] Documentation * [[PACS/Rsync|Data Pool Rsync Client]] Documentation = Overview = The rsync daemon has access to one or more sets of data pools. Each set of pools is associated with an rsync module name. For example, you could have a set of pools associated with the module name ''PV'' and another set with the module name ''IST''. You can also use module names with subsets of larger pools. For example, if you have a root directory for the pools created during each week of PV, you could have module names ''PV1'', ''PV2'', and so on. The generic ''PV'' module could then be associated with the directory above all the weekly pool directories. The installation would look something like this: {{{ pools | |-IST | |-PV | |-PV1 | |-PV2 | |-PV... }}} You'll find an example showing how module names and directories are related in the [[#configuration|configuration file]]. When a client makes a request of the daemon, it uses a module name, so it could request all ''PV'' pools, or just those for a specific week, for example, requesting module ''PV2''. As pools are created and updated, clients continually make requests of the daemon. These requests make a mirror of the ICC module at the client site. == Security == The daemon allows an authorized client to get lists of items and to copy directories, files and symbolic links. It does not allow the client to write into the module area. Clients are authorized in two ways: 1. The client must supply a password to the daemon. This password is associated with a user name and password in the daemon's {{{rsyncd.secrets}}} file. Authorized users can be allowed access to all modules or just to some. 1. The machines and subnets that can connect to the daemon are listed in the {{{rsyncd.config}}} file by IP address or domain name. As with user names, you can limit module access to only selected machines or subnets. Currently, traffic between the daemon and a client is not encrypted; but that can be added. <> = Software Installation = The [[http://samba.anu.edu.au/rsync/documentation.html|rsync documentation]] describes how to set-up an rsync daemon. In this section, we give a basic example. Our example does not run as root, so we explicitly start it and give it it's own TCP port number. Use a machine that has rynsc installed, or install it yourself. Check for installation with the command {{{ > rsync --version rsync version 2.6.3 protocol version 28 Copyright (C) 1996-2004 by Andrew Tridgell and others Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, inplace, IPv6, 64-bit system inums, 64-bit internal inums }}} Once you have the software installed, you'll need to create some configuration files: *'''rsyncd''' The script used to start the daemon manually. *'''rsyncd.config''' The configuration file read by the rsync daemon. *'''rsyncd.secrets''' The file containing the list of authorized users. When you run the daemon, it will create some additional files: *'''rsyncd.lock''' This supports the ''maximum connections'' option specified in the {{{rsyncd.config}}} file. *'''rsyncd.log''' Errors and authorization failures are written to this file. *'''rsyncd.pid''' This file contains the daemons process ID. == Daemon Startup == Here's a simple shell script to start the rsync daemon as a non-root process. The command takes three parameters: 1. Run rsync as a daemon, or server, process. 1. Listen on TCP port 44520. 1. Read the daemon's configuration from file {{{/home/poolman/rsync/rsyncd.config}}} Start the process in the background and then wait 5 seconds. After that, check that the server is indeed running. The script's file protection is set such that the owner can execute the file. File name: '''rsyncd''' {{{ #!/bin/sh # File rsyncd echo "Starting rsyncd `date`" rsync --daemon --port=44520 --config=/home/poolman/rsync/rsyncd.config & sleep 5 ps -elf | grep 'rsync --daemon' | grep -v grep }}} <> == The Configuration File == The rsync daemon's cofiguration commands are described in [[http://samba.anu.edu.au/ftp/rsync/rsyncd.conf.html|rsyncd.conf]]. Here's an example configuration file. In this configuration, most options are global. Module names are give in square brackets. Indented items below the module name, including the root directory for the module, are given below the name. File name: '''rsyncd.config''' {{{ # File: rsyncd.config # rsync daemon command # rsync --daemon --port=44520 --config=/home/poolman/rsync/configd.rsync secrets file = /home/poolman/rsync/rsyncd.secrets # If you want something to appear when someone logs on, put it in the # rsyncd.motd file #motd file = /home/poolman/rsync/rsyncd.motd lock file = /home/dppool/rsync/rsyncd.lock pid file = /home/dppool/rsync/rsyncd.pid # Use if run by root #use chroot = yes read only = yes list = yes uid = poolman gid = users max connections = 16 # Could use syslogs if running as root log file = /home/dppool/rsync/rsyncd.log #log format = %t %a %m %f %b timeout = 300 auth users = mpe, nhsc, leuven saclay marseille liege wien hosts allow = herschel6.kluvent.be hcss1.ipac.caltech.edu 139.8.289.0/8 165.3.183/8 194.43.294.0/8 122.9.4.0/8 173.50.199.0/8 hosts deny = * [PV] comment = All PV obsid pools path = /pools/pv [PV1] comment = Week 1 pools path = /pools/pv/pv1 [PV2] comment = Week 2 pools path = /pools/pv/pv2 [PVCAL] comment = Calibration files contributed by PACS calibration scientists. path = /cal/pv/special }}} == Security == The user names and passwords of agents allowed to access the server are kept in the rsyncd.secrets file. The file protections are set so only the owner can read and write the file. File name: '''rsyncd.secrets''' {{{ mpe:password1 leuven:password2 nhsc:password3 saclay:password4 marseille:password5 liege:password6 wien:password7 }}}