Differences between revisions 71 and 72
Revision 71 as of 2007-10-29 14:14:08
Size: 15835
Editor: Ost
Comment:
Revision 72 as of 2007-11-07 10:26:36
Size: 15857
Editor: Ost
Comment:
Deletions are marked like this. Additions are marked like this.
Line 53: Line 53:
         * schema 32

TableOfContents(2)

PACS Databases

a. Databases @ MPE 12000 - 12699

  • 12000 - 12250 : public databases for operations <br>

    • 12001 : pacseqmimt1e : PACS1 - database copy from Astrium EQM tests end 2005
      • serious problems with schema evolutions etc.
      • schema 26
    • 12004 : sftcode : PACS1 - public CUS development database
      • schema 32
    • 12006 : aotcode : PACS1 - AOT reference database 20060426 / for warm AOT tests
      • schema 32
    • 12003 : leuven : PACS1 - schema 26
    • 1200_ : aottest : PACS1 - Simulated AOT data (DC) about thre days of operation
      • schema 32
    • 12008 : aottest3 : PACS1 - Simulated AOT data (DC) about thre days of operation
      • schema 26
    • 12011 : pacs_fm_ilt_1 : PACS5 (64Bit) - Operational Database for FM ILT
      • active 10. October 2006 all Warm Fuctional tests until 27 October 2006
      • schema 32
    • 12012 : pacs_fm_ilt_1_prop : PACS1 - Propagated Database pacs_fm_ilt_1 from PACS5
      • schema 26
    • 12013 : pacseqmilt : PACS1 - Test-DataBase

      • schema 26
    • 12014 : pacs_fm_ilt_2 : PACS5 (64Bit) - Operational Database for FM ILT
      • active 10. October 2006 all Warm Fuctional tests until 27 October 2006
      • schema 32
    • 12015 : pacs_fm_ilt_2_prop : PACS1 - Propagated Database pacs_fm_ilt_1 from PACS5
      • schema 26
    • 12022 : pacs_fm_ilt_3 : PACS5 (64Bit) - Operational Database for FM ILT
      • active 29.October 2006 all Cold Functional tests until 25.April 2007
      • schema 32
    • 12032 : pacs_fm_ilt_4 : PACS5 (64Bit) - Operational Database for FM ILT
      • active 25.May 2007 until 23.June 2007
      • schema 32
    • 12033 : ist_develop : PACS1 - Development Database for IST
      • schema 32
  • 12251 - 12699 : additional datbases

b. Databases @KUL 12700 - 13399

  • 12700 - 13000 : public databases for operations
  • 13001 - 13399 : additional datbases

c. EQM databases 13400 - 13420

  • 13400 : EQMILT1@Astrium

d. IST databases 13421 - 13430

e. Spare 13500 - 13999

Seting the Database IDs

I have the feeling that a given DB ID cannot be re-used also if the database has been removed. Need clarification...

dbid -r cusdefinitions -> show DBID
dbid -d cusdefinitions -> delete DBID
setdbid 12002 cusdefinitions -> set DBID
dbid -C 12002 cusdefinitions 

Updating Database Engine

  • Log in as dbsa on PACS5.
  • Check Database engine :
    •      oscp -i 
           oscp -l
  • Download the new Database engine version from ESA page :
  • Save license file of previous version (not necessary if new version goes to new dir)
    •       cp  /opt/versant/<versant-previous>/license.xml ~
  • Sometimes you may request new license file (license.xml) from tzaeschk@rssd.esa.int

  • Log in as root on PACS5.
  • Check disk space /usr/local is quite full already.
    •      df -h
  • Change to /usr/local/versant and make a directory to copy and extract the tar file.
    •      cd /opt/versant
           mkdir 7.0.1.3
           cd 7.0.1.3
           tar -zxvf versant_7.0.1.3_RHEL4.0_64bit_release_by_TZ.tar
           ./install.bin
  • Just to be sure check again the system setup and restart the xinetd :
    •      oscp -i
           oscp -l
           /etc/rc.d/xinetd restart
  • Install new license file
    •       cp  license.xml /opt/versant/<versant-new> 
  • Modify psetup scipt on PACS5 : account pcss
    •       emacs /home/pcss/software/psetup
      For example :
          # Can use /etc/.osc... instead of these 4 variables
          setenv VERSANT_ROOT      /usr/local/versant/7.0.1.3
          setenv VERSANT_DB        /dbOper/versant
          setenv VERSANT_DBID      /dbOper/versant
          setenv VERSANT_DBID_NODE pacs5
      
          set path=( /usr/local/versant/7.0.1.3/bin $path )
          set LIB_DIR=/usr/local/versant/7.0.1.3/lib
      
          setenv LD_LIBRARY_PATH /usr/local/versant/7.0.1.3/lib:${LD_LIBRARY_PATH}
      
          setenv THREADS_FLAG native

Schema evolution

  • Download and install PCSS containing the new Schema.
  • Check LD_LIBRARY_PATH (need to point into newest database engine directory
    • non proper settings can lead to unfinished evolution which destroy the database !! e.g. /opt/versant/7.0.1.3
           setenv LD_LIBRARY_PATH /opt/versant/7.0.1.3/lib  
    • !!! The PCSS libjvi7.0.1.so caused problems in the past. You should set it to the proper VERSANT one ONLY !!!
  • Check whether actualVersant engine match the reference platform engine : ftp://ftp.rssd.esa.int/pub/HERSCHEL/csdt/releases/doc/refPlatformVersion

    •        oscp -l
  • Send out Schema evolution warning.
  • Make sure that Databases are not used :
    •      dbtool -trans -info <DBName>
  • Make a backup of the database :
    •      vbackup -dev <DBName>.vbck -backup <DBName>
  • Set Databases to DBAdmin only mode :
    •      dbinfo -d <DBName>
  • Perform schema evolution as described in the Database Administration Manual (HSCDT/TN044) :
    •      schema_evolver  <DBName>@pacs5
      For HCSS 1052/1055 use the hidden -s option :
           schema_tool -e -s <DBName>
      Tilmann :
      • The required usage of the hidden '-s' switch is only required in build 1052, 1055. This switch is somewhat dangerous, because it disables the initial checking done in the schema tool. Please do not use this switch if you can use build 1055 or later.
  • Set the Database back to multi user mode :
    •      dbinfo -m <DBName>
  • Test from an arbitrary account
    • set in .pcss/props the Database to the schema evolved one
    • start cusgui
  • Send message to user.

Traps and Tricks

vbackup trap

Mail from Tilmann :

This bug is known to Versant as bug# 20945. It occurs when using 'vbackup' with the device name of an existing file. In that case, the backup will finish within second, creating a small file between 100KB and 1MB, claiming that the backup was successful. The resulting file cannot be restored to a database, but will cause 'vbackup -restore' to hang.

To avoid the problem, please do not use an existing file name as backup file.

Replication aware Datbase or not

Sometimes you get an error that the Database is Replication aware or not and does not match to your access

You have two options :

  • set the property :
    •       hcss.store.factory = herschel.versant.store.ReplStoreFactoryImpl
  • Remove the replication mechansim from the Database :
    •       db_admin -u <DBName>

Hot Spot error for remote access

When a remote user (like Diego) start cusgui and he/she gets a java hotspot whatever error it might be caused by the user access rights !

A strange error for such a problem. Locally access you get a proper error message.

Do :

  dbuser -add -P <DBName>

Check whether there are open transactions

   dbtool -trans -info <DBName>

Hot Spot and Low Level Connection Error

Sometimes a user starting cus may get an JAVA Hotspot error like this :

comm/CVS> cus -f -import DMC_grat_scan_chop2_fast.txt
Jun 26, 2006 11:16:34 AM herschel.share.log.util.LogInitialiser init
INFO: Using default HCSS log settings
26-Jun-06 11:16:34.543 Configuration: Build number is 931
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0x47cfb3bc, pid=7360, tid=1075185344
#
# Java VM: Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed mode, sharing)
# Problematic frame:
# C  [liboscfe.so+0xa93bc]  se_beginSession+0x677
#
# An error report file with more information is saved as hs_err_pid7360.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
/home/dcesarsk/pcss/lib/hcss/bin/cus: line 3:  7360 Aborted     java -Xmx512M -Dproperty.path="${HCSS_PROPS}" herschel.cus.gui.Cus "$@"

Then you may try on PACS1 the db2tty on that database. In case you get here a different error (Low Level connection error) you may need to restart the xinetd. Don´t ask me why.

/etc/init.d/xinetd restart

If it still not work , try a second time. At least I was successful with it...

Funny enough you may get also this HotSpot error in case the license.xml file is not at the right place !!

Delivery of Databases to ESAC

Make a backup of the Database :

   vbackup -dev <name>_<version>.vbck -backup <name>

and put the backup on PACS1 /home/esac/databases

send an email to John Dowson <John.Dowson@sciops.esa.int> Larry O'Rourke <Laurence.O'Rourke@esa.int>

Offline DataFrame Generation into Database

Assumption: Database is backed up in case something goes wrong!!!!!!

* Log into PACS1 account oper

* type psetup

* Edit .pcss/props : Configure the TM ngestion process (see HCSS TM Ingestion User Manual - HERSCHEL-HCS-DOC-0231

  • hcss.tmingest.database <name of the database>

  • hcss.tmingest.transactions 1000
  • hcss.tmingest.tmpacketprocessor.pacs Y
  • hcss.tmingest.tmpacketprocessor.pacs.name herschel.pacs.spu/PacsTmPacketProcessor)
  • hcss.tmingest.server pacs1
  • hcss.tmingest.port 9875
  • hcss.tmingest.name TelemetryIngestion

  • hcss.tmingest.apidnnn : e.g. hcss.tmingest.apid1156=N (by default all APIDs are requested Y)
  • hcss.tmingest.retry 60000

* Start the EGSE router.

startRouter 9875

* Start dfgen

dfgen

This has the same properties as tmingest.

* Use rtaplay to retrieve telemetry packets from the database and pass them to the router

rtaplay
  • On the source tab select the database (and whether it is local or remote)
  • Also make sure the port number for the EGSE router is correct.
  • On the time tab, select times which cover the telemetry period.
  • On the data tab, ensure packets is selected.
  • On the play tab, you can select the rate to play the telemetry, the period field.
  • Then click play

* Check progress

db2tty -d <database> | grep PacsDataFr

* In case of problems you may connect the TmPacketDumper to the Router and check :

  •    TmPacketDumper -port 9875 -a 1154

    If you get the TmSource packest in a different order then time you may check whether the Database is indexed :

      dbtool -index -info -list <dbname>
    You may index it
      dbtool -space -volume -all <dbname>   # 25% space for it
      dbtool -index -create -btree herschel.versant.ccm.TmSourcePacketImpl _time <dbname> 

Usage of 64 Bit Versant Beta Client in PCSS

You need to get the beta Version of Versant for 64Bit Linux from Tilmann Zaeschke <tzaeschk@rssd.esa.int>. Then your CLASSPATH and LD_LIBRARY_PATH need to point to the right libraries.

Example in csh (installation of Versant 64Bit beta under :/opt/vds7020Blinux64_opt_beta1) :

# 64 Bit Versant test
setenv LD_LIBRARY_PATH /opt/vds7020Blinux64_opt_beta1/lib:${LD_LIBRARY_PATH}

setenv CLASSPATH /opt/vds7020Blinux64_opt_beta1/lib/jvi7.0.2-jdk1.4.jar:${CLASSPATH}

On PACS4 done in : /etc/csh.cshrc.local and /user/pcss/psetup

Additionally we introduced in this file :

setenv VERSANT_SERVICE_PORT 5019
setenv VERSANT_ROOT /opt/vds7020Blinux64_opt_beta1
setenv PATH /opt/vds7020Blinux64_opt_beta1/bin:${PATH}

Usage of 64 Bit Versant Beta Server und Suse

Be careful. In the README of vds7020Blinux64_opt_beta1 the following oscssd file example is given :

service oscssd
{
socket_type = stream
protocol = tcp
wait = no
user = root
server = $VERSANT_ROOT/bin/ss.d
server_args = in.oscssd
disable = no
}

First of all the environment variables are not recognized. But even if you give the full name we get another error.

To solve this you need to use :

service oscssd
{
        socket_type             = stream
        protocol                = tcp
        wait                    = no
        user                    = root
        server                  = /opt/versant/vds7020Blinux64_opt_beta1/bin/ss.d
        disable                 = no
}

Indexing the Database

Make sure that you have enough space in the database left:

dbtool -space -volume -all <dbname>

Free space should be at least 20%.

java -Xmx512M herschel.access.db.CreateIndexes <dbname>

Alternatively, using the versant commands:

dbtool -index -create -btree herschel.versant.ccm.TmSourcePacketImpl _time <dbname>

Expand database volume

   check volume 
   > dbtool -F pacseqmimt1  # VERSANT 6
   > dbtool -space -volume -all pacseqmimt1 # VERSANT 7 -- man ! I neede quite some time to get this syntax change ...

Peer Zaal wrote : From the Friday talking, with John and Albrecht and Kevin, I learned that you always should include the "- i" option in order to pre-allocate disc space, i.e. you never run the risk that your database crashes because the disc space added is not available due to limited disc-resources. John mentioned that the "-i" also improves the performance of a versant (write) interaction with the database of interest. Furthermore Albrecht always add junks of about 2G, and increase as many volumes expected to be needed:

 > addvol -i -n volume1 -p volume1 -s 2000M database_name

Note, never add full path definitions in your addVol call as this will cause problems at the moment you do a backup/restore to another machine.

Backup the Database

There is a hidden keyword which speed on the backup process :

vbackup -nobuffering -dev <name of the Database>.vbck -backup <name of the Database>

Indexing the Database

Make sure that you have enough space in the database left:

dbtool -space -volume -all <dbname>

Free space should be at least 20%.

java -Xmx512M herschel.access.db.CreateIndexes <dbname>

Alternatively, using the versant commands:

dbtool -index -create -btree herschel.versant.ccm.TmSourcePacketImpl _time <dbname>

Improving Backup speed

vbackup works very slow (especially on PACS1 Database server (1.2 Ghz / 50G in 2.5 h). From Albrecht I learned a not documented keyword which improves the speed significant :

 vbackup -nobuffering -dev <dbname>.vbck -backup <dbname>

Database propagation with ESAC

HW and OS setup

For the network setup please see http://www.xray.mpe.mpg.de/~paul/netz/pacs.html

Currently following network addresses are used for PACS:

  • 195.74.188.85 --- Erich's laptop used for testing
  • 195.74.188.86 --- pacs6

The host pacs6 is configured in a way that all packets for the network 195.74.188.80/28 are routed to the network interface eth2 which is connected to the ESA router in the basement.

Herschel: PACS/Database (last edited 2009-07-15 14:32:38 by localhost)