PACS Data Pools

General

Poolmanager : poolman@pacs1

[wiki:PACS/Rsync Copying Pools with Rsync]

Available Pools

# ********** The official PACS Pool for Level 0 data of the ILT2 ************
# Status empty
# Port   44530
hcss.dbpool.database  = pacs_standard_products_fmilt@pacs1.mpe-garching.mpg.de

# ********** The official PACS calibration data Pool ******************
# Status : empty 
# Port   : 44520
hcss.dbpool.cal   =  pacs_calibration_products@pacs1.mpe-garching.mpg.de

Populating the pacs_standard_products_fmilt Pool

Set the Pool directors in .pcss/props :

hcss.ia.pal.pool.simple.dir = /pacs/dbBck/PacsProductPools

Currently oper@pacs5 is writing the pool to our server computer pacs1 using nfs mounted disks!

Automatic processing after an Observation has been finished is not yet implemented !

The following script is used for Bulk Processing :

import sys
from herschel.store.api          import *
from herschel.ia.pal.pool.lstore import *
from herschel.ia.pal.pool.db     import DbPool
from java.util                   import Date

obsid = 536870913l

#  Database from which we fetch the TmSourcePackets
database = "pacs_fm_ilt_3@pacs5"

# Logfile to trace the bulk processing
logfile     = open("BulkProcessing.log",'w')
# Logfile to trace problems 
problemfile = open("BulkProcessingProblems.log",'w')

# The first 1000 Observations
for i in range (1000) :
  #
  try :
     startMessage = "Processing started   : "+ str(obsid) + " - " + Date().toString() + "\n"
     print startMessage
     logfile.write(startMessage)
     logfile.flush()
     #
     # Actual Processing
     # 
     stat = populatePacsPool(obsid= obsid, poolName = poolName, poolType=pType, database=database , hklimit = 100000, sclimit = 1000)
     #
     endMessage = "Processing finished  : "+ str(obsid) + " - " + Date().toString()   + "\n"
     print endMessage
     logfile.write(endMessage)
     logfile.flush()
     #
     obsid+=1
     #
  except :
     exceptMessage = "Processing Exception  : "+ str(obsid) +" - " +Date().toString()  + "\n"
     print exceptMessage
     obsid+=1
     logfile.write(exceptMessage)
     problemfile.write(exceptMessage)
     e,inf,tb = sys.exc_info()
     print inf
     logfile.write(str(inf))
     problemfile.write(str(inf))
     logfile.flush()
     problemfile.flush()
     #
     pass
  pass

logfile.close()
problemfile.close()

Make the Pool available for all

java -Xmx800m -Dproperty.path=$HCSS_PROPS herschel.ia.pal.pool.serial.PoolDaemon 44529 pacs_standard_products_fmilt

Accessing the Pacs Product Pools

The direct way is the following :

pool    = CachedPool(SerialClientPool("pacs1.mpe-garching.mpg.de", 44530, "pacs_standard_products_fmilt"))
storage = ProductStorage()
storage.register(pool)

# Browse the Pool with GUI
result  = browseProduct(storage)

Using the Poolmanager : In the <PCSS_ROOT>/config/storage.props the properties pointing to files containing the Pool definitions are located. One among them is the <stdprod> Pool :

cat pcss/config/pooldefs/stdprod.py
pool = CachedPool(SerialClientPool("pacs1.mpe-garching.mpg.de", 44530, "pacs_standard_products_fmilt"))

In the jide session you do :

pool    = PoolManager.getPool("stdprod")
storage = ProductStorage()
storage.register(pool)

# Browse the Pool with GUI
result = browseProduct(storage)

# After filling your shopping basket you may retrieve the products
prod=result[0].getProduct()

You may also define your own pools. In your .pcss directory you define a Pool (in this case PoolExample.py :

pool = SimplePool.getInstance("PoolExample")

Now you may access it in your jide session :