3638
Comment:
|
4027
|
Deletions are marked like this. | Additions are marked like this. |
Line 5: | Line 5: |
*[wiki:Self:PACS/Rsync Copying Pools with Rsync] *[wiki:Self:PACS/RsyncAdmin Data Pool Rsync Administration] |
|
Line 134: | Line 137: |
You may also define your own pools. In your .pcss directory you define a Pool (in this case PoolExample.py : {{{ pool = SimplePool.getInstance("PoolExample") }}} Now you may access it in your jide session : {{{ pool = PoolManager.getPool("PoolExample") }}} |
PACS Data Pools
General
Poolmanager : poolman@pacs1
[wiki:PACS/Rsync Copying Pools with Rsync]
[wiki:PACS/RsyncAdmin Data Pool Rsync Administration]
Available Pools
# ********** The official PACS Pool for Level 0 data of the ILT2 ************ # Status empty # Port 44530 hcss.dbpool.database = pacs_standard_products_fmilt@pacs1.mpe-garching.mpg.de # ********** The official PACS calibration data Pool ****************** # Status : empty # Port : 44520 hcss.dbpool.cal = pacs_calibration_products@pacs1.mpe-garching.mpg.de
Populating the pacs_standard_products_fmilt Pool
Login : oper@pacs5
Set the Pool directors in .pcss/props :
hcss.ia.pal.pool.simple.dir = /pacs/dbBck/PacsProductPools
Currently oper@pacs5 is writing the pool to our server computer pacs1 using nfs mounted disks!
Automatic processing after an Observation has been finished is not yet implemented !
The following script is used for Bulk Processing :
import sys from herschel.store.api import * from herschel.ia.pal.pool.lstore import * from herschel.ia.pal.pool.db import DbPool from java.util import Date obsid = 536870913l # Database from which we fetch the TmSourcePackets database = "pacs_fm_ilt_3@pacs5" # Logfile to trace the bulk processing logfile = open("BulkProcessing.log",'w') # Logfile to trace problems problemfile = open("BulkProcessingProblems.log",'w') # The first 1000 Observations for i in range (1000) : # try : startMessage = "Processing started : "+ str(obsid) + " - " + Date().toString() + "\n" print startMessage logfile.write(startMessage) logfile.flush() # # Actual Processing # stat = populatePacsPool(obsid= obsid, poolName = poolName, poolType=pType, database=database , hklimit = 100000, sclimit = 1000) # endMessage = "Processing finished : "+ str(obsid) + " - " + Date().toString() + "\n" print endMessage logfile.write(endMessage) logfile.flush() # obsid+=1 # except : exceptMessage = "Processing Exception : "+ str(obsid) +" - " +Date().toString() + "\n" print exceptMessage obsid+=1 logfile.write(exceptMessage) problemfile.write(exceptMessage) e,inf,tb = sys.exc_info() print inf logfile.write(str(inf)) problemfile.write(str(inf)) logfile.flush() problemfile.flush() # pass pass logfile.close() problemfile.close()
Make the Pool available for all
Login : poolman@pacs1 Then :
java -Xmx800m -Dproperty.path=$HCSS_PROPS herschel.ia.pal.pool.serial.PoolDaemon 44529 pacs_standard_products_fmilt
Accessing the Pacs Product Pools
The direct way is the following :
pool = CachedPool(SerialClientPool("pacs1.mpe-garching.mpg.de", 44530, "pacs_standard_products_fmilt")) storage = ProductStorage() storage.register(pool) # Browse the Pool with GUI result = browseProduct(storage)
Using the Poolmanager : In the <PCSS_ROOT>/config/storage.props the properties pointing to files containing the Pool definitions are located. One among them is the <stdprod> Pool :
cat pcss/config/pooldefs/stdprod.py pool = CachedPool(SerialClientPool("pacs1.mpe-garching.mpg.de", 44530, "pacs_standard_products_fmilt"))
In the jide session you do :
pool = PoolManager.getPool("stdprod") storage = ProductStorage() storage.register(pool) # Browse the Pool with GUI result = browseProduct(storage) # After filling your shopping basket you may retrieve the products prod=result[0].getProduct()
You may also define your own pools. In your .pcss directory you define a Pool (in this case PoolExample.py :
pool = SimplePool.getInstance("PoolExample")
Now you may access it in your jide session :
pool = PoolManager.getPool("PoolExample")