PACS - PCSS

TableOfContents(2)

Procedure to develop Pipeline Tasks

  1. Select an assigned IA application from the IA Functionality list at the pacs website BR

  2. Check whether you are assigned to develop this application BR

  3. Contact Juergen (to update the website) and Ekki (to have an informed person) BR

  4. Juergen will flag the application as "under construction" in the IA Functionality list BR

  5. Write the Chapter in the User Manual (against this description you will develop) BR

    • Covering API (parameter, keywords, etc.) BR

    • Algorithm (references if possible) BR

    • Examples how the data is processed (before - after, figures) BR

  6. Contact a "scientific expert" to verify that your application is doing what is expected (if not already done earlier) BR

  7. Select and document Testdata (currently in User manual) BR

  8. Write a TestHarness - please contact Rik, Michael or Ekki if you do not know howBR

  9. Write the application (TestHarness may help) BR

  10. Test the application with the TestHarness BR

  11. Check the application into CVS BR

  12. Inform Juergen/Ekki that its finished BR

  13. Juergen will flag the application as done BR

  14. Tag the application BR

Generate User Manual from XML as PDF

Attention: for the conversion to work properly you have to install a version of dbdoclet.jar. This software comes with its own version of jfreechart. If the dbdoclet.jar is in the classpath before the hcss external libraries, also hcss will use the jfreechart version of dbdoclet. This leads to problems at least with PlotXY and PacsQla. So make sure dbdoclet.jar is added at the end of your classpath!

Usage of TestExecutionBrowser just to get an overview

In JIDE :

from herschel.access.util import *
t=TestExecutionBrowser()

Size of level 0.5 data structures in PCSS

Typical Spectrometer observation after decompression :

One second of Spectrometer data (default mode/3 raw channels) :

Basic science data + reduction add on (error, wavelength)


Detectors : 26 * 18 Ramp length : 64 Readout frequency : 256 Hz NrSubRamps : 8 All data as doubles : 8 Byte

Readouts : 26 * 18 * 256/64 * 8 * 8 = 120 KB Wavelength : same as above : 120 KB Error : same as above : 120 KB

Raw data


Readout frequency : 256 Hz All data as doubles : 8 Byte Nr raw channels : 3

Raw data : 256 * 8 * 3 = 6 KB

DecMec data:


Readout frequency : 256 Hz All data as int : 4 Byte DecMec entries : 13

256 * 4 * 13 = 13KB

==> ~ 380 KB per second

One hour :


==> ~ 1,3 GB per hour

Observation duration <= 3hour (there may be exceptions, but then we reduce the number of user)


==> ~ 4 GB per observation

Before and after reduction step


==> ~ two times before and after processing step 8 GB /person

Product Access Layer

The Product Access Layer (PAL) is now available in PCSS. We encourage everybody to use it. Basic documentation you find under :

ftp://ftp.rssd.esa.int/pub/HERSCHEL/csdt/releases/doc/ia/document/um/html/ch12s02.html

A typical example for Frames is :

storage=ProductStorage()
storage.register(SimplePool.getInstance()) 
reference=storage.save(frames)
print reference.urn

from herschel.pacs.signal import *
reference = storage.load("urn:simple.default:herschel.pacs.signal.Frames:1")

#get back your saved product
frame = reference.getProduct()

To use the GUI Browser to find your Products you may use :

from herschel.ia.pal.browser import ProductBrowser
from herschel.ia.pal.browser import ProductBrowserFrame

p=ProductBrowser(storage)
d=ProductBrowserFrame(p)

To get out the Data you need to move it in the Basket (marking in the GUI). Then you may extract it from the basket :

prod=p.getJideBasket().toArray()[0].getProduct()

Work distribution move to new Calibration Framework

PacsCal_BoloCSs_CQM_1_0.fits
PacsCal_BoloCSs_FM_1_0.fits

PacsCal_BoloFlatFields_CQM_1_0.fits
PacsCal_BoloFlatFields_FM_1_0.fits

PacsCal_BoloMasks_CQM_1_0.fits

PacsCal_ChopJitterThreshold_CQM_1_0.fits
PacsCal_ChopJitterThreshold_FM_1_0.fits
PacsCal_ChopJitterThreshold_FM_2_0.fits

PacsCal_ChopperAngle_CQM_1_0.fits
PacsCal_ChopperAngle_FM_1_0.fits
PacsCal_ChopperAngle_FM_2_0.fits

PacsCal_ChopperAngleRedundant_FM_1_0.fits
PacsCal_ChopperAngleRedundant_FM_2_0.fits

PacsCal_ChopperSkyAngle_CQM_1_0.fits
PacsCal_ChopperSkyAngle_FM_2_0.fits

PacsCal_DiscardRampHooks_CQM_1_0.fits

PacsCal_EffectiveCapacitance_CQM_1_0.fits
PacsCal_EffectiveCapacitance_FM_1_0.fits

PacsCal_FilterTransmissionB2_CQM_1_0.fits
PacsCal_FilterTransmissionB3_CQM_1_0.fits
PacsCal_FilterTransmissionBL_CQM_1_0.fits
PacsCal_FilterTransmissionBS_CQM_1_0.fits
PacsCal_FilterTransmission_FM_1_0.fits
PacsCal_FilterTransmission_FM_1_1.fits
PacsCal_FilterTransmission_FM_1_2.fits
PacsCal_FilterTransmissionR1_CQM_1_0.fits
PacsCal_FilterTransmissionR_CQM_1_0.fits

PacsCal_FilterWheel2Band_CQM_1_0.fits

PacsCal_GPRHall_CQM_2_0.fits

PacsCal_GratingJitterThreshold_CQM_1_0.fits

PacsCal_KeyWavelength_CQM_1_0.fits

PacsCal_LabelDescription_CQM_1_0.fits

PacsCal_LittrowParameters_CQM_1_0.fits
PacsCal_LittrowParameters_CQM_2_0.fits
PacsCal_LittrowParameters_FM_1_0.fits

PacsCal_OBCPDescription_CQM_1_0.fits

PacsCal_PhotAbsorption_FM_1_0.fits
PacsCal_PhotAbsorption_FM_1_1.fits

PacsCal_PhotArrayInstrument_CQM_1_0.fits
PacsCal_PhotArrayInstrument_FM_1_0.fits
PacsCal_PhotArrayInstrument_FM_2_0.fits
PacsCal_PhotArrayInstrument_FM_2_1.fits

PacsCal_PhotBadPixelsMask_FM_1_0.fits

PacsCal_PhotCorrZeroLevel_FM_1_0.fits

PacsCal_PhotCrosstalkMatrix_FM_1_0.fits

PacsCal_PhotGain_CQM_1_0.fits
PacsCal_PhotGain_FM_1_0.fits

PacsCal_PhotSatLimits_CQM_1_0.fits
PacsCal_PhotSatLimits_FM_1_0.fits


PacsCal_PhotSortMatrix_CQM_1_0.fits     - MW
PacsCal_PhotSortMatrix_FM_0_0.fits      - MW
PacsCal_PhotSortMatrix_FM_1_0.fits      - MW
PacsCal_PhotSortMatrix_FM_1_1.fits      - MW

PacsCal_RampModel_CQM_1_0.fits

PacsCal_RsrfB2_CQM_1_0.fits
PacsCal_RsrfB2_CQM_2_0.fits
PacsCal_RsrfR1_CQM_1_0.fits
PacsCal_RsrfR1_CQM_2_0.fits

PacsCal_SatLimits_CQM_1_0.fits
PacsCal_SatLimits_FM_1_0.fits

PacsCal_Siam_CQM_1_0.fits

PacsCal_SpecBadPixelsMask_FM_1_0.fits

PacsCal_SpecRedPSF_CQM_1_0.fits

PacsCal_SpecSatLimits_CQM_1_0.fits
PacsCal_SpecSatLimits_FM_1_0.fits

PacsCal_SpecSortMatrix_CQM_1_0.fits     - MW    
PacsCal_SpecSortMatrix_FM_1_0.fits      - MW

PacsCal_SpecVolts_CQM_1_0.fits          
PacsCal_SpecVolts_FM_1_0.fits

PacsCal_SubarrayArray_CQM_1_0.fits      - EkW
PacsCal_SubarrayArray_FM_1_0.fits       - EkW
PacsCal_SubarrayArray_FM_2_0.fits       - Ekw
PacsCal_SubarrayArray_FM_2_1.fits       - Ekw