|
Size: 13527
Comment:
|
Size: 13443
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 195: | Line 195: |
| PacsCal_LittrowParameters_CQM_1_0.fits -> PCalSpectrometer_LittrowParameters_QM_v1.fits - EkW PacsCal_LittrowParameters_CQM_2_0.fits -> PCalSpectrometer_LittrowParameters_QM_v2.fits - EkW PacsCal_LittrowParameters_FM_1_0.fits -> PCalSpectrometer_LittrowParameters_FM_v1.fits - EkW |
PacsCal_LittrowParameters_CQM_1_0.fits -> PCalSpectrometer_LittrowParameter_QM_v1.fits - EkW PacsCal_LittrowParameters_CQM_2_0.fits -> PCalSpectrometer_LittrowParameter_QM_v2.fits - EkW PacsCal_LittrowParameters_FM_1_0.fits -> PCalSpectrometer_LittrowParameter_FM_v1.fits - EkW |
| Line 233: | Line 233: |
| PacsCal_SatLimits_CQM_1_0.fits -> PCal_SatLimits_QM_v1.fits - EkW PacsCal_SatLimits_FM_1_0.fits -> PCal_SatLimits_FM_v1.fits - EkW --> What is this compared to the other (Photometer/Spectrometer) SatLimits ???? |
PacsCal_SatLimits_CQM_1_0.fits -> PCalSpectrometer_RampSatLimits_QM_v1.fits - EkW PacsCal_SatLimits_FM_1_0.fits -> PCalSpectrometer_RampSatLimits_FM_v1.fits - EkW |
PACS - PCSS
Procedure to develop Pipeline Tasks
Select an assigned IA application from the IA Functionality list at the pacs website BR
Check whether you are assigned to develop this application BR
Contact Juergen (to update the website) and Ekki (to have an informed person) BR
Juergen will flag the application as "under construction" in the IA Functionality list BR
Write the Chapter in the User Manual (against this description you will develop) BR
Contact a "scientific expert" to verify that your application is doing what is expected (if not already done earlier) BR
Select and document Testdata (currently in User manual) BR
Write a TestHarness - please contact Rik, Michael or Ekki if you do not know howBR
Write the application (TestHarness may help) BR
Test the application with the TestHarness BR
Check the application into CVS BR
Inform Juergen/Ekki that its finished BR
Juergen will flag the application as done BR
Tag the application BR
Generate User Manual from XML as PDF
cd vs-pcss-src>/pacs/develop/pcss/
- checkout the doc-files from the herschel.pacs.doc-files
cd to <cvs-pcss-src>/pacs/develop/pcss/herschel/pacs/doc-files
- make you changes in the *.xml files in help/um/xml
- make sure the following env variables are set:
- $HCSS_DIR pointing to your hcss distribution
- $PCSS_OUT pointing to your development out directory (in this example $HOME/out)
- make a file jake.root
hcss.jake.outdir = $PCSS_OUT
- run the make_um.local from the doc-files directory
help/bin/make_um.local pdf
In the moment you might get an error : xception in thread "main" java.lang.OutOfMemoryError: Java heap space
- If you get no error
- check the result in your web browser pointing to $HOME/out/pcss/doc/help/um/html/um.html
if you get an error : exception in thread "main" java.lang.OutOfMemoryError: Java heap space
you find the $HOME/out/doc/help/um/pdf/um.fo file which will need to be converted (with the help of FOP http://xmlgraphics.apache.org/fop/download.html#binary) to PDF
java -Xmx3g herschel.ia.document.tools.Fo \ -fo ${LOCATION_PDF}/${DOC_NAME}.fo ${LOCATION_PDF}/${DOC_NAME}.pdf- check the result in your web browser pointing to $HOME/out/pcss/doc/help/um/html/um.html
Attention: for the conversion to work properly you have to install a version of dbdoclet.jar. This software comes with its own version of jfreechart. If the dbdoclet.jar is in the classpath before the hcss external libraries, also hcss will use the jfreechart version of dbdoclet. This leads to problems at least with PlotXY and PacsQla. So make sure dbdoclet.jar is added at the end of your classpath!
Usage of TestExecutionBrowser just to get an overview
In JIDE :
from herschel.access.util import * t=TestExecutionBrowser()
Size of level 0.5 data structures in PCSS
Typical Spectrometer observation after decompression :
One second of Spectrometer data (default mode/3 raw channels) :
Basic science data + reduction add on (error, wavelength)
Detectors : 26 * 18 Ramp length : 64 Readout frequency : 256 Hz NrSubRamps : 8 All data as doubles : 8 Byte
Readouts : 26 * 18 * 256/64 * 8 * 8 = 120 KB Wavelength : same as above : 120 KB Error : same as above : 120 KB
Raw data
Readout frequency : 256 Hz All data as doubles : 8 Byte Nr raw channels : 3
Raw data : 256 * 8 * 3 = 6 KB
DecMec data:
Readout frequency : 256 Hz All data as int : 4 Byte DecMec entries : 13
256 * 4 * 13 = 13KB
==> ~ 380 KB per second
One hour :
==> ~ 1,3 GB per hour
Observation duration <= 3hour (there may be exceptions, but then we reduce the number of user)
==> ~ 4 GB per observation
Before and after reduction step
==> ~ two times before and after processing step 8 GB /person
Product Access Layer
The Product Access Layer (PAL) is now available in PCSS. We encourage everybody to use it. Basic documentation you find under :
ftp://ftp.rssd.esa.int/pub/HERSCHEL/csdt/releases/doc/ia/document/um/html/ch12s02.html
A typical example for Frames is :
storage=ProductStorage()
storage.register(SimplePool.getInstance())
reference=storage.save(frames)
print reference.urn
from herschel.pacs.signal import *
reference = storage.load("urn:simple.default:herschel.pacs.signal.Frames:1")
#get back your saved product
frame = reference.getProduct()To use the GUI Browser to find your Products you may use :
from herschel.ia.pal.browser import ProductBrowser from herschel.ia.pal.browser import ProductBrowserFrame p=ProductBrowser(storage) d=ProductBrowserFrame(p)
To get out the Data you need to move it in the Basket (marking in the GUI). Then you may extract it from the basket :
prod=p.getJideBasket().toArray()[0].getProduct()
Work distribution move to new Calibration Framework
PacsCal_BoloCSs_CQM_1_0.fits -> PCalPhotometer_CalSources_QM_v1.fits - EkW PacsCal_BoloCSs_FM_1_0.fits -> PCalPhotometer_CalSources_FM_v1.fits - EkW PacsCal_BoloFlatFields_CQM_1_0.fits -> PCalPhotometer_FlatField_QM_v1.fits - MW PacsCal_BoloFlatFields_FM_1_0.fits -> PCalPhotometer_FlatField_FM_v1.fits - MW PacsCal_BoloMasks_CQM_1_0.fits -> PCalPhotometer_Masks_QM_v1.fits - MW PacsCal_ChopJitterThreshold_CQM_1_0.fits -> PCalCommon_ChopperJitterThreshold_QM_v1.fits - RH PacsCal_ChopJitterThreshold_FM_1_0.fits -> PCalCommon_ChopperJitterThreshold_FM_v1.fits - RH PacsCal_ChopJitterThreshold_FM_2_0.fits -> PCalCommon_ChopperJitterThreshold_FM_v2.fits - RH PacsCal_ChopperAngle_CQM_1_0.fits -> PCalCommon_ChopperAngle_QM_v1.fits - RH PacsCal_ChopperAngle_FM_1_0.fits -> PCalCommon_ChopperAngle_FM_v1.fits - RH PacsCal_ChopperAngle_FM_2_0.fits -> PCalCommon_ChopperAngle_FM_v2.fits - RH PacsCal_ChopperAngleRedundant_FM_1_0.fits -> PCalCommon_ChopperAngleRedundant_FM_v1.fits - JS PacsCal_ChopperAngleRedundant_FM_2_0.fits -> PCalCommon_ChopperAngleRedundant_FM_v2.fits - JS PacsCal_ChopperSkyAngle_CQM_1_0.fits -> PCalCommon_ChopperSkyAngle_QM_v1.fits - RH PacsCal_ChopperSkyAngle_FM_2_0.fits -> PCalCommon_ChopperSkyAngle_FM_v2.fits - RH PacsCal_DiscardRampHooks_CQM_1_0.fits -> PCalSpectrometer_DiscardRampHooks_QM_v1.fits - JS PacsCal_EffectiveCapacitance_CQM_1_0.fits -> PCalSpectrometer_EffectiveCapacitance_QM_v1.fits - EkW PacsCal_EffectiveCapacitance_FM_1_0.fits -> PCalSpectrometer_EffectiveCapacitance_FM_v1.fits - EkW PacsCal_FilterTransmissionB2_CQM_1_0.fits -> PCalCommon_FilterTransmissionB2_QM_v1.fits - PacsCal_FilterTransmissionB3_CQM_1_0.fits -> PCalCommon_FilterTransmissionB3_QM_v1.fits - PacsCal_FilterTransmissionBL_CQM_1_0.fits -> PCalCommon_FilterTransmissionBL_QM_v1.fits - PacsCal_FilterTransmissionBS_CQM_1_0.fits -> PCalCommon_FilterTransmissionBS_QM_v1.fits - PacsCal_FilterTransmission_FM_1_0.fits -> PCalCommon_FilterTransmission_FM_v1.fits - PacsCal_FilterTransmission_FM_1_1.fits -> PCalCommon_FilterTransmission_FM_v2.fits - PacsCal_FilterTransmission_FM_1_2.fits -> PCalCommon_FilterTransmission_FM_v3.fits - PacsCal_FilterTransmissionR1_CQM_1_0.fits -> PCalCommon_FilterTransmissionR1_QM_v1.fits - PacsCal_FilterTransmissionR_CQM_1_0.fits -> PCalCommon_FilterTransmissionR_QM_v1.fits - PacsCal_FilterWheel2Band_CQM_1_0.fits -> PCalCommon_FilterWheel2Band_QM_v1.fits - JS PacsCal_GPRHall_CQM_2_0.fits -> PCalSpectrometer_GprHall_QM_v2.fits - PR PacsCal_GratingJitterThreshold_CQM_1_0.fits -> PCalSpectrometer_GratingJitterThreshold_QM_v1.fits - JS PacsCal_KeyWavelength_CQM_1_0.fits -> PCalSpectrometer_KeyWavelengths_QM_v1.fits - MW - done PacsCal_LabelDescription_CQM_1_0.fits -> PCalCommon_LabelDescription_QM_v1.fits - RH PacsCal_LittrowParameters_CQM_1_0.fits -> PCalSpectrometer_LittrowParameter_QM_v1.fits - EkW PacsCal_LittrowParameters_CQM_2_0.fits -> PCalSpectrometer_LittrowParameter_QM_v2.fits - EkW PacsCal_LittrowParameters_FM_1_0.fits -> PCalSpectrometer_LittrowParameter_FM_v1.fits - EkW PacsCal_OBCPDescription_CQM_1_0.fits -> PCalCommon_ObcpDescription_QM_v1.fits - JS PacsCal_PhotAbsorption_FM_1_0.fits -> PCalPhotometer_Absorption_FM_v1.fits - EkW PacsCal_PhotAbsorption_FM_1_1.fits -> PCalPhotometer_Absorption_FM_v2.fits - EkW PacsCal_PhotArrayInstrument_CQM_1_0.fits -> PCalPhotometer_ArrayInstrument_QM_v1.fits - EkW - done PacsCal_PhotArrayInstrument_FM_1_0.fits -> PCalPhotometer_ArrayInstrument_FM_v1.fits - EkW - done PacsCal_PhotArrayInstrument_FM_2_0.fits -> PCalPhotometer_ArrayInstrument_FM_v2.fits - EkW - done PacsCal_PhotArrayInstrument_FM_2_1.fits -> PCalPhotometer_ArrayInstrument_FM_v3.fits - EkW - done PacsCal_PhotBadPixelsMask_FM_1_0.fits -> PCalPhotometer_BadPixelsMask_FM_v1.fits - MW - done PacsCal_PhotCorrZeroLevel_FM_1_0.fits -> PCalPhotometer_CorrZeroLevel_FM_v1.fits - MW - done PacsCal_PhotCrosstalkMatrix_FM_1_0.fits -> PCalPhotometer_CrosstalkMatrix_FM_v1.fits - MW - done PacsCal_PhotGain_CQM_1_0.fits -> PCalPhotometer_Gain_QM_v1.fits - EkW - done PacsCal_PhotGain_FM_1_0.fits -> PCalPhotometer_Gain_FM_v1.fits - EkW - done PacsCal_PhotSatLimits_CQM_1_0.fits -> PCalPhotometer_SatLimits_QM_v1.fits - EkW - done PacsCal_PhotSatLimits_FM_1_0.fits -> PCalPhotometer_SatLimits_FM_v1.fits - EkW - done PacsCal_PhotSortMatrix_CQM_1_0.fits -> PCalPhotometer_DetectorSortMatrix_QM_v1.fits - MW - done PacsCal_PhotSortMatrix_FM_0_0.fits -> PCalPhotometer_DetectorSortMatrix_FM_v1.fits - MW - done PacsCal_PhotSortMatrix_FM_1_0.fits -> PCalPhotometer_DetectorSortMatrix_FM_v2.fits - MW - done PacsCal_PhotSortMatrix_FM_1_1.fits -> PCalPhotometer_DetectorSortMatrix_FM_v3.fits - MW - done PacsCal_RampModel_CQM_1_0.fits -> PCalSpectrometer_RampModel_QM_v1.fits - RH PacsCal_RsrfB2_CQM_1_0.fits -> PCalSpectrometer_RsrfB2_QM_v1.fits - RH PacsCal_RsrfB2_CQM_2_0.fits -> PCalSpectrometer_RsrfB2_QM_v2.fits - RH PacsCal_RsrfR1_CQM_1_0.fits -> PCalSpectrometer_RsrfR1_QM_v1.fits - RH PacsCal_RsrfR1_CQM_2_0.fits -> PCalSpectrometer_RsrfR1_QM_v2.fits - RH PacsCal_SatLimits_CQM_1_0.fits -> PCalSpectrometer_RampSatLimits_QM_v1.fits - EkW PacsCal_SatLimits_FM_1_0.fits -> PCalSpectrometer_RampSatLimits_FM_v1.fits - EkW PacsCal_Siam_CQM_1_0.fits -> PCalCommon_Siam_QM_v1.fits - RH PacsCal_SpecBadPixelsMask_FM_1_0.fits -> PCalSpectrometer_BadPixelMask_FM_v1.fits - JS PacsCal_SpecRedPSF_CQM_1_0.fits -> PCalSpectrometer_Psf_QM_v1.fits - MW - done PacsCal_SpecSatLimits_CQM_1_0.fits -> PCalSpectrometer_SatLimits_QM_v1.fits - JS PacsCal_SpecSatLimits_FM_1_0.fits -> PCalSpectrometer_SatLimits_FM_v1.fits - JS PacsCal_SpecSortMatrix_CQM_1_0.fits -> PCalSpectrometer_DetectorSortMatrix_QM_v1.fits - MW - done PacsCal_SpecSortMatrix_FM_1_0.fits -> PCalSpectrometer_DetectorSortMatrix_FM_v1.fits - MW - done PacsCal_SpecVolts_CQM_1_0.fits -> PCalSpectrometer_Volts_QM_v1.fits - JS PacsCal_SpecVolts_FM_1_0.fits -> PCalSpectrometer_Volts_FM_v1.fits - JS PacsCal_SubarrayArray_CQM_1_0.fits -> PCalPhotometer_SubarrayArray_QM_v1.fits - EkW - done PacsCal_SubarrayArray_FM_1_0.fits -> PCalPhotometer_SubarrayArray_FM_v1.fits - EkW - done PacsCal_SubarrayArray_FM_2_0.fits -> PCalPhotometer_SubarrayArray_FM_v2.fits - EkW - done PacsCal_SubarrayArray_FM_2_1.fits -> PCalPhotometer_SubarrayArray_FM_v3.fits - Ekw - done