eSASSdevel_210126 - EROSITAwiki

eSASS developers Zoom meeting

Start of meeting: 2021-01-26 13:30 (1:30pm)

NEW Zoom link: https://zoom.us/j/97439409844?pwd=YXVkK0hBMnpybm5tL2VyYTh2Qld2UT09

Participants: Ian Stewart, Hermann Brunner, Miriam Ramos, Alain Gueguen, Konrad Dennerl, Mara Salvato, Sabine Osterhage, Susanne Friedrich, Jeremy Sanders. Chandeyee Maitra, Andrea Merloni, Michael Freyberg

Open actions
- Act20200929/1_all make sure html task descriptions are current and consistent with user release
  - updates by Jeremy and Ian (evtool, expmap, catprep)
  - still missing: Hermann (radec2xy), Georg (detection + evatt)
- Act20201006/1_SF clean event file description, update source list description
Preprocessing / NRTA
eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page
- eRASS1 code updates/code freeze (Ian's email)
- expmap: parallel version? Jeremy <--> Ian
- catprep: making number of bands variable (1..5)?
- expmerge: 947 crashes
- detection task updates (ongoing, perhaps ready for 947)
  - clobber option
  - improved RADEC_ERR, upper/lower count rate errors
- Pipeline tasks (shadow developer/pipeline operator): eropipe, proc, prexec, telstage, dalist, rmvers
- Recent task updates:
  - expmap-2.1, erolib-2.38, evtool-2.37, apetool-1.10
Pipeline / ongoing processing
- processing wiki page (Miriam's changes)
- eRASS3 (946) exposure deficit (Chandreyee's analysis)
  - Georg thinks this is a telgti problem - might also have h ed in pevious eRASSes - will investigate
- Missing soft sources (Frank)
- Artefacts in CalPV (c001) ObsID 710000 - Teng
  - issue with subpixel calculation, due to unfortunate placement o sourc on pixel grid?
- bad pixels in caldb (TM2/TM3) - Susanne will put them in the caldb
- Temperature dependence in task energy (HK access) - could be part of 947 reprocessing if timescales align
  - should go in 947 pipeline / timescale ca. 2 weeks
  - KOnrad will pove list of eROday to Hermann
- Treating zip patterns as bad pixels?
  - Data quality of events not on zip pattern?
- evprep summary reports - thresholds, incomplete eROdays, etc.
  - threshold raised for TM5/7
- Discrepancies in source list of demo script runs / ermldet uninitialized variables
  - Still different results of same binaries on different machines
  - Different from pervious problem, all sources are affected (parameter are sightly different) - high priority
  - Confirm whether ermldet patch changes the eFEDS source list (Teng?)
  - Patch user release / update Docker version? Will be fixed for Georg's/Teng's CALPV papers
- Hanging pipeline jobs
  - LOCK, ERROR, START files?
- MIP threshold increase (threshold from HK)
  - should be patched in 946 pipeline eSASS
EDR
- Web pages, documentation
- Validation of CalPV/c001 data: guidlines by eSASS team
  - https://wiki.mpe.mpg.de/eRosita/guide_validationEDR
Longterm activities
- BTI/GTI: user access to bad data
  - PDF for CalPV linked for wiki page?
  - one PSF per eRASS in wiki? -> 7
- ersensmap: implement pointing PSF -> 4
- expmap
  - Parallel version -> 10
- ermldet
  - ca. 2% of sources have positional error 0, some also have count error 0
    - can't be fixed quickly, may need replacement of algorithm -> 8
  - Johan's proposal? Georg to discuss with Johan (long timescale project)
- evprep: experimental event flags (space invaders)
  - post-processing: space invaders, frames with many counts, statistical event flagging -> 9
- ftfindhotpix: test and update algorithm
- Light leak in eSASS/pipeline? -> 1
- DAtool: filtering catalogs (separate from DAtool)? -> 1
- Time dependent RMFs/ARFs (srctool?) -> 8 (RMF) 7 (ARF) +/-5
- Background analysis: projecting closed data onto the sky (XMM SAS) -> 6
- Moving to git -> ?

DAtool
- providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
- move simple copying request to Python / should everything be moved to Python? - ongoing
- DAtool / eROSE interface: requires change in frontend
- Retrieving eRASS3/c946 data?
- Other issues
ProcStat
- eRASS1/946?
- eRASS:n -> eRASSn

Previous meeting:

Open actions
- Act20200929/1_all make sure html task descriptions are current and consistent with user release
  - updates by Jeremy and Ian (evtool, expmap, catprep)
  - still missing: Hermann (radec2xy), Georg (detection + evatt)
- Act20201006/1_SF clean event file description, update source list description
Preprocessing / NRTA
- CALCTIME, DATE-OBS, sun angle, etc.
- he3ero maintainance

eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page
- detection task updates (ongoing, perhaps ready for 947)
  - clobber option
  - improved RADEC_ERR, upper/lower count rate errors
- expmap: new coord transformations (v2.0)
- allocatable array vs. pointers
- Recent task uploads
  - expmap-1.36.2/2.0, erolib-2.36, evprep-3.17, evtool-2.17 Ian's email
  - attprep-1.8
EDR
- Web pages, documentation
- Validation of CalPV/c001 data: guidlines by eSASS team
  - https://wiki.mpe.mpg.de/eRosita/guide_validation
Pipeline / ongoing processing
- processing issues wiki page
- c001: RXJ 1856 reprocessed with attprep-1.7/1.8 (700008 ongoing)
- eRASS3(946 )exposure deficit (related to data redump?)
- 947 status
  - SEP
- bad pixels in caldb (TM2/TM3)
- Artefacts in CalPV(c001) ObsID 710000 - Teng
- Temperature dependence in task energy (HK access) - Konrad will provide location of required HK value
  - could be part of 947 reprocessing if timescales align
- Treating zip patterns as bad pixels?
  - data quality of events not on zip pattern? Susanne + perhaps Chandreyee could investigate (input by Teng?)
- Light leak in eSASS/pipeline?
- evprep summary reports - thresholds, incomplete eROdays, etc.
- Discrepancies in source list of demo script runs / ermldet uninitialized variables
  - Still different results of same binaries on different machines
  - Confirm whether ermldet patch changes the eFEDS source list (Teng?)
  - Patch user release / update Docker version? Will be fixed for Georg's/Teng's CALPV papers
- MIP threshold increase (threshold hardcoded?)
- DAtool
  - providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
  - move simple copying request to Python / should everything be moved to Python? - ongoing
  - DAtool / eROSE interface: requires change in frontend
  - Retrieving eRASS3/c946 data?
  - Other issues
- ProcStat
  - eRASS1/946?
  - eRASS:n -> eRASSn
- Longterm activities
  - BTI/GTI: user access to bad data?
  - ersensmap: implement pointing PSF
  - expmap
    - Parallel version
  - ermldet
    - ca. 2% of sources have positional error 0, some also have count error 0
      - can't be fixed quickly, may need replacement of algorithm
    - Johan's proposal? Georg to discuss with Johan (long timescale project)
  - evprep: experimental event flags
  - post-processing: space invaders, frames with many counts, statistical event flagging
  - ftfindhotpix: test and update algorithm
  - DAtool: filtering catalogs?
  - Light leak mitigation
  - Time dependent RMFs/ARFs (srctool?)
  - Background analysis: projecting closed data onto the sky (XMM SAS)
  - Moving to git

EROSITAwiki: eSASSdevel_210126 (last edited 2021-01-26 14:12:04 by HermannBrunner)