eSASSdevel_210706 - EROSITAwiki

Monthly eSASS Q&A / weekly eSASS developers meeting

Start of meeting: 2021-07-06 13:30

Zoom link: https://zoom.us/j/97439409844?pwd=YXVkK0hBMnpybm5tL2VyYTh2Qld2UT09

eSASS Q&A
Preprocessing / NRTA
EDR
- data downloads, etc.
- docker eSASS, cone search?
- forum, subforums?
- Internal task descriptions: should link to current user release
License for public eSASS
- -> GPL2 or later
- where to add the license text?
  - should be in download folder and each tar ball (and in docker)
    - in the future each developer should put the license file in the tar ball (put in make file)
    - are there any MPG considerations (ask Achim Bohner)
eFEDS referee report
- ask Antonis about hi availability
- Andrea/Mara are available till Aug 7.
- Figure on scanning pattern?
DATool
- srctool products for cat+products region requests
  - fixed in Alain's dev version, will be brought online after more testing
980 test pipeline for caldb
- Susanne will use F003 in raw/proc
Pipeline / ongoing processing
- processing wiki page Pipeline / ongoing
- bti, badpix, held cameras
- 946 / eRASS4
- 947 / eRASS1/2
  - checking results?
  - processing speed
- 946/947 errors?
  - TEL chain: 47102..05 TM4?
- HiPS maps
  - IKI/MPE owner flag
eSASS tasks eSASSdevel page / Error tracking page
- From previous meeting
  - PSF normalization
    - repeat test with new ermldet version (to be uploaded) Teng+Georg
      - Georg will upload ermldet update today (for 947)
  - detection vs sensitivity map
    - Frank <-> Georg (both are/will go on vacation soon)
      - may be due to chopper issues in CalPV
      - no further tests before Georg's vacation
  - srctool option to use multipe envent files
    - if lightcurve time bin spans two event files rates will be wrong (documentation issue)
    - should be placed in known issues for CalPV - done
  - Check whether there are still keyword issues in recent survey data (TSTART/TSTOP, RA_CEN/DEC_CEN, OWNER)
    - check in 947 test dasta
    - TSTART/TSTOP still open
    - FRAMETIM: hardcoded (eroio?), caldb?
      - fix in eroio for now (email by Michael?)
  - expmap
    - add a warning, potential inconsitency between withdetmap command line parameter and selection events on out of fov bit (is on Ian's todo list)
- ftfindhotpix (further testing?)
  - v1.18 should also go in 946
  - Ian will still do in depth tests (compare history of changes for TM4)
- Recent task updates
  - erolib-2.58.1 / 2.58.2
  - evprep-3.33
  - ftfindhotpix-1.17 / 1.18
  - expmerge-1.25
  - tel-1.26
    - some crashes du to multiple concurrent runs of same dataset
  - flaregti-1.24
- future updates for evtool (947) and expmap
- HB: update telatt (v1.6) in 946
Longterm activities
- [?] Moving task archive to git
- evprep
  - [9] evprep: space invaders event flag
    - evtool should update the gti according to this flag
    - post-processing: frames with many counts?
  - BTI/GTI:
    - PDF for CalPV linked on wiki page (done?)
    - [7] one PDF per eRASS in wiki
- ermldet
  - [8] ca. 2% of sources have positional error 0, some also have count error 0
    - can't be fixed quickly, may need replacement of algorithm
    - Johan's proposal? Georg to discuss with Johan (long timescale project)
- Time dependent RMFs/ARFs (srctool?)
  - [8] RMF
  - [7] ARF
- [7] EV_WEIGHT: event based vignetting correction
- [6] Background analysis: projecting closed data onto the sky (XMM SAS)
- [4] ersensmap: implement pointing PSF
- [3] ftfindhotpix: test and update algorithm
- [1] Light leak in eSASS/pipeline?
- [1] DAtool: filtering catalogs (separate from DAtool)?

Previous meeting:

Last minute EDR related?
- forum
  - Alain and Miriam will be managers
  - All eRO_DE users should contribute
  - internal (protected) subforums?
- Page counters
Preprocessing / NRTA
Pipeline / ongoing processing
- processing wiki page Pipeline / ongoing process
- caldb
  - badpix / badcamt
    - strategy ('soft' zip patterns, etc.)
    - TM5/7 (held data since June 4)
    - TM4 (held data since June 1)
      - Chandreyee will send Ian TM4 test data
      - Susanne will update bad pixel file
- 946
  - move to ftfindhotpix-1.16 after testing in 947
    - should also work with xextent column
    - start eRASS4 once FTFINDHOTPIX is in 946
      - reprocess from 47027 (all TMs)
      - 46959 -> put in error tracking page (eRASS3)
  - finishing eRASS3 (-470027) NO
    - ftfindhotpix-1.13 (rawx=0)
      - from 470018 (check)
    - TM5/7 testing / hold files
    - all LOCK files removed
      - still incomplete sky tiles?
        093156 DET chain currently running
        SASS.LOG not updating??
    - SOU chain runs
      - 86 running / 41 waiting
  - starting eRASS4 after ftffindhotpix-1.16 is in 946
- 947 (100 eROday test)
  - TEL chain errors (Miriam's list)
  - EXP chain
    - PE0040
    - using FLAREGTI?
      - test in 947 pipeline
  - DET chain
    - update chain script and eropipe2
    - default.libs: eSASSdevel -> eROproc_201125
  - SOU chain
    - subdirectories for srctool products
      - subdirectories for srctool products: split by eRASS and by groups of each 100 sources if needed
    - lightcurve binning (try LCTYPE=SRCGTI- or SRGGTI+ -Tom)
    - specfit/varichck (Ian/Hermann)
- New test pipeline for s/w+caldb testing
  - eROproc_testDev (eroproc user)
  - also make (duplicate) test caldb, test new caldb files in this test environment
  - dedicated meeting?
- Shutdown of HE computers on Sat. July 3
  - HB will stop eropipe on all machines on Friday.
eSASS tasks eSASSdevel page / Error tracking page
- From last meeting (June 15):
  - PSF normalization
    - repeat test with new ermldet version (to be uploaded) Teng+Georg
  - detection vs sensitivity map
    - Frank <-> Georg (both are/will go on vacation soon)
      - may be due to chopper issues in CalPV
  - evprep - chopper handling (handled correctly in evprep-3.31)
  - srctool - varying number of TMs
    - fixed in 1.67
  - srctool option to use multipe envent files
    - if lightcurve time bin spans two event files rates will be wrong (documentation issue)
    - should be placed in known issues for CalPV
  - Check whether there are still keyword issues in recent survey data (TSTART/TSTOP, RA_CEN/DEc_CEN, OWNER)
    - Chandreyee will have a look
- Recent task updates (since June 15)
  - Which task version in 946/947/new test pipeline?
  - evprep-3.29...3.32.1 (947)
  - ftfindhotpix-1.13...1.16
  - erolib-2.57/2.58
  - expmap-2.6.3
    - add a warning, ptential inconsitency between withdetmap command line parameter and selection events on out of fov bit
  - erotools-1.2
DAtool
- srctool products for 'catalog with products'
  - Alain will fix this
Longterm activities
- [?] Moving task archive to git
- evprep
  - [9] evprep: space invaders event flag
    - evtool should update the gti according to this flag
    - post-processing: frames with many counts?
  - BTI/GTI:
    - PDF for CalPV linked on wiki page (done?)
    - [7] one PDF per eRASS in wiki
- ermldet
  - [8] ca. 2% of sources have positional error 0, some also have count error 0
    - can't be fixed quickly, may need replacement of algorithm
    - Johan's proposal? Georg to discuss with Johan (long timescale project)
- Time dependent RMFs/ARFs (srctool?)
  - [8] RMF
  - [7] ARF
- [7] EV_WEIGHT: event based vignetting correction
- [6] Background analysis: projecting closed data onto the sky (XMM SAS)
- [4] ersensmap: implement pointing PSF
- [3] ftfindhotpix: test and update algorithm
- [1] Light leak in eSASS/pipeline?
- [1] DAtool: filtering catalogs (separate from DAtool)?

EROSITAwiki: eSASSdevel_210706 (last edited 2021-07-06 15:28:19 by HermannBrunner)