eSASSdevel_201215 - EROSITAwiki

eSASS developers Zoom meeting

Start of meeting: 2020-12-15 13:30 (1:30pm)

Zoom link: https://zoom.us/j/7909307159?pwd=ZEdGeTJQUzdsRGpkd2Q5aUZXQmZZdz09

Participants: Ian Stewart, Hermann Brunner, Ingo Kreykenbohm, Miriam Ramos, Alain Gueguen, Konrad Dennerl, Mara Salvato, Sabine Osterhage, Susanne Friedrich

Open actions
- Act20200929/1_all make sure html task descriptions are current and consistent with user release
  - updates by Jeremy and Ian (evtool - done, expmap, catprep)
  - still missing: Hermann (radec2xy), Georg (detection + evatt)
- Act20201006/1_SF clean event file description, update source list description
  - Miriam will provide HTML style input
- Act20201117/1_GL: check for out of sync attitude in eRASS (946) and CalPV data (001) - done
  - c946 reruns? Georg will investigate to which extent this effects the data quality
    - List of out of order attitude will be placed in wiki / should be linked from Andreas CalPV page
    - Chandeyee has script to convert s/c clock into ObsIDs (using planning file) - done
    - GL will provde attprep patch, HB will rerun RXJ1856 obs + eROday 45648 (eRASS2)
NRTA / preproc
- DRAM turned on again
- Space on /ero_archive
eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page
- catprep: set output and output2 (947 pipeline)
- detection task updates (ongoing, perhaps ready for 947)
  - clobber option
  - improved RADEC_ERR
  - upper/lower count rate errors
  - ermldet: ca. 2% of sources have positional error 0, some also have count error 0
    - can't be fixed quickly, may need replacement of algorithm
  - Johan's proposal? Georg discuss with Johan (long timescale project)
- evtool: doesn't accept symbolic link in input file path
- evprep-3.16 crashes in c946 processing; requires new task parameter, Ian will provide info
- library order in .libs files
  - Antonis, Georg, and Jeremy should change .libs order in their tasks
    - tasks in libs directory should be in correct order - we don't understand why in some cases it also works with the wrong oder (or if libs are not included, e.g. srctool)
- NaNs in physical DS9 coordinates in exposure maps, missing keyword (see Ian's emails), eroio needs to be updated (not yet uploaded)
- heasoft 6.28 (was 6.27.2) / user release - patch by Philipp , now compiles on bionic
  - Alain will patch the autoconf, Miriam will remove warning from Wiki
- Recent task uploads
  - srctool-1.64, flaregti-1.21 (libs order)
  - ermldet-1.36 (uninitialized values, libs order)
  - erbackmap-1.18 (accepts I*1 mask)
  - erolib-2.24.6 (bug fixes)
Ongoing processing
- processing issues wiki page
- backup issues (discussion Harald/Achim/computer center - ongoing)
- 947 pipeline (eROproc_201125): TEL_947 chain of first 100 eRASS1 eROdays run
- eRASS2/3
  - c946 ongoing, switching to eRASS3 processing tomorrow
    - telstage update today
  - test sky tile updates every other day
  - evprep summary reports
- User release / 947 pipeline: discrepancies in source list of demo script runs (ermldet)
  - Confirm whether ermldet patch changes the eFEDS source list (Teng?)
  - Patch user release / update Docker version? Will be fixed for Georg's/Teng's CALPV papers
- HiPS maps
  - eRASS3/946
  - update frequency, RGB maps in optimized bands (for 947 pipeline)
- RXJ 1856 flux discrepancies
  - out of sync values in gyro attitude - will be fixed in attprep
  - attprep will discard affected entries, and report about number of occurences
  - fixing c001/c946 processing?
- flaregti testing
  - Jeremy's analysis
- archive rsync jobs: /data53s -> /ero_archive7
CalPV data release / public software release
- Updating documentation /new Web pages
  - https://erosita.mpe.mpg.de/index_new.html (password required)
  - password access for internal pages
- Calibration pages?
- Validation of CalPV/c001 data: guidlines by eSASS team?
DAtool
- Frank's email / accessing RXJ2143 observation did not work
- providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
- move simple copying request to Python / should everything be moved to Python? - ongoing
- DAtool / eROSE interface: requires change in frontend
- Other issues
  - first request by a user occasionally doesn't work
  - no data retrieved if multiple entries per ObsID in planning file (RXJ 1856)
Longterm activities
- BTI/GTI: user access to bad data?
- ersensmap: implement pointing PSF
- expmap
  - Bring coordinate transformations into line with evatt
  - Parallel version
- evprep: experimental event flags
- post-processing: space invaders, frames with many counts, statistical event flagging
- DAtool: filtering catalogs?
- Light leak mitigation
- Time dependent RMFs/ARFs (srctool?)
- Background analysis: projecting closed data onto the sky (XMM SAS)
- Moving to git
Meeting next week (Dec. 22)?

Previous Meeting:

Open actions
- Act20200929/1_all make sure html task descriptions are current and consistent with user release
- Act20201006/1_SF clean event file description, update source list description
  - Miriam will provide HTML style input
- Act20201102/1 AG+HB: test how many regions per request DATool can handle - done
  - warning in frontend: SO+HB - done
- Act20201117/1_GL: check for out of sync attitude in eRASS (946) and CalPV data (001) - done
  - c946 reruns? Georg will investigate to which extent this effects the data quality
    - List of out of order attitude will be placed in wiki / should be linked from Andreas CalPV page
    - Chandeyee has script to convert s/c clock into ObsIDs (using planning file)
    - GL will provde attprep attprep patch, HB will rerun RXJ1656 obs + eROday 45648 (eRASS2)
NRTA / preproc
- out of order records / bad FRAMETIME values since ITC reset?
eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page
- catprep: set output and output2 (947 pipeline)
- detection task updates (ongoing, perhaps ready for 947)
  - clobber option
  - improved RADEC_ERR
  - upper/lower count rate errors
  - ermldet: ca. 2% of sources have positional error 0, some also have count error 0
  - Johan's proposal?
- library order in .libs files
  - Antonis, Georg, and Jeremy should change .libs order in their tasks
    - tasks in libs directory should be in correct order - we don't understand why in some cases it also works with the wrong oder (or if libs are not included, e.g. srctool)
- NaNs in physical DS9 coordinates in exposure maps, missing keyword (see Ian's emails), eroio nees to be updated
- Recent task uploads
Ongoing processing
- processing issues wiki page
  - exposure map issues (may be related to data dump procedure w/o DRAM memory use):
    - single FoV depressions in survey exposure
    - long stripes with reduced exposure
    - bad FRAMETIME values (Ian's plots)
      - Ian will write up current analysis, to be discussed at next oepration meeting
- backup issues (discussion Harald/Achim/computer center - ongoing)
- 947 pipeline (eROproc_201125): TEL_947 chain of first 100 eRASS1 eROdays run
- eRASS2
  - c946 ongoing, no hold files, no held eROdays
  - SOU chains restarted, was stopped since weekend
  - evprep summary reports
  - test sky tile updates every other day
  - evprep summary reports
- CalPV / c001
  - all CalPV ObsIDs processed
  - Fixing header keywords - fixed by Chandreyee
- User release: discrepancies in source list of demo script runs (ermldet)
  - Different results on different machines, happens in multi source fits
    - ermldet fix by Georg: update for 947 pipeline and users release?
      - Should Teng confirm whether ermldet patch changes the eFEDS source list?
      - patch user release / update Docker version? Will be fixed for Georg's/Teng's CALPV papers
- HiPS maps
  - update frequency, RGB maps in optimized bands (for 947 pipeline)
- RXJ 1856 flux discrepancies
  - out of sync values in gyro attitude - will be fixed in attprep
  - attprep should discard affected entries, and report about number of occurances
  - fixing c001/c946 processing?
- flaregti testing
  - Jeremy's analysis
heasoft 6.28 (was 6.27.2) / user release
CalPV data release / public software release
- Updating documentation /new Web pages
  - https://erosita.mpe.mpg.de/index_new.html (password required)
  - password access for internal pages?
DAtool
- eRASS3 data: e[m,b]0[1-3]*
  - providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
  - move simple copying request to Python / should everything be moved to Python? - ongoing
  - DAtool / eROSE interface: requires change in frontend
  - Other issues
    - first request by a user occasionally doesn't work
    - no data retrieved if multiple entries per ObsID in planning file (RXJ 1856)
Longterm activities
- ersensmap: implement pointing PSF
- expmap
  - Bring coordinate transformations into line with evatt
  - Parallel version
- evprep: experimental event flags
- post-processing: space invaders, frames with many counts, statistical event flagging
- DAtool: filtering catalogs?
- Light leak mitigation
- Time dependent RMFs/ARFs (srctool?)
- Background analysis: projecting closed data onto the sky (XMM SAS)
- Moving to git