EROSITAwiki: eSASSdevel_201201

eSASS Q&A / eSASS developers Zoom meeting

Start of meeting: 2020-12-01 13:30 (1:30pm)

Zoom link: https://zoom.us/j/7909307159?pwd=ZEdGeTJQUzdsRGpkd2Q5aUZXQmZZdz09

• eSASS Q&A

• Open actions
  • Act20200929/1_all (Ian, Georg, Jeremy, Antonis, Miriam and Hermann will provide recomendations): make sure html task descriptions are current and consistent with user release
    • task documention for eSASSuser + CalPV/001 should be sent be email, will be integrated in Web page by Miriam
  • Act20201006/1_HB clean event file description, update source list description
  • Act20201006/2_SF document cal files (done) and bti
    • static version on Web page / live version in wiki
  • Act20201020/1_HB+SF check why some ObsIDs were only processed in 945
    • telgti-1.23 running in 001 pipeline
    • processed: 700203, 700204, 700040
    • still missing: 700201, 700202, 900046, 900047, 700015 (c945)
  • Act20201102/1 AG+HB: test how many regions per request DATool can handle - done
    • limit in the frontend: 150 regions (can be changed) / also: 3h time limit
    • backend: probably only time limit (150 regions take more than 10 hours)
    • warning in frontend: SO+HB
  • Act20201117/1_GL: check for out of sync attitude in eRASS (946) and CalPV data (001) - done
    • c946 reruns? Georg will investigate ro which extnt this effects the data
• New actions
• NRTA / preproc
  • DIMM memory error
  • offset /w noise maps CTIMETAG overflow, computed full time in keyword CALCTIME
  • raw frames April 1st, 2020
• Ongoing processing

  • processing issues wiki page

  • Jira discussion?
  • backup issues (discussion Harald/Achim/computer center - ongoing)
    • excluding srtool products from backup? Not running SOU chain every day? Will test every other day
  • turn on optimization in 947 pipeline
  • eRASS2
    • c946 ongoing
    • eROproc_201125 for 947 (task versions)
    • TM1-7 hold files (ITC reset) - removed
  • CalPV / c001
    • missing ObsIDs processed with telgti-1.23, some successes, some still missing
    • 2020 pointings processed
  • RXJ 1856 flux discrepancies (also RXJ2143??)
    • out of sync values in gyro attitude - will be fixed in attprep
    • attprep should discard affected entries, and report about number of occurances
    • fixing c001/c946 processing?
  • flaregti in eFEDS field (Teng/Jeremy)
    • parameters need to be adjusted to field size, should be in documentation, flaregti could also read the field size from input, Teng is using changed parameters for eFEDS
    • Jeremy is investigating flaregti behaviour in survey; parameters for 947
    • simulating random light curves - ongoing
    • testing whether flaregti improve detection - use current settings for c947 for now / use harder band
• Discrepancies in source list of demo script runs (ermldet)
  • Different results results on different machines using same binaries, probably due to uninitialized variables
    • results are consistent on the same machine
    • seems to happen in multi source fits
    • ermldet fix by Georg: update for 947 pipeline and users release?
      • Should Teng confirm whether ermldet patch changes the eFEDS source list?
  • patch user release / update Docker version? Will be fixed for Georg's/Teng's CALPV papers
• heasoft 6.28 (was 6.27.2) / user release
• cron job / email notifications for evprep anomalies - current sttings seem to be ok

• eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page

  • catprep: set output and output2 (947 pipeline)
  • telgti: attitude issues

    • sky tile 062093 (events outside of sky tile) -> eSASS error tracking page

  • detection task updates (ongoing, perhaps ready for 947)
    • clobber option
    • improved RADEC_ERR
    • upper/lower count rate errors
    • ermldet: ca. 2% of sources have positional error 0, some also have count error 0
    • Johan's proposal?
  • library order in .libs files
    • Antonis, Gerog, and Jeremy should change .libs order in their tasks

  • NaNs in physical DS9 coordinates in exposure maps, missing keyword (see Ian's emails), eroio nees to be updated

  • Recent task uploads
    • erolib-2.33.5, erolib-2.34
    • general-1.8
    • eropipe-1.19, dalist-1.4, rmvers-1.4
    • expmap-1.35, expmap-1.35.1, expmap-1.36
• CalPV data release / public software release
  • Following up on meeting last week
  • Updating documentation
    • Task descriptions / default values
    • Wiki documentation
    • Public Web pages
    • Susanne's caldb description
• DAtool
  • Frank's email / accessing RXJ2143 observation did not work
  • Region requests issues
    • owner filtering / evtool: use MPE owner eventflag
    • Were all of Georg's bug reports fixed?
  • providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
  • move simple copying request to Python / should everything be moved to Python? - ongoing
  • DAtool / eROSE interface: requires change in fronted
  • Frontend problem: first request by a user occasionally doesn't work
• Longterm activities
  • ersensmap: implement pointing PSF
  • expmap
    • Bring coordinate transformations into line with evatt
    • Parallel version
  • evprep: experimental event flags
  • post-processing: space invaders, frames with many counts, statistical event flagging
  • DAtool: filtering catalogs?
  • Light leak mitigation
  • Time dependent RMFs/ARFs (srctool?)
  • Background analysis: projecting closed data onto the sky (XMM SAS)
  • Moving to git

Previous meeting:

• Open actions
  • Act20200929/1_all (Ian, Georg, Jeremy, Antonis, Miriam and Hermann will provide recomendations): make sure html task descriptions are current and consistent with user release
    • first version of updates should be available before Nov 16
    • update by Antonis
    • task documention for eSASSuser + CalPV/001 should be sent be email, will be integrated in Web page by Miriam
    • Miriam has initial template for updated Web page, to be discussed at meeting next week
  • Act20201006/1_HB clean event file description, update source list description
  • Act20201006/2_SF document cal files (done) and bti
    • static version on Web page / live version in wiki
  • Act20201020/1_HB+SF check why some ObsIDs were only processed in 945
    • GL: fixed in telgti-1.21 (not yet tested at MPE)
  • Act20201102/1 AG+HB: test how many regions per request DATool can handle
    • limit in the frontend: 150 regions (can be changed) / also: 3h time limit
    • backend: probably only time limit (150 regions take more than 10 hours - warning in frontend?)
  • Act20201117/1_GL: check for out of sync attitude in eRASS (946) and CalPV data (001) - done

• New action
• NRTA / preproc
  • eSASS pipleine will start if H_SciHKA00... and H_SciHKA06... are not (yet) present
    • evprep will not fail if no SciHK are present, but it will not be able to perform some corrupted frame flagging
  • offset / noise maps
• Ongoing processing

  • processing issues wiki page

    • Jira discussion?
  • backup issues (discussion Harald/Achim/computer center - ongoing)
    • excluding srtool products from backup? Not running SOU chain every day? Will test every other day
      • turn on optimization in pipeline
  • eRASS1/2:
    • recent eRASS2 errors?
      • missing sky tile, SOU chain errors
    • starting 947 event chains
  • 001 processing:
    • process missing ObsIDs (upgrade to telgti-1.21), also eRASS cal pointings
    • check whether BTI were considered for 001 processing
    • Error in header keywords
  • RXJ 1856 flux discrepancies
    • out of sync values in gyro attitude - will be fixed in attprep
    • look for differene between record and frame time
      • four cases after eROday 45000 / many early in mission (Georg's email)
      • Georg will check 001 CALPV event files
      • fixing 946 eRASS, CalPV
      • attprep should discard affected entries, and report about number of occurances
  • flaregti in eFEDS field (Teng/Jeremy)
    • parameters need to be adjusted to field size, should be in documentation, flaregti could also read the field size from input, Teng is using changed parameters for eFEDS
    • Jeremy is investigating flaregti behaviour in survey; parameters for 947
    • simulating random light curves - ongoing
    • testing whether flaregti improve detection
    • First resuls by Jeremy
  • eSASSuser release
    • Discrepancies in source list of demo script runs
      • Different results results on different machines using same binaries, probably due to uninitialized variables
        • results are consistent on the same machine
        • seems to happen in multi source filts
      • patch user release / update Docker verskon? Will be fixed for Georg's/Teng's CALPV papers
  • For future releases
    • Removing build warnings in eSASS release (unused variables, etc.)
  • cronjobs, scripts, etc.
    • Script for evaluating evprep log info?
      • task by Ian, Susanne's plots, Chandreyee's script?
        • works in eSASSdevel, move to eroproc / cronjob - send to NOSKY distribution

  • eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page

    • shapelib - Pavel's email
    • catprep: set output and output2 (947 pipeline)
    • telgti: attitude issues
      • sky tile 062093 (events outside of sky tile)
    • expmap: effective area weighting?
    • detection task updates (ongoing)
      • clobber option
      • improved RADEC_ERR
      • upper/lower count rate errors
      • ermldet: ca. 2% of sources have positional error 0, some also have count error 0
      • Johan's proposal?
    • Recent task uploads
      • erolib-2.33.4?
      • cal-1.17?
  • CalPV data release / public software release
    • Following up on meeting last week
    • Updating documentation
      • Task descriptions / default values
      • Wiki documentation
      • Public Web pages
      • Susanne's caldb description
  • DAtool
    • Region requests issues
      • owner filtering / evtool: use MPE owner eventflag
      • Were all of Georg's bug reports fixed?
    • providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
    • move simple copying request to Python / should everything be moved to Python? - ongoing
    • DAtool / eROSE interface: requires change in fronted
    • Frontend problem: first request by a user occasionally doesn't work
  • Longterm activities
    • ersensmap: implement pointing PSF
    • expmap
      • Bring coordinate transformations into line with evatt
      • Parallel version
    • evprep: experimental event flags
    • post-processing: space invaders, frames with many counts, statistical event flagging
    • DAtool: filtering catalogs?
    • Light leak mitigation
    • Time dependent RMFs/ARFs (srctool?)
    • Background analysis: projecting closed data onto the sky (XMM SAS)
    • Moving to git

Describe eSASSdevel_201201 here.