Differences between revisions 21 and 23 (spanning 2 versions)
⇤ ← Revision 21 as of 2021-02-23 13:46:49 →
Size: 10441
Editor: HermannBrunner
Comment:
← Revision 23 as of 2021-03-02 08:48:02 → ⇥
Size: 10501
Editor: HermannBrunner
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from eSASSdevel_210208 #acl EroCatGroup:read,write,revert All:read
Line 52: Line 52:
       * testing still ongoing
Line 67: Line 68:
       * Timescale for full 947 TEL chain runs?        * Timescale for full 947 TEL/EXP chain runs?
Line 74: Line 75:
     * TM3/TM4 badpix

Weekly eSASS developers meeting

Start of meeting: 2021-02-23 13:30 (usual time again)

Zoom link: https://zoom.us/j/97439409844?pwd=YXVkK0hBMnpybm5tL2VyYTh2Qld2UT09

Participants: Ian Stewart, Hermann Brunner, Miriam Ramos, Alain Gueguen, Konrad Dennerl, Mara Salvato, Sabine Osterhage, Susanne Friedrich, Jeremy Sanders. Chandeyee Maitra, Andrea Merloni, Michael Freyberg

  • Open actions from previous developers meetings
    • Act20200929/1_all make sure html task descriptions are current and consistent with user release
      • updates by Jeremy and Ian (evtool, expmap, catprep)
      • still missing: Hermann (radec2xy), Georg (detection + evatt)
    • Act20201006/1_SF clean event file description, update source list description
  • Preprocessing / NRTA
  • eFEDS catalog paper
    • contributers
    • division of work
    • technical appendices
      • level of detail?
      • copying from wiki?
    • times scales
    • further meetings

  • eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page

    • evprep
      • discuss what needs to be updated for 947 proessing
    • expmap
      • withintputmaps=yes
      • new parameter inputmaps
    • catprep
    • makefile flags
    • ermldet
      • positional error / count error 0
        • problem identified but not yet fixed
      • upper/lower count rate errors - needs to be propagated by catprep

      • now works with non-automatic flag: -> ermldet-1.37

    • ftfindhotpix / badpix / caldb access in general
      • ero_cifsl now return files in decending validity start date order (current is first)
      • now evaluates TMIN/TMAX / needs to be consistent with pattern (Ian/Konrad will discuss offline)
      • event list extension seems to have fewer badpix entries than caldb (TM7) - Susanne
    • Temperature dependent energy calibration
      • New energy caldb file, calvers.indx needed for eSASSdevel/001/946/947 (make sure that this works)
    • fftw version (Alain)
      • different ermldet results on old/new machines

      • upgrading -> 3.3.9, also Docker

      • still different results between xenial and eSASSuser
      • pipeline eSASS/he*ero machines: fftw taken from environment
        • probably fftw-3.3.7 in pipeline eSASS
      • testing still ongoing
    • Pipeline tasks (shadow developer/pipeline operator)
      • eropipe, proc, prexec - Alain
      • telstage, dalist, rmvers - Miriam, ?
      • test environment, current task versions
    • Recent task updates:
      • gtcal-1.13
      • ftfindhotpix-1.8
      • expmap-2.3
      • ermldet-1.37
  • Pipeline / ongoing processing

    • processing wiki page

    • 947 test runs
      • GL's boresight analysis
      • missing polar sky tiles (087156, 093156)
      • Timescale for full 947 TEL/EXP chain runs?
        • Required update/fixes?

      • DAtool / ProcStat?

    • 001 rerun of 082156 (eDays 43546-56)
    • eRASS2/3 946 exposure deficit
      • use latest HK file versions + tel-1.24 (telgti border issues)
      • Sky tile 46008 + 46085, 946 processing sine Feb. 17
    • TM3/TM4 badpix
    • TEL_949 chain
      • test data for for temperature dependent energy calibration

  • All-sky data access / SciServer

  • EDR
    • Web pages, documentation
    • Validation of CalPV/c001 data: guidlines by eSASS team
  • DAtool / ProcSTat
    • access restrictions for data request logs + project descriptions (discussion ongoing - details tbd)
    • providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
    • move simple copying request to Python - ongoing
  • Longterm activities

    • [?] Moving task archive to git

    • evprep

      • [9] evprep: space invaders event flag

        • evtool should update the gti according to this flag
        • post-processing: frames with many counts?
      • BTI/GTI:
        • PDF for CalPV linked on wiki page (done?)

        • [7] one PDF per eRASS in wiki

    • ermldet

      • [8] ca. 2% of sources have positional error 0, some also have count error 0

        • can't be fixed quickly, may need replacement of algorithm
        • Johan's proposal? Georg to discuss with Johan (long timescale project)
    • Time dependent RMFs/ARFs (srctool?)

      • [8] RMF

      • [7] ARF

    • [6] Background analysis: projecting closed data onto the sky (XMM SAS)

    • [4] ersensmap: implement pointing PSF

    • [3] ftfindhotpix: test and update algorithm

    • [1] Light leak in eSASS/pipeline?

    • [1] DAtool: filtering catalogs (separate from DAtool)?

Previous meeting:

  • Open actions from previous developers meetings
    • Act20200929/1_all make sure html task descriptions are current and consistent with user release
      • updates by Jeremy and Ian (evtool, expmap, catprep)
      • still missing: Hermann (radec2xy), Georg (detection + evatt)
    • Act20201006/1_SF clean event file description, update source list description
  • Preprocessing / NRTA

  • eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page

    • eRASS1/2 947 code updates/code freeze
      • perhaps small evprep updates
      • expmap:
        • parallel version (7 cores)
        • withinputmaps=yes
        • testing?
      • catprep: making number of bands variable (1..5) - in progress
      • detection task updates (ongoing, perhaps ready for 947)
        • positional error / count error 0
        • upper/lower count rate errors - will also be propagated by catprep

        • -> ermldet new task version

      • ftfindhotpix / badpix / caldb access in general
        • accessing highest validity date file vs all overlapping file
        • TMIN/TMAX columns not use in current version
        • Ian will make ero_cifsl return files in decending validity start date order
      • Temperature dependent energy calibration
        • Konrad will provide update of task energy (will require new caldb file)
        • Pattern update: read badpix start and eion)
    • fftw version (Alain)
      • different ermldet results on old/new machines
    • srctool vs. apetool exposure (help desk questes may be related to fftw version (results with v3.3.9 are ok)
      • v3.3.8 in current user release should be upgraded - should be documented (eSASSnews)
      • Docker version needs to be updated
      • Check version in pipeline eSASS
    • eSASSuser on Ubuntu 20 - after fftw issue resolved
      • eropipe, proc, prexec - Alain
      • telstage, dalist, rmvers - Miriam

      • test environment, current task versions -> eSASSdevel/task archive

    • Recent task updates:
  • Pipeline / ongoing processing

    • processing wiki page

    • performance issues?
      • replacement for get_dir routine?
    • 947 test runs
      • eRASS1/2, eRASS:2 (first 100 eDays) are in archive

        • HiPS (eRASS1/2), DAtool, ProcStat (ok)

          • DAtool (947): ImageTot -> Image, energy bands: 0-9, a-d for e?01..., e?02..., s?02...

          • HiPS energy bands? stay with the same bands for now (more bands perhaps later)
        • Astrometry?
    • eRASS2/3 946 exposure deficit
      • use latest HK file versions + tel-1.24 (telgti border issues)
      • fixed in 947 / no full 946 rerun of affected data planned
      • Problem might still exist in recently processed data? e.g., eROday 46283, HB will have look
      • tel-1.24 not yet updated
    • Missing soft sources (Frank)
      • pre-946 processing versions contained CalPV test scans
      • confirmation: rerun test scans in 946 pipeline (or 001?)
      • Survey scans on Dec 8 and Dec 10 should not be part of eRASS1
      • em00 files end of Nov. cover part of LMC. These eDays should be rerun in 001 pipeline
    • TM7 bad pixels
      • caldb was updated, eDays 46199+ restarted
      • data look clean now
    • 949 TEL chain
      • for Konrad's temperature dependent energy calibraton
      • run TEL chain up until pattern
      • CalPV + recent data
  • Data access
    • eRASS event files / images
      • FWC data
  • EDR
    • Web pages, documentation
    • Validation of CalPV/c001 data: guidlines by eSASS team
  • DAtool / ProcSTat
    • access restrictions for data request logs + project descriptions (discussion ongoing - details tbd)
    • providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
    • move simple copying request to Python / should everything be moved to Python? - ongoing
    • DAtool / eROSE : requires change in frontend
  • Longterm activities

    • [?] Moving task archive to git

    • evprep

      • [9] evprep: space invaders event flag

        • evtool should update the gti according to this flag
        • post-processing: frames with many counts?
      • BTI/GTI:
        • PDF for CalPV linked on wiki page (done?)

        • [7] one PDF per eRASS in wiki

    • ermldet

      • [8] ca. 2% of sources have positional error 0, some also have count error 0

        • can't be fixed quickly, may need replacement of algorithm
        • Johan's proposal? Georg to discuss with Johan (long timescale project)
    • Time dependent RMFs/ARFs (srctool?)

      • [8] RMF

      • [7] ARF

    • [6] Background analysis: projecting closed data onto the sky (XMM SAS)

    • [4] ersensmap: implement pointing PSF

    • [3] ftfindhotpix: test and update algorithm

    • [1] Light leak in eSASS/pipeline?

    • [1] DAtool: filtering catalogs (separate from DAtool)?

EROSITAwiki: eSASSdevel_210223 (last edited 2021-03-02 08:48:02 by HermannBrunner)