Differences between revisions 10 and 11
⇤ ← Revision 10 as of 2021-02-09 11:47:20 →
Size: 10557
Editor: HermannBrunner
Comment:
← Revision 11 as of 2021-02-09 11:59:08 → ⇥
Size: 10767
Editor: HermannBrunner
Comment:
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:

Participants: Ian Stewart, Hermann Brunner, Miriam Ramos, Alain Gueguen, Konrad Dennerl, Mara Salvato, Sabine Osterhage, Susanne Friedrich, Jeremy Sanders. Chandeyee Maitra, Andrea Merloni, Michael Freyberg

Weekly eSASS developers meeting

Start of meeting: 2021-02-09 13:30 (1:30pm)

Zoom link: https://zoom.us/j/97439409844?pwd=YXVkK0hBMnpybm5tL2VyYTh2Qld2UT09

Participants: Ian Stewart, Hermann Brunner, Miriam Ramos, Alain Gueguen, Konrad Dennerl, Mara Salvato, Sabine Osterhage, Susanne Friedrich, Jeremy Sanders. Chandeyee Maitra, Andrea Merloni, Michael Freyberg

  • Open actions from previous developers meetings
    • Act20200929/1_all make sure html task descriptions are current and consistent with user release
      • updates by Jeremy and Ian (evtool, expmap, catprep)
      • still missing: Hermann (radec2xy), Georg (detection + evatt)
    • Act20201006/1_SF clean event file description, update source list description
  • Preprocessing / NRTA
    • Noise/offset maps?

  • eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page

    • eRASS1/947 code updates/code freeze (Ian's email)
    • expmap:
      • parallel version (7 cores)
      • second order parallelization (by GTI)
      • withinputmaps=yes
      • testing?
    • catprep: making number of bands variable (1..5) - in progress
    • detection task updates (ongoing, perhaps ready for 947)
      • clobber option
      • improved RADEC_ERR, upper/lower count rate errors
    • Pipeline tasks (shadow developer/pipeline operator)
      • eropipe, proc, prexec - Alain
      • telstage, dalist, rmvers - Miriam

      • test environment, current task versions -> eSASSdevel/task archive

    • Recent task updates:
      • expmap: v2.1.1-3, v2.2, v1.33.4
  • Pipeline / ongoing processing

    • processing wiki page

    • performance issues?
      • replacement for get_dir routine?
      • ds53 speed issues?
    • 947 test runs
      • eRASS1/2, eRASS:2 (first 100 eDays) are in archive

        • HiPS (eRASS1/2), DAtool, ProcStat (ok)

          • DAtool (947): ImageTot -> Image, energy bands: 0-9, a-d for e?01..., e?02..., S?02...

          • HiPS energy bands?
        • Astrometry?
    • eRASS3 (946) exposure deficit
      • caused (in part) by incomplete/not-current HK file versions - fixed
      • telgti issues at border of planning files (fixed in tel-1.24)
      • 46008 and 46085 rerun in eRASS3 946 processing - HiPS maps?
      • similar exposure deficits in eRASS1/2
    • Missing soft sources (Frank)
      • Ian, Chandreyee, Frank investigated
        • probable cause: previous processings contained CalPV test scans
        • confirmation: rerun test scans in 946 pipeline (or 001?)
    • bad pixels
      • TM7:
        • 42 bad pixels in caldb (onboard vs. caldb?)
        • How to proceed with eDay 46199?
    • Temperature dependence in task energy (HK access)
      • should go in 947 pipeline
      • Konrad proved list of eROdays to Hermann - TEL_949 chains currently running
    • Zip pattern analysis
      • updating badpixel / bti in caldb
    • Discrepancies in source list of demo script runs / ermldet uninitialized variables
      • Still different results of same binaries on different machines
      • Different from pervious problem, all sources are affected (parameter are sightly different) - high priority
      • Confirm whether ermldet patch changes the eFEDS source list (Teng?)
  • DAtool
    • access restrictions sto data request logs + project descriptions
    • providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
    • move simple copying request to Python / should everything be moved to Python? - ongoing
    • DAtool / eROSE interface: requires change in frontend
  • Longterm activities

    • [?] Moving task archive to git

    • evprep

      • [9] evprep: space invaders event flag

        • which flag location: bit 3, parameter in pipeline needs to be turned on
        • evtool should update the gti according to this flag
      • post-processing: frames with many counts?
    • BTI/GTI:
      • PDF for CalPV linked on wiki page (done?)

      • [7] one PDF per eRASS in wiki

    • ermldet

      • [8] ca. 2% of sources have positional error 0, some also have count error 0

        • can't be fixed quickly, may need replacement of algorithm
      • Johan's proposal? Georg to discuss with Johan (long timescale project)
    • Time dependent RMFs/ARFs (srctool?)

      • [8] RMF

      • [7] ARF

    • [6] Background analysis: projecting closed data onto the sky (XMM SAS) -> 6

    • [4] ersensmap: implement pointing PSF

    • ftfindhotpix: test and update algorithm

    • [1] Light leak in eSASS/pipeline?

    • [1] DAtool: filtering catalogs (separate from DAtool)? -> 1

    • DAtool
      • providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
      • move simple copying request to Python / should everything be moved to Python? - ongoing
      • DAtool / eROSE interface: requires change in frontend

Previous meeting:

  • eSASS Q&A

  • Open actions from previous developers meetings
    • Act20200929/1_all make sure html task descriptions are current and consistent with user release
      • updates by Jeremy and Ian (evtool, expmap, catprep)
      • still missing: Hermann (radec2xy), Georg (detection + evatt)
    • Act20201006/1_SF clean event file description, update source list description
  • Preprocessing / NRTA
    • Noise/offset maps?
    • TEL chain trigger files / HK file versions

  • eSASS tasks (ongoing/planned) / current processing errors - eSASSdevel page / Error tracking page

    • eRASS1 code updates/code freeze (Ian's email)

    • expmap: parallel version? Jeremy <--> Ian

      • still under investigation, patch for 947 may be needed
    • catprep: making number of bands variable (1..5) - in progress
    • detection task updates (ongoing, perhaps ready for 947)
      • clobber option
      • improved RADEC_ERR, upper/lower count rate errors
    • Pipeline tasks (shadow developer/pipeline operator): eropipe, proc, prexec, telstage, dalist, rmvers
    • Recent task updates:
      • evprep-3.18, evprep-3.11.3, erolib-2.39, tel-1.24
  • Pipeline / ongoing processing

    • processing wiki page

      • Processing artefacts page rearranged by Chandreyee
    • Computing resources for eSASS pipeline/ground segment vs. heg general
      • NFS server issues
      • Splitting, taring srctool output files, flux threshold
      • Locate command?
    • 947 test runs
      • eRASS1 (first 100 eDays) are in archive

        • HiPS (eRASS1/2?), ProcStat (ok), DAtool

          • DAtool (947): ImageTot -> Image, energy bands: 0-9, a-d

        • Astrometry?
    • eRASS3 (946) exposure deficit
      • caused (in part) by incomplete/not-current HK file versions?
      • telgti issues at border of planning files (fixed in tel-1.24)
      • 46008 and 46085 rerun in eRASS3 946 processing
      • similar exposure deficits in eRASS1/2
    • Missing soft sources (Frank)
      • Ian + Chandreyee will investigate
    • bad pixels in caldb (TM1/2/3/7)
      • TM7: bti for about two eROdays days, many (ca. 50) bad pixels (onboard+eSASS?)
    • Temperature dependence in task energy (HK access)
      • should go in 947 pipeline
      • Konrad will prove list of eROday to Hermann - HB will run eROdays in 947 pipeline!!
    • Zip pattern analysis
      • updating badpixel / bti in caldb
    • Discrepancies in source list of demo script runs / ermldet uninitialized variables
      • Still different results of same binaries on different machines
      • Different from pervious problem, all sources are affected (parameter are sightly different) - high priority
      • Confirm whether ermldet patch changes the eFEDS source list (Teng?)
    • pipeline (eropipe2) machine utilization
      • potential improvements
    • MIP threshold increase (get threshold from HK)
      • evprep-3.18/3.11.3
    • evprep summary reports
      • will be updated by Chandreyee to check TM5/7 against successive eROdays
    • HiPS / ds53 issues / processing speed
      • migrating to He4ero?
  • s/c clock drift / clock resets
  • Longterm activities

    • [?] Moving to git

    • expmap

      • [10] Parallel version

    • evprep

      • [9] evprep: space invaders event flag

        • which flag location: bit 3, parameter in pipeline needs to be turned on
        • evtool should update the gti according to this flag
      • post-processing: frames with many counts?
    • BTI/GTI:
      • PDF for CalPV linked on wiki page (done?)

      • [7] one PDF per eRASS in wiki

    • ermldet

      • [8] ca. 2% of sources have positional error 0, some also have count error 0

        • can't be fixed quickly, may need replacement of algorithm
      • Johan's proposal? Georg to discuss with Johan (long timescale project)
    • Time dependent RMFs/ARFs (srctool?)

      • [8] RMF

      • [7] ARF

    • [6] Background analysis: projecting closed data onto the sky (XMM SAS) -> 6

    • [4] ersensmap: implement pointing PSF

    • ftfindhotpix: test and update algorithm

    • [1] Light leak in eSASS/pipeline?

    • [1] DAtool: filtering catalogs (separate from DAtool)? -> 1

  • DAtool
    • providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine
    • move simple copying request to Python / should everything be moved to Python? - ongoing
    • DAtool / eROSE interface: requires change in frontend

Describe eSASSdevel_210208 here.

EROSITAwiki: eSASSdevel_210209 (last edited 2021-03-02 08:46:10 by HermannBrunner)