EROSITAwiki: eSASSdevel_210831

Weekly eSASS developers Zoom meeting

Start of meeting: 2021-08-31 13:30

Zoom link: https://zoom.us/j/97439409844?pwd=YXVkK0hBMnpybm5tL2VyYTh2Qld2UT09

Participants: Ian Stewart, Hermann Brunner, Georg Lamer, Susanne Friedrich, Michael Freyberg, Felix Schmidt, Teng Liu, Sabine Osterhage, Mara Salvato, Andrea Merloni

• Preprocessing / NRTA
• eFEDS catalog paper referee report
  • flux limit in abstract
  • Mara's comments
  • ermldet algorithm description duplication in chapter 3.3 and Appendix A.5 (where it should be)
  • Teng's optional text

  • US/UK spelling (catalog<->catalogue, color<->colour, center-> centre, neighbor -> neighbour)

• Pipeline / ongoing processing

  • processing wiki page

  • 947 100 eROday test
    • eRASS1 DET chains reprocessed (four sky tiles need to be reprocessed)
    • eRASS2 DET chains wer run but - needs to be redone because:
    • ero_archive3 full (now 3.8 TB free again)
      • declination zones: 039, 075, 087, 099, 111, 123, 135, 147, 159
    • other crashes?
    • differences DET_947/eFEDS detection
    • delivering eRASS1 merged catalog (Goerg)
    • delivering 947 products in DAtool
  • 946
    • EXP/DET chains of sky tiles on /ero_archive3 processed since Sunday need to be redone

• eSASS tasks eSASSdevel page / Error tracking page

  • evprep
    • new MIPs/dead time computation + MIPNEIGH cal component / testing

      • latest evprep version will need to go in both 946 & 947 to be able to read the new I*4 threshold maps

      • PMENV2 code needs to be corrected (after Konrad/Ian are back in October) - may not be needed for eRASS1? Only relevant once there are significant onboard bad pixels
    • catprep - new ermldet columns? - already included
  • evtool: TM filtering - already fixed
  • apetool/shapelib messages
  • from previous meetings:
    • ftfindhotpix / caldb

      • Konrad's proposal (discarding/renaming flags) - should not break backward compatibility -> after 947 is out

    • evprep
      • Konrad provided scaled threshold map
      • dedicated meeting of Konrad and Ian
      • Susanne should put new threshold maps in caldb
    • recent task updates (license file included+functional updates):
      • param-1.41
      • general-1.11
      • gtcal-1.14
      • erolib-2.64
      • catprep-2.8
      • onetools-1.10
      • erotools-1.6
      • evprep-3.41, 3.42, 3.43
      • evtool-2.26
      • expmap-2.9
      • ftfindhotpix-1.22
      • specfit-1.5
      • varicheck-1.9
      • ermldet-1.48, 1.49
      • eroio-2.67
  • DATool
    • gziped FITS files (Thomas Boller)
      • DAtool should follow links to srctool products and provide the files with the original names, consistent with the headers.
    • Jan's DATool question
• Longterm activities

  • [?] Moving task archive to git

  • evprep

    • [9] evprep: space invaders event flag

      • implemented - should be documented in next users release
      • more detailed, quantitative checks regarding improvement of data quality
      • evtool should update the gti according to this flag
        • wider issue: should discarded frames due to flag selection b tracked in GTI?
    • BTI/GTI:

      • [7] one PDF per eRASS in wiki

  • [7] PSF normalization

    • caldb vs. shaplet PSF (Antonis)

    • may be necessary to put 1 arcmin vs. 4 armin flux correction into shaplet caldb file (needs testing) -> longterm activity

    • ermldet

      • [3] Johan's proposal? Georg to discuss with Johan (long timescale project)

      • [7] Proposal by Florian: don't use fixed extrction radius but one dependent on flux

    • Time dependent RMFs/ARFs (srctool?)

      • [8] RMF

      • [7] ARF

    • [1] EV_WEIGHT: event based vignetting correction (would require dedicated tests)

    • [6] Background analysis: projecting closed data onto the sky (XMM SAS)

    • [6] ersensmap: implement pointing PSF

    • [3] ftfindhotpix: test and update algorithm

    • [1] Light leak in eSASS/pipeline?

    • [1] DAtool: filtering catalogs?

    • [6] add option to provide larger sky areas (Jeremy's script)

Previous meeting:

• Preprocessing / NRTA
• eFEDS catalog paper referee report
• Pipeline / ongoing processing

  • processing wiki page

  • 947 100 eROday test
    • update of eRASS1 DET_947 chains (apetool, forced photometry background, OWNER column)

    • eRASS1 products validation

      • timescales, start of full reprocessing mid September
  • 946
    • OWNER column fixed
    • no done files for Puppis A?

• eSASS tasks eSASSdevel page / Error tracking page

  • evprep - MIPS/dead times
    • evprep update - testing?
      • needs editing of TEL chain script!!
    • on ground MIP suppression (Ian's single frame images)
      • Michael will provide subroutine for on-ground MIP rejection, interface to be defined offline
  • cal messages
    • avoiding unnecessary caldb accesses
    • would removing obsolete caldb files help?
  • next user release / eSASSdevel (Ian's email)
    • task doc for next user release should be updated/checked
  • from previous meetings:
    • ermldet-1.47/apetool conflict - resolved
    • Check whether there are still keyword issues in recent survey data
      • TSTART/TSTOP
        • ok now (confirmed by Chandreyee)
      • FRAMETIM=50.0584ms
        • fix in eroio for now - fixed
    • PSF normalization
      • caldb vs. shaplet PSF (Antonis)

      • may be necessary to put 1 arcmin vs. 4 armin flux correction into shaplet caldb file (needs testing) -> longterm activity

    • ftfindhotpix / caldb

      • Konrad's proposal (discarding/renaming flags) - should not break backward compatibility -> after 947 is out

    • evprep
      • Konrad provided scaled threshold map
      • dedicated meeting of Konrad and Ian
      • Susanne should put new threshold maps in caldb
  • recent task updates:
    • eroio-2.65, 2.66
    • erbackmap-1.19, erbackmap-1.20
    • aptool-1.15
    • erolib-2.63
    • evtool-2.25
    • catprep-2.7
    • (param-1.40)
• DAtool
• Longterm activities

  • [?] Moving task archive to git

  • evprep

    • [9] evprep: space invaders event flag

      • implemented - should be documented in next users release
      • more detailed, quantitative checks regarding improvement of data quality
      • evtool should update the gti according to this flag
        • wider issue: should discarded frames due to flag selection b tracked in GTI?
    • BTI/GTI:

      • [7] one PDF per eRASS in wiki

  • ermldet
    • [3] Johan's proposal? Georg to discuss with Johan (long timescale project)
    • [7] Proposal by Florian: don't use fixed extrction radius but one dependent on flux
  • Time dependent RMFs/ARFs (srctool?)

    • [8] RMF

    • [7] ARF

  • [1] EV_WEIGHT: event based vignetting correction (would require dedicated tests)

  • [6] Background analysis: projecting closed data onto the sky (XMM SAS)

  • [6] ersensmap: implement pointing PSF

  • [3] ftfindhotpix: test and update algorithm

  • [1] Light leak in eSASS/pipeline?

  • [1] DAtool: filtering catalogs?

  • [6] add option to provide larger sky areas (Jeremy's script)

Previous meeting:

• Preprocessing / NRTA
• eFEDS referee report
• Pipeline / ongoing processing

  • processing wiki page

  • 947 100 eROday test
    • eRASS1
      • DET_947 upgrade
      • erbackmap - infinite loop?
      • evprep upgade (MIPs dead time calculation)
      • Hips maps

      • Products validation

  • 946
    • Miriam will retrigger DET chains which have wrong owenr ID in catalogs

• eSASS tasks eSASSdevel page / Error tracking page

  • evtool - time selection (GL)
    • would be good if srctool could do time selections
    • GTI should match region selections (region gti filter shoul be enbled
    • Georg's student should send emails to helpdesk
  • from previous meetings:
    • ermldet-1.47/apetool conflict (resolved?)
    • Check whether there are still keyword issues in recent survey data
      • TSTART/TSTOP still open (?)
        • Chandreyee will check once more in most latest 947 runs
      • FRAMETIM=50.0584ms
        • fix in eroio for now (email by Michael?)
    • detection vs sensitivity map

      • Frank <-> Georg (both are/will go on vacation soon)

        • may be due to chopper issues in CalPV
        • no further tests before Georg's vacation
        • closed - may have been special problem in LMC (chopper issues?)
    • PSF normalization
      • caldb vs. shaplet PSF (Antonis)
      • may be necessary to put 1 arcmin vs. 4 armin flux correction into shaplet caldb file (needs testing)
    • ftfindhotpix / caldb
      • Konrad's proposal (discarding/renaming flags) - should not break backward compatibility
    • evprep
      • Konrad provided scaled threshold map
      • dedicated meeting of Konrad and Ian
  • recent task updates:
    • eroio-2.63, 264
    • ftfindhotpix-1.21.1
    • eropipe2-1.22, 1.23
    • ermldet-1.48
• DAtool
  • testi[ng AO data access (when Alain is back)
• Longterm activities

  • [?] Moving task archive to git

  • evprep

    • [9] evprep: space invaders event flag

      • implemented - should be documented in next users release
      • more detailed, quantitative checks regarding improvement of data quality
      • evtool should update the gti according to this flag
        • wider issue: should discarded frames due to flag selection b tracked in GTI?
    • BTI/GTI:

      • [7] one PDF per eRASS in wiki

  • ermldet
    • [3] Johan's proposal? Georg to discuss with Johan (long timescale project)
    • [7] Proposal by Florian: don't use fixed extrction radius but one dependent on flux
  • Time dependent RMFs/ARFs (srctool?)

    • [8] RMF

    • [7] ARF

  • [1] EV_WEIGHT: event based vignetting correction (would require dedicated tests)

  • [6] Background analysis: projecting closed data onto the sky (XMM SAS)

  • [6] ersensmap: implement pointing PSF

  • [3] ftfindhotpix: test and update algorithm

  • [1] Light leak in eSASS/pipeline?

  • [1] DAtool: filtering catalogs?

  • [6] add option to provide larger sky areas (Jeremy's script)