Differences between revisions 1 and 14 (spanning 13 versions)
Revision 1 as of 2021-09-07 09:23:24
Size: 34
Comment:
Revision 14 as of 2021-09-08 09:29:26
Size: 8110
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
Describe eSASSdevel_210907 here. #acl EroCatGroup:read,write,revert All:read

== Monthly eSASS Q&A / weekly eSASS developers Zoom meeting ==

'''Start of meeting: 2021-09-07 13:30'''

Zoom link: https://zoom.us/j/97439409844?pwd=YXVkK0hBMnpybm5tL2VyYTh2Qld2UT09

 * Preprocessing / NRTA

 * eFEDS catalog paper referee report resubmitted
   * unrecognized TM6 time shift in eFEDS sub-field I
   * Report on EDR page, upate data

 * Pipeline / ongoing processing
   * [[https://wiki.mpe.mpg.de/eRosita/PipeProc|processing wiki page]]

   * 947 100 eROday test
     * eROproc_201125 is now current
     * task updates before restarting DET_947 chains?
     * third erbackmap run still needs to be included in DET_947
     * Rerunning TEL_947
     * Validation: Georg's merged catalogs / comparison with 946
     * Starting TEL_947 all sky?

   * 946

 * eSASS tasks [[https://wiki.mpe.mpg.de/eRosita//eSASSdevel|eSASSdevel page]] / [[https://wiki.mpe.mpg.de/eRosita/pipelineErrorTracking|Error tracking page]]

   * ermldet
     * Mara: RA/DEC/RADEC_ERR contained -999 (ermlet-1.47?) Julien will report details

   * from previous meetings:

     * ftfindhotpix / caldb
       * Konrad's proposal (discarding/renaming flags) - should not break backward compatibility -> after 947 is out

     * evprep
       * Konrad provided scaled threshold map
       * dedicated meeting of Konrad and Ian
       * Susanne should put new threshold maps in
       * will only affect data after 47036 (is in eRASS4) - not need for eRASS:3 reprocessing
       * Update needed to address issue with bad pixel neighbours - could be implemented before eRASS3 947 reprocessing (see previous meeting)

   * License files (once again)
     * Correct name of license file?

   * New user release
     * Test version by Ian
     * Reporting version number in user release (method by Philipp Weber)
     
   * recent task updates (license file included+functional updates):
     * catprep-2.8.1, 2.8.2
     * apetool- 1.17, 1.18
     * erolib-2.65
     * evtool-2.26.1
     * expmap-2.9.1
     * gtcal-1.14.1
     * radec2xy-1.8
     * evatt-1.13
     * astrolib-2.7.tgz
     * cal-1.18
     * flaregit-1.25
     * srctool-1.68
     * eroio-2.68
     * ermldet-1.50

 * DATool
   * 947 products
   * Delivering products with version numbers in file names
   * request log, etc.

 * Longterm activities
    * '''[?]''' Moving task archive to git
    * evprep
      * '''[9]''' evprep: space invaders event flag
          * implemented - should be documented in next users release
          * more detailed, quantitative checks regarding improvement of data quality
        * evtool should update the gti according to this flag
          * wider issue: should discarded frames due to flag selection b tracked in GTI?
      * BTI/GTI:
        * '''[7]''' one PDF per eRASS in wiki
   * '''[7]''' PSF normalization
      * caldb vs. shaplet PSF (Antonis)
      * may be necessary to put 1 arcmin vs. 4 armin flux correction into shaplet caldb file (needs testing) -> longterm activity
    * ermldet
        * '''[3]''' Johan's proposal? Georg to discuss with Johan (long timescale project)
        * '''[7]''' Proposal by Florian: don't use fixed extrction radius but one dependent on flux
    * Time dependent RMFs/ARFs (srctool?)
      * '''[8]''' RMF
      * '''[7]''' ARF
    * '''[1]''' EV_WEIGHT: event based vignetting correction (would require dedicated tests)
    * '''[6]''' Background analysis: projecting closed data onto the sky (XMM SAS)
    * '''[6]''' ersensmap: implement pointing PSF
    * '''[3]''' ftfindhotpix: test and update algorithm
    * '''[1]''' Light leak in eSASS/pipeline?
    * '''[1]''' DAtool: filtering catalogs?
    * '''[6]''' add option to provide larger sky areas (Jeremy's script)
 
   
----

'''''Previous meeting:'''''

 * Preprocessing / NRTA

 * eFEDS catalog paper referee report
   * flux limit in abstract
   * Mara's comments
   * ermldet algorithm description duplication in chapter 3.3 and Appendix A.5 (where it should be)
   * Teng's optional text
   * US/UK spelling (catalog<->catalogue, color<->colour, center-> centre, neighbor -> neighbour)

 * Pipeline / ongoing processing
   * [[https://wiki.mpe.mpg.de/eRosita/PipeProc|processing wiki page]]
  
   * 947 100 eROday test
     * eRASS1 DET chains reprocessed (four sky tiles need to be reprocessed)
     * eRASS2 DET chains wer run but - needs to be redone because:
     * ero_archive3 full (now 3.8 TB free again)
       * declination zones: 039, 075, 087, 099, 111, 123, 135, 147, 159
     * other crashes?
     * differences DET_947/eFEDS detection
     * delivering eRASS1 merged catalog (Goerg)
     * delivering 947 products in DAtool

   * 946
     * EXP/DET chains of sky tiles on /ero_archive3 processed since Sunday need to be redone

 * eSASS tasks [[https://wiki.mpe.mpg.de/eRosita//eSASSdevel|eSASSdevel page]] / [[https://wiki.mpe.mpg.de/eRosita/pipelineErrorTracking|Error tracking page]]

   * evprep
     * new MIPs/dead time computation + MIPNEIGH cal component / testing
       * latest evprep version will need to go in both 946 & 947 to be able to read the new I*4 threshold maps
       * PMENV2 code needs to be corrected (after Konrad/Ian are back in October) - may not be needed for eRASS1? Only relevant once there are significant onboard bad pixels
     * catprep - new ermldet columns? - already included

   * evtool: TM filtering - already fixed

   * apetool/shapelib messages

   * from previous meetings:

     * ftfindhotpix / caldb
       * Konrad's proposal (discarding/renaming flags) - should not break backward compatibility -> after 947 is out

     * evprep
       * Konrad provided scaled threshold map
       * dedicated meeting of Konrad and Ian
       * Susanne should put new threshold maps in caldb

    * recent task updates (license file included+functional updates):
      * param-1.41
      * general-1.11
      * gtcal-1.14
      * erolib-2.64
      * catprep-2.8
      * onetools-1.10
      * erotools-1.6
      * evprep-3.41, 3.42, 3.43
      * evtool-2.26
      * expmap-2.9
      * ftfindhotpix-1.22
      * specfit-1.5
      * varicheck-1.9
      * ermldet-1.48, 1.49
      * eroio-2.67
 
  * DATool
    * gziped FITS files (Thomas Boller)
      * DAtool should follow links to srctool products and provide the files with the original names, consistent with the headers.
    * Jan's DATool question
 
 * Longterm activities
    * '''[?]''' Moving task archive to git
    * evprep
      * '''[9]''' evprep: space invaders event flag
          * implemented - should be documented in next users release
          * more detailed, quantitative checks regarding improvement of data quality
        * evtool should update the gti according to this flag
          * wider issue: should discarded frames due to flag selection b tracked in GTI?
      * BTI/GTI:
        * '''[7]''' one PDF per eRASS in wiki
   * '''[7]''' PSF normalization
      * caldb vs. shaplet PSF (Antonis)
      * may be necessary to put 1 arcmin vs. 4 armin flux correction into shaplet caldb file (needs testing) -> longterm activity
    * ermldet
        * '''[3]''' Johan's proposal? Georg to discuss with Johan (long timescale project)
        * '''[7]''' Proposal by Florian: don't use fixed extrction radius but one dependent on flux
    * Time dependent RMFs/ARFs (srctool?)
      * '''[8]''' RMF
      * '''[7]''' ARF
    * '''[1]''' EV_WEIGHT: event based vignetting correction (would require dedicated tests)
    * '''[6]''' Background analysis: projecting closed data onto the sky (XMM SAS)
    * '''[6]''' ersensmap: implement pointing PSF
    * '''[3]''' ftfindhotpix: test and update algorithm
    * '''[1]''' Light leak in eSASS/pipeline?
    * '''[1]''' DAtool: filtering catalogs?
    * '''[6]''' add option to provide larger sky areas (Jeremy's script)

Monthly eSASS Q&A / weekly eSASS developers Zoom meeting

Start of meeting: 2021-09-07 13:30

Zoom link: https://zoom.us/j/97439409844?pwd=YXVkK0hBMnpybm5tL2VyYTh2Qld2UT09

  • Preprocessing / NRTA
  • eFEDS catalog paper referee report resubmitted
    • unrecognized TM6 time shift in eFEDS sub-field I
    • Report on EDR page, upate data
  • Pipeline / ongoing processing
    • processing wiki page

    • 947 100 eROday test
      • eROproc_201125 is now current
      • task updates before restarting DET_947 chains?
      • third erbackmap run still needs to be included in DET_947
      • Rerunning TEL_947
      • Validation: Georg's merged catalogs / comparison with 946
      • Starting TEL_947 all sky?
    • 946
  • eSASS tasks eSASSdevel page / Error tracking page

    • ermldet
      • Mara: RA/DEC/RADEC_ERR contained -999 (ermlet-1.47?) Julien will report details
    • from previous meetings:
      • ftfindhotpix / caldb
        • Konrad's proposal (discarding/renaming flags) - should not break backward compatibility -> after 947 is out

      • evprep
        • Konrad provided scaled threshold map
        • dedicated meeting of Konrad and Ian
        • Susanne should put new threshold maps in
        • will only affect data after 47036 (is in eRASS4) - not need for eRASS:3 reprocessing
        • Update needed to address issue with bad pixel neighbours - could be implemented before eRASS3 947 reprocessing (see previous meeting)
    • License files (once again)
      • Correct name of license file?
    • New user release
      • Test version by Ian
      • Reporting version number in user release (method by Philipp Weber)
    • recent task updates (license file included+functional updates):
      • catprep-2.8.1, 2.8.2
      • apetool- 1.17, 1.18
      • erolib-2.65
      • evtool-2.26.1
      • expmap-2.9.1
      • gtcal-1.14.1
      • radec2xy-1.8
      • evatt-1.13
      • astrolib-2.7.tgz
      • cal-1.18
      • flaregit-1.25
      • srctool-1.68
      • eroio-2.68
      • ermldet-1.50
  • DATool
    • 947 products
    • Delivering products with version numbers in file names
    • request log, etc.
  • Longterm activities
    • [?] Moving task archive to git

    • evprep
      • [9] evprep: space invaders event flag

        • implemented - should be documented in next users release
        • more detailed, quantitative checks regarding improvement of data quality
        • evtool should update the gti according to this flag
          • wider issue: should discarded frames due to flag selection b tracked in GTI?
      • BTI/GTI:
        • [7] one PDF per eRASS in wiki

    • [7] PSF normalization

      • caldb vs. shaplet PSF (Antonis)
      • may be necessary to put 1 arcmin vs. 4 armin flux correction into shaplet caldb file (needs testing) -> longterm activity

      • ermldet
        • [3] Johan's proposal? Georg to discuss with Johan (long timescale project)

        • [7] Proposal by Florian: don't use fixed extrction radius but one dependent on flux

      • Time dependent RMFs/ARFs (srctool?)
        • [8] RMF

        • [7] ARF

      • [1] EV_WEIGHT: event based vignetting correction (would require dedicated tests)

      • [6] Background analysis: projecting closed data onto the sky (XMM SAS)

      • [6] ersensmap: implement pointing PSF

      • [3] ftfindhotpix: test and update algorithm

      • [1] Light leak in eSASS/pipeline?

      • [1] DAtool: filtering catalogs?

      • [6] add option to provide larger sky areas (Jeremy's script)


Previous meeting:

  • Preprocessing / NRTA
  • eFEDS catalog paper referee report
    • flux limit in abstract
    • Mara's comments
    • ermldet algorithm description duplication in chapter 3.3 and Appendix A.5 (where it should be)
    • Teng's optional text
    • US/UK spelling (catalog<->catalogue, color<->colour, center-> centre, neighbor -> neighbour)

  • Pipeline / ongoing processing
    • processing wiki page

    • 947 100 eROday test
      • eRASS1 DET chains reprocessed (four sky tiles need to be reprocessed)
      • eRASS2 DET chains wer run but - needs to be redone because:
      • ero_archive3 full (now 3.8 TB free again)
        • declination zones: 039, 075, 087, 099, 111, 123, 135, 147, 159
      • other crashes?
      • differences DET_947/eFEDS detection
      • delivering eRASS1 merged catalog (Goerg)
      • delivering 947 products in DAtool
    • 946
      • EXP/DET chains of sky tiles on /ero_archive3 processed since Sunday need to be redone
  • eSASS tasks eSASSdevel page / Error tracking page

    • evprep
      • new MIPs/dead time computation + MIPNEIGH cal component / testing
        • latest evprep version will need to go in both 946 & 947 to be able to read the new I*4 threshold maps

        • PMENV2 code needs to be corrected (after Konrad/Ian are back in October) - may not be needed for eRASS1? Only relevant once there are significant onboard bad pixels
      • catprep - new ermldet columns? - already included
    • evtool: TM filtering - already fixed
    • apetool/shapelib messages
    • from previous meetings:
      • ftfindhotpix / caldb
        • Konrad's proposal (discarding/renaming flags) - should not break backward compatibility -> after 947 is out

      • evprep
        • Konrad provided scaled threshold map
        • dedicated meeting of Konrad and Ian
        • Susanne should put new threshold maps in caldb
      • recent task updates (license file included+functional updates):
        • param-1.41
        • general-1.11
        • gtcal-1.14
        • erolib-2.64
        • catprep-2.8
        • onetools-1.10
        • erotools-1.6
        • evprep-3.41, 3.42, 3.43
        • evtool-2.26
        • expmap-2.9
        • ftfindhotpix-1.22
        • specfit-1.5
        • varicheck-1.9
        • ermldet-1.48, 1.49
        • eroio-2.67
    • DATool
      • gziped FITS files (Thomas Boller)
        • DAtool should follow links to srctool products and provide the files with the original names, consistent with the headers.
      • Jan's DATool question
  • Longterm activities
    • [?] Moving task archive to git

    • evprep
      • [9] evprep: space invaders event flag

        • implemented - should be documented in next users release
        • more detailed, quantitative checks regarding improvement of data quality
        • evtool should update the gti according to this flag
          • wider issue: should discarded frames due to flag selection b tracked in GTI?
      • BTI/GTI:
        • [7] one PDF per eRASS in wiki

    • [7] PSF normalization

      • caldb vs. shaplet PSF (Antonis)
      • may be necessary to put 1 arcmin vs. 4 armin flux correction into shaplet caldb file (needs testing) -> longterm activity

      • ermldet
        • [3] Johan's proposal? Georg to discuss with Johan (long timescale project)

        • [7] Proposal by Florian: don't use fixed extrction radius but one dependent on flux

      • Time dependent RMFs/ARFs (srctool?)
        • [8] RMF

        • [7] ARF

      • [1] EV_WEIGHT: event based vignetting correction (would require dedicated tests)

      • [6] Background analysis: projecting closed data onto the sky (XMM SAS)

      • [6] ersensmap: implement pointing PSF

      • [3] ftfindhotpix: test and update algorithm

      • [1] Light leak in eSASS/pipeline?

      • [1] DAtool: filtering catalogs?

      • [6] add option to provide larger sky areas (Jeremy's script)

EROSITAwiki: eSASSdevel_210907 (last edited 2021-09-08 10:32:38 by HermannBrunner)