#acl EroCatGroup:read,write,revert All:read == eSASS developers Zoom meeting == '''Start of meeting: 2020-12-08 13:30 (1:30pm)''' Zoom link: https://zoom.us/j/7909307159?pwd=ZEdGeTJQUzdsRGpkd2Q5aUZXQmZZdz09 Participants: Ian Stewart, Hermann Brunner, Ingo Kreykenbohm, Miriam Ramos, Chandreyee Maitra, Alain Gueguen, Konrad Dennerl, Mara Salvato, Sabine Osterhage, Susanne Friedrich, Teng Liu * Open actions * Act20200929/1_all make sure html task descriptions are current and consistent with user release * Act20201006/1_SF clean event file description, update source list description * Miriam will provide HTML style input * Act20201102/1 AG+HB: test how many regions per request DATool can handle - done * warning in frontend: SO+HB - done * Act20201117/1_GL: check for out of sync attitude in eRASS (946) and CalPV data (001) - done * c946 reruns? Georg will investigate to which extent this effects the data quality * List of out of order attitude will be placed in wiki / should be linked from Andreas CalPV page * Chandeyee has script to convert s/c clock into ObsIDs (using planning file) * GL will provde attprep attprep patch, HB will rerun RXJ1656 obs + eROday 45648 (eRASS2) * NRTA / preproc * out of order records / bad FRAMETIME values since ITC reset? * eSASS tasks (ongoing/planned) / current processing errors - [[https://wiki.mpe.mpg.de/eRosita/EroCat/eSASSdevel|eSASSdevel page]] / [[https://wiki.mpe.mpg.de/eRosita/pipelineErrorTracking|Error tracking page]] * catprep: set output and output2 (947 pipeline) * detection task updates (ongoing, perhaps ready for 947) * clobber option * improved RADEC_ERR * upper/lower count rate errors * ermldet: ca. 2% of sources have positional error 0, some also have count error 0 * Johan's proposal? * library order in .libs files * Antonis, Georg, and Jeremy should change .libs order in their tasks * tasks in libs directory should be in correct order - we don't understand why in some cases it also works with the wrong oder (or if libs are not included, e.g. srctool) * NaNs in physical DS9 coordinates in exposure maps, missing keyword (see Ian's emails), eroio nees to be updated * Recent task uploads * Ongoing processing * [[https://wiki.mpe.mpg.de/eRosita/PipeProc|processing issues wiki page]] * exposure map issues (may be related to data dump procedure w/o DRAM memory use): * single FoV depressions in survey exposure * long stripes with reduced exposure * bad FRAMETIME values (Ian's plots) * Ian will write up current analysis, to be discussed at next oepration meeting * backup issues (discussion Harald/Achim/computer center - ongoing) * 947 pipeline (eROproc_201125): TEL_947 chain of first 100 eRASS1 eROdays run * eRASS2 * c946 ongoing, no hold files, no held eROdays * SOU chains restarted, was stopped since weekend * evprep summary reports * test sky tile updates every other day * evprep summary reports * CalPV / c001 * all CalPV ObsIDs processed * Fixing header keywords - fixed by Chandreyee * User release: discrepancies in source list of demo script runs (ermldet) * Different results on different machines, happens in multi source fits * ermldet fix by Georg: update for 947 pipeline and users release? * Should Teng confirm whether ermldet patch changes the eFEDS source list? * patch user release / update Docker version? Will be fixed for Georg's/Teng's CALPV papers * HiPS maps * update frequency, RGB maps in optimized bands (for 947 pipeline) * RXJ 1856 flux discrepancies * out of sync values in gyro attitude - will be fixed in attprep * attprep should discard affected entries, and report about number of occurances * fixing c001/c946 processing? * flaregti testing * [[https://wiki.mpe.mpg.de/eRosita/flaregti_testing|Jeremy's analysis]] * heasoft 6.28 (was 6.27.2) / user release * CalPV data release / public software release * Updating documentation /new Web pages * [[https://erosita.mpe.mpg.de/index_new.html]] (password required) * password access for internal pages? * DAtool * Frank's email / accessing RXJ2143 observation did not work * providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine * move simple copying request to Python / should everything be moved to Python? - ongoing * DAtool / eROSE interface: requires change in frontend * Other issues * first request by a user occasionally doesn't work * no data retrieved if multiple entries per ObsID in planning file (RXJ 1856) * Longterm activities * ersensmap: implement pointing PSF * expmap * Bring coordinate transformations into line with evatt * Parallel version * evprep: experimental event flags * post-processing: space invaders, frames with many counts, statistical event flagging * DAtool: filtering catalogs? * Light leak mitigation * Time dependent RMFs/ARFs (srctool?) * Background analysis: projecting closed data onto the sky (XMM SAS) * Moving to git ---- '''''Previous meeting:''''' * Open actions * Act20200929/1_all (Ian, Georg, Jeremy, Antonis, Miriam and Hermann will provide recomendations): make sure html task descriptions are current and consistent with user release * task documention for eSASSuser + CalPV/001 should be sent be email, will be integrated in Web page by Miriam * Act20201006/1_HB clean event file description, update source list description * Act20201006/2_SF document cal files (done) and bti * static version on Web page / live version in wiki * Act20201020/1_HB+SF check why some ObsIDs were only processed in 945 * telgti-1.23 running in 001 pipeline * processed: 700203, 700204, 700040 * still missing: 700201, 700202, 900046, 900047, 700015 (c945) * Act20201102/1 AG+HB: test how many regions per request DATool can handle - done * limit in the frontend: 150 regions (can be changed) / also: 3h time limit * backend: probably only time limit (150 regions take more than 10 hours) * warning in frontend: SO+HB * Act20201117/1_GL: check for out of sync attitude in eRASS (946) and CalPV data (001) - done * c946 reruns? Georg will investigate ro which extnt this effects the data * New actions * NRTA / preproc * DIMM memory error * offset /w noise maps CTIMETAG overflow, computed full time in keyword CALCTIME * raw frames April 1st, 2020 * Ongoing processing * [[https://wiki.mpe.mpg.de/eRosita/PipeProc|processing issues wiki page]] * Jira discussion? * backup issues (discussion Harald/Achim/computer center - ongoing) * excluding srtool products from backup? Not running SOU chain every day? Will test every other day * turn on optimization in 947 pipeline * eRASS2 * c946 ongoing * eROproc_201125 for 947 (task versions) * TM1-7 hold files (ITC reset) - removed * CalPV / c001 * missing ObsIDs processed with telgti-1.23, some successes, some still missing * 2020 pointings processed * RXJ 1856 flux discrepancies (also RXJ2143??) * out of sync values in gyro attitude - will be fixed in attprep * attprep should discard affected entries, and report about number of occurances * fixing c001/c946 processing? * flaregti in eFEDS field (Teng/Jeremy) * parameters need to be adjusted to field size, should be in documentation, flaregti could also read the field size from input, Teng is using changed parameters for eFEDS * Jeremy is investigating flaregti behaviour in survey; parameters for 947 * simulating random light curves - ongoing * testing whether flaregti improve detection - use current settings for c947 for now / use harder band * Discrepancies in source list of demo script runs (ermldet) * Different results results on different machines using same binaries, probably due to uninitialized variables * results are consistent on the same machine * seems to happen in multi source fits * ermldet fix by Georg: update for 947 pipeline and users release? * Should Teng confirm whether ermldet patch changes the eFEDS source list? * patch user release / update Docker version? Will be fixed for Georg's/Teng's CALPV papers * heasoft 6.28 (was 6.27.2) / user release * cron job / email notifications for evprep anomalies - current sttings seem to be ok * eSASS tasks (ongoing/planned) / current processing errors - [[https://wiki.mpe.mpg.de/eRosita/EroCat/eSASSdevel|eSASSdevel page]] / [[https://wiki.mpe.mpg.de/eRosita/pipelineErrorTracking|Error tracking page]] * catprep: set output and output2 (947 pipeline) * telgti: attitude issues * sky tile 062093 (events outside of sky tile) -> eSASS error tracking page * detection task updates (ongoing, perhaps ready for 947) * clobber option * improved RADEC_ERR * upper/lower count rate errors * ermldet: ca. 2% of sources have positional error 0, some also have count error 0 * Johan's proposal? * library order in .libs files * Antonis, Gerog, and Jeremy should change .libs order in their tasks * NaNs in physical DS9 coordinates in exposure maps, missing keyword (see Ian's emails), eroio nees to be updated * Recent task uploads * erolib-2.33.5, erolib-2.34 * general-1.8 * eropipe-1.19, dalist-1.4, rmvers-1.4 * expmap-1.35, expmap-1.35.1, expmap-1.36 * CalPV data release / public software release * Following up on meeting last week * Updating documentation * Task descriptions / default values * Wiki documentation * Public Web pages * Susanne's caldb description * DAtool * Frank's email / accessing RXJ2143 observation did not work * Region requests issues * owner filtering / evtool: use MPE owner eventflag * Were all of Georg's bug reports fixed? * providing large multi-sky tile images - Jeremy will share his Python script with Alain and Sabine * move simple copying request to Python / should everything be moved to Python? - ongoing * DAtool / eROSE interface: requires change in fronted * Frontend problem: first request by a user occasionally doesn't work * Longterm activities * ersensmap: implement pointing PSF * expmap * Bring coordinate transformations into line with evatt * Parallel version * evprep: experimental event flags * post-processing: space invaders, frames with many counts, statistical event flagging * DAtool: filtering catalogs? * Light leak mitigation * Time dependent RMFs/ARFs (srctool?) * Background analysis: projecting closed data onto the sky (XMM SAS) * Moving to git