|
Size: 25
Comment:
|
Size: 2380
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Every Morning check = | = Every Morning checks = == PACSSCOS == S2K, Event monitor and CDMS sim. diskspaces {{{ ssh -X oper@pacsscos df }}} == PACS1 - PACS5 (at present only for PACS3 & PACS4) == Check PCSS version. {{{ ssh -X pcss@pacsx cat software/pcss/RELEASE }}} if not up-to-date: {{{ cd software ./fetchpcss }}} == PACS 5 == === Is TmIngestion still running ? === {{{ ssh -X oper@pacs5 ps -elf | grep TmIngestion }}} === TC History and OOL Ingestion running ? (at present not because of deadlock) === {{{ > ssh -X oper@pacs5 > ps -elf | grep thfser > ps -elf | grep tchi > ps -elf | grep orf > ps -elf | grep ool }}} === Check TmSourcePackets arrive and get propagated === {{{ ssh -X oper@pacs5 psetup db2tty -d pacs_fm_ilt_2 herschel.versant.ccm.TmSourcePacketImpl | grep Total sleep 20 db2tty -d pacs_fm_ilt_2 herschel.versant.ccm.TmSourcePacketImpl | grep Total }}} Check for an increasing number of objects. === Check disk space and space of the operational Database === {{{ ssh -X dbsa@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_2 }}} if less than 20% free: {{{ oscp -i ls -als /home/db/versant/pacs_fm_ilt_2 (not during tests) addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_2/extraspace_ -s 2047M pacs_fm_ilt_2 }}} === Database Backup done ? === {{{ ssh -X dbsa@pacs5 ls /home/db/dbBck }}} == PACS5 == === Check Disk space and space of the propagated database (at present not) === {{{ ssh -X oper@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_1_prop }}} == Check daily and hourly TmPacketRecorder files == {{{ ssh -X oper@pacs1 ps -elf | grep PacketRecorder }}} == Check whether Hardware support Monitor runs == {{{ ssh -X oper@pacs1 cat /home/jakob/FMMonitor.txt }}} If not : {{{ ssh jakob@pacs1 psetup cd monitor ./startMonitor >& /dev/null & ps -elf | grep writeHk }}} = Monday Morning checks = == PACS2 == === QLA running ? === = PACS-SV = == Check disk space == {{{ ssh oper@pacs-sv df -h . }}} To recover disk space try to delete "old" TM/TC packet files. {{{ ssh oper@pacs-sv cd packetArchive find . -mtime +10 | xargs rm # to delete files older than 10 days # or find . -mtime +7| xargs rm # to delete files older than one week }}} |
Every Morning checks
PACSSCOS
S2K, Event monitor and CDMS sim. diskspaces
ssh -X oper@pacsscos df
PACS1 - PACS5 (at present only for PACS3 & PACS4)
Check PCSS version.
ssh -X pcss@pacsx cat software/pcss/RELEASE
if not up-to-date:
cd software ./fetchpcss
PACS 5
Is TmIngestion still running ?
ssh -X oper@pacs5 ps -elf | grep TmIngestion
TC History and OOL Ingestion running ? (at present not because of deadlock)
> ssh -X oper@pacs5 > ps -elf | grep thfser > ps -elf | grep tchi > ps -elf | grep orf > ps -elf | grep ool
Check TmSourcePackets arrive and get propagated
ssh -X oper@pacs5 psetup db2tty -d pacs_fm_ilt_2 herschel.versant.ccm.TmSourcePacketImpl | grep Total sleep 20 db2tty -d pacs_fm_ilt_2 herschel.versant.ccm.TmSourcePacketImpl | grep Total
Check for an increasing number of objects.
Check disk space and space of the operational Database
ssh -X dbsa@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_2
if less than 20% free:
oscp -i ls -als /home/db/versant/pacs_fm_ilt_2 (not during tests) addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_2/extraspace_ -s 2047M pacs_fm_ilt_2
Database Backup done ?
ssh -X dbsa@pacs5 ls /home/db/dbBck
PACS5
Check Disk space and space of the propagated database (at present not)
ssh -X oper@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_1_prop
Check daily and hourly TmPacketRecorder files
ssh -X oper@pacs1 ps -elf | grep PacketRecorder
Check whether Hardware support Monitor runs
ssh -X oper@pacs1 cat /home/jakob/FMMonitor.txt
If not :
ssh jakob@pacs1 psetup cd monitor ./startMonitor >& /dev/null & ps -elf | grep writeHk
Monday Morning checks
PACS2
QLA running ?
PACS-SV
Check disk space
ssh oper@pacs-sv df -h .
To recover disk space try to delete "old" TM/TC packet files.
ssh oper@pacs-sv cd packetArchive find . -mtime +10 | xargs rm # to delete files older than 10 days # or find . -mtime +7| xargs rm # to delete files older than one week