|
Size: 1888
Comment:
|
Size: 2734
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 8: | Line 8: |
| df | df |
| Line 11: | Line 11: |
| == PACS1 - PACS5 == | == PACS1 - PACS6 (at present only for PACS3 & PACS4 & Pacs6) == |
| Line 16: | Line 16: |
| }}} if not up-to-date: {{{ cd software ./fetchpcss |
|
| Line 24: | Line 30: |
| if not: {{{ psetup TmIngestion [start|stop|status|log] }}} |
|
| Line 25: | Line 36: |
| === TC History and OOL Ingestion running ? === | === TC History and OOL Ingestion running ? (at present not because of deadlock) === |
| Line 38: | Line 50: |
| db2tty -d pacs_fm_ilt_1 | grep TmsourcePackets ssh -X oper@pacs1 psetup db2tty -d pacs_fm_ilt_1_prop | grep TmsourcePackets |
db2tty -d pacs_fm_ilt_3 herschel.versant.ccm.TmSourcePacketImpl | grep Total sleep 20 db2tty -d pacs_fm_ilt_3 herschel.versant.ccm.TmSourcePacketImpl | grep Total |
| Line 43: | Line 54: |
| Check for an increasing number of objects. |
|
| Line 46: | Line 59: |
| ssh -X oper@pacs5 | ssh -X dbsa@pacs5 |
| Line 49: | Line 62: |
| dbtool -space -volume -all pacs_fm_ilt_1 | dbtool -space -volume -all pacs_fm_ilt_3 |
| Line 51: | Line 64: |
if less than 20% free: {{{ oscp -i ls -als /home/db/versant/pacs_fm_ilt_3 (not during tests) addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_3/extraspace_ -s 2047M pacs_fm_ilt_3 }}} |
|
| Line 59: | Line 81: |
| == PACS1 == | == PACS5 == |
| Line 63: | Line 85: |
| ssh -X oper@pacs1 | ssh -X oper@pacs5 |
| Line 66: | Line 88: |
| dbtool -space -volume -all pacs_fm_ilt_1_prop | dbtool -space -volume -all pacs_fm_ilt_3_prop |
| Line 73: | Line 95: |
| E.g for 01-Mar-2007 : ls /dbBck/PacketFiles/*20070301* }}} restart if necessary: {{{ psetup PacketRecorder -host pacs-sv -daily /dbBck/PacketFiles/fm%.tm < /dev/null >& /dev/null & PacketRecorder -host pacs-sv -hourly /dbBck/PacketFiles/fm_%.tm < /dev/null > & /dev/null & |
|
| Line 78: | Line 108: |
| cat /home/jakob/TMMonitor.txt | more /home/jakob/FMMonitor.txt |
| Line 83: | Line 113: |
| ssh -X jakob@pacs1 | ssh jakob@pacs1 psetup |
| Line 85: | Line 116: |
| jylaunch writeHkNrt.py | ./startMonitor >& /dev/null & ps -elf | grep writeHk |
| Line 87: | Line 119: |
= Monday Morning checks = |
|
| Line 90: | Line 125: |
= Monday Morning checks = |
Every Morning checks
PACSSCOS
S2K, Event monitor and CDMS sim. diskspaces
ssh -X oper@pacsscos df
PACS1 - PACS6 (at present only for PACS3 & PACS4 & Pacs6)
Check PCSS version.
ssh -X pcss@pacsx cat software/pcss/RELEASE
if not up-to-date:
cd software ./fetchpcss
PACS 5
Is TmIngestion still running ?
ssh -X oper@pacs5 ps -elf | grep TmIngestion
if not:
psetup TmIngestion [start|stop|status|log]
TC History and OOL Ingestion running ? (at present not because of deadlock)
> ssh -X oper@pacs5 > ps -elf | grep thfser > ps -elf | grep tchi > ps -elf | grep orf > ps -elf | grep ool
Check TmSourcePackets arrive and get propagated
ssh -X oper@pacs5 psetup db2tty -d pacs_fm_ilt_3 herschel.versant.ccm.TmSourcePacketImpl | grep Total sleep 20 db2tty -d pacs_fm_ilt_3 herschel.versant.ccm.TmSourcePacketImpl | grep Total
Check for an increasing number of objects.
Check disk space and space of the operational Database
ssh -X dbsa@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_3
if less than 20% free:
oscp -i ls -als /home/db/versant/pacs_fm_ilt_3 (not during tests) addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_3/extraspace_ -s 2047M pacs_fm_ilt_3
Database Backup done ?
ssh -X dbsa@pacs5 ls /home/db/dbBck
PACS5
Check Disk space and space of the propagated database
ssh -X oper@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_3_prop
Check daily and hourly TmPacketRecorder files
ssh -X oper@pacs1 ps -elf | grep PacketRecorder E.g for 01-Mar-2007 : ls /dbBck/PacketFiles/*20070301*
restart if necessary:
psetup PacketRecorder -host pacs-sv -daily /dbBck/PacketFiles/fm%.tm < /dev/null >& /dev/null & PacketRecorder -host pacs-sv -hourly /dbBck/PacketFiles/fm_%.tm < /dev/null > & /dev/null &
Check whether Hardware support Monitor runs
ssh -X oper@pacs1 more /home/jakob/FMMonitor.txt
If not :
ssh jakob@pacs1 psetup cd monitor ./startMonitor >& /dev/null & ps -elf | grep writeHk
Monday Morning checks
PACS2
QLA running ?
PACS-SV
Check disk space
ssh oper@pacs-sv df -h .
To recover disk space try to delete "old" TM/TC packet files.
ssh oper@pacs-sv cd packetArchive find . -mtime +10 | xargs rm # to delete files older than 10 days # or find . -mtime +7| xargs rm # to delete files older than one week