|
Size: 2297
Comment:
|
Size: 2673
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 8: | Line 8: |
| df | df |
| Line 11: | Line 11: |
| == PACS1 - PACS5 (at present only for PACS3 & PACS4) == | == PACS1 - PACS6 (at present only for PACS3 & PACS4 & Pacs6) == |
| Line 18: | Line 18: |
| if not actuell: | if not up-to-date: |
| Line 30: | Line 30: |
| if not: {{{ psetup TmIngestion [start|stop|status|log] }}} |
|
| Line 31: | Line 36: |
| === TC History and OOL Ingestion running ? === | === TC History and OOL Ingestion running ? (at present not because of deadlock) === |
| Line 44: | Line 50: |
| db2tty -d pacs_fm_ilt_2 | grep TmsourcePackets ssh -X oper@pacs1 psetup db2tty -d pacs_fm_ilt_2_prop | grep TmsourcePackets |
db2tty -d pacs_fm_ilt_3 herschel.versant.ccm.TmSourcePacketImpl | grep Total sleep 20 db2tty -d pacs_fm_ilt_3 herschel.versant.ccm.TmSourcePacketImpl | grep Total |
| Line 49: | Line 54: |
| Check for an increasing number of objects. |
|
| Line 55: | Line 62: |
| dbtool -space -volume -all pacs_fm_ilt_2 | dbtool -space -volume -all pacs_fm_ilt_3 |
| Line 61: | Line 68: |
| ls -als /home/db/versant/pacs_fm_ilt_2 | ls -als /home/db/versant/pacs_fm_ilt_3 |
| Line 63: | Line 70: |
| addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_2/extraspace_ -s 2047M pacs_fm_ilt_2 preferable add 4x 2 GB - 2 GB quicker acess - 8 GB should last one week total of DB should not be more than 20 GB |
addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_3/extraspace_ -s 2047M pacs_fm_ilt_3 |
| Line 77: | Line 81: |
| == PACS1 == | == PACS5 == |
| Line 81: | Line 85: |
| ssh -X oper@pacs1 | ssh -X oper@pacs5 |
| Line 84: | Line 88: |
| dbtool -space -volume -all pacs_fm_ilt_1_prop | dbtool -space -volume -all pacs_fm_ilt_3_prop |
| Line 92: | Line 96: |
| restart if necessary: {{{ psetup PacketRecorder -host pacs-sv -daily /dbBck/PacketFiles/fm%.tm < /dev/null >& /dev/null & PacketRecorder -host pacs-sv -hourly /dbBck/PacketFiles/fm_%.tm < /dev/null > & /dev/null & }}} |
|
| Line 96: | Line 106: |
| cat /home/jakob/FMMonitor.txt | more /home/jakob/FMMonitor.txt |
| Line 101: | Line 111: |
| ssh -X jakob@pacs1 | ssh jakob@pacs1 psetup |
| Line 103: | Line 114: |
| jylaunch writeHkNrt.py | ./startMonitor >& /dev/null & ps -elf | grep writeHk |
| Line 105: | Line 117: |
= Monday Morning checks = |
|
| Line 108: | Line 123: |
= Monday Morning checks = |
Every Morning checks
PACSSCOS
S2K, Event monitor and CDMS sim. diskspaces
ssh -X oper@pacsscos df
PACS1 - PACS6 (at present only for PACS3 & PACS4 & Pacs6)
Check PCSS version.
ssh -X pcss@pacsx cat software/pcss/RELEASE
if not up-to-date:
cd software ./fetchpcss
PACS 5
Is TmIngestion still running ?
ssh -X oper@pacs5 ps -elf | grep TmIngestion
if not:
psetup TmIngestion [start|stop|status|log]
TC History and OOL Ingestion running ? (at present not because of deadlock)
> ssh -X oper@pacs5 > ps -elf | grep thfser > ps -elf | grep tchi > ps -elf | grep orf > ps -elf | grep ool
Check TmSourcePackets arrive and get propagated
ssh -X oper@pacs5 psetup db2tty -d pacs_fm_ilt_3 herschel.versant.ccm.TmSourcePacketImpl | grep Total sleep 20 db2tty -d pacs_fm_ilt_3 herschel.versant.ccm.TmSourcePacketImpl | grep Total
Check for an increasing number of objects.
Check disk space and space of the operational Database
ssh -X dbsa@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_3
if less than 20% free:
oscp -i ls -als /home/db/versant/pacs_fm_ilt_3 (not during tests) addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_3/extraspace_ -s 2047M pacs_fm_ilt_3
Database Backup done ?
ssh -X dbsa@pacs5 ls /home/db/dbBck
PACS5
Check Disk space and space of the propagated database
ssh -X oper@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_3_prop
Check daily and hourly TmPacketRecorder files
ssh -X oper@pacs1 ps -elf | grep PacketRecorder
restart if necessary:
psetup PacketRecorder -host pacs-sv -daily /dbBck/PacketFiles/fm%.tm < /dev/null >& /dev/null & PacketRecorder -host pacs-sv -hourly /dbBck/PacketFiles/fm_%.tm < /dev/null > & /dev/null &
Check whether Hardware support Monitor runs
ssh -X oper@pacs1 more /home/jakob/FMMonitor.txt
If not :
ssh jakob@pacs1 psetup cd monitor ./startMonitor >& /dev/null & ps -elf | grep writeHk
Monday Morning checks
PACS2
QLA running ?
PACS-SV
Check disk space
ssh oper@pacs-sv df -h .
To recover disk space try to delete "old" TM/TC packet files.
ssh oper@pacs-sv cd packetArchive find . -mtime +10 | xargs rm # to delete files older than 10 days # or find . -mtime +7| xargs rm # to delete files older than one week