|
Size: 1245
Comment:
|
Size: 2356
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| == Is TmIngestion still running ? == | == PACSSCOS == S2K, Event monitor and CDMS sim. diskspaces {{{ ssh -X oper@pacsscos df }}} == PACS1 - PACS5 (at present only for PACS3 & PACS4) == Check PCSS version. {{{ ssh -X pcss@pacsx cat software/pcss/RELEASE }}} if not up-to-date: {{{ cd software ./fetchpcss }}} == PACS 5 == === Is TmIngestion still running ? === |
| Line 9: | Line 31: |
| If it is not running type {{{ restartTmIngestion |
=== TC History and OOL Ingestion running ? (at present not because of deadlock) === {{{ > ssh -X oper@pacs5 > ps -elf | grep thfser > ps -elf | grep tchi > ps -elf | grep orf > ps -elf | grep ool |
| Line 14: | Line 40: |
| == Check TmSourcePackets arrive and get propagated == | === Check TmSourcePackets arrive and get propagated === |
| Line 18: | Line 44: |
| db2tty -d pacs_fm_ilt_1 | grep TmsourcePackets ssh -X oper@pacs1 |
db2tty -d pacs_fm_ilt_2 | grep TmsourcePackets ssh -X oper@pacs1 (at present not) |
| Line 21: | Line 48: |
| db2tty -d pacs_fm_ilt_1_prop | grep TmsourcePackets | db2tty -d pacs_fm_ilt_2_prop | grep TmsourcePackets |
| Line 24: | Line 51: |
| == Check disk space and space of the Database == | === Check disk space and space of the operational Database === {{{ ssh -X dbsa@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_2 }}} if less than 20% free: {{{ oscp -i ls -als /home/db/versant/pacs_fm_ilt_2 (not during tests) addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_2/extraspace_ -s 2047M pacs_fm_ilt_2 }}} === Database Backup done ? === {{{ ssh -X dbsa@pacs5 ls /home/db/dbBck }}} == PACS5 == === Check Disk space and space of the propagated database (at present not) === |
| Line 27: | Line 80: |
| psetup dbtool -space -volume -all pacs_fm_ilt_1 }}} If necessary expand the size : {{{ addvol -n extraspace -p /data/versant/db/pacs_fm_ilt_1/extraspace -s 2047M pacs_fm_ilt_1 }}} {{{ ssh -X oper@pacs1 |
df |
| Line 42: | Line 85: |
| If necessary expand the size : {{{ addvol -n extraspace -p /data/versant/db/pacs_fm_ilt_1_prop/extraspace -s 2047M pacs_fm_ilt_1_prop }}} |
|
| Line 51: | Line 88: |
| ps -elf | grep TmPacketRecorder | ps -elf | grep PacketRecorder |
| Line 57: | Line 94: |
| cat /home/jakob/TMMonitor.txt | cat /home/jakob/FMMonitor.txt |
| Line 62: | Line 99: |
| ssh -X jakob@pacs1 | ssh jakob@pacs1 psetup |
| Line 64: | Line 102: |
| jylaunch writeHkNrt.py | ./startMonitor >& /dev/null & ps -elf | grep writeHk |
| Line 66: | Line 105: |
= Monday Morning checks = == PACS2 == === QLA running ? === = PACS-SV = == Check disk space == {{{ ssh oper@pacs-sv df -h . }}} To recover disk space try to delete "old" TM/TC packet files. {{{ ssh oper@pacs-sv cd packetArchive find . -mtime +10 | xargs rm # to delete files older than 10 days # or find . -mtime +7| xargs rm # to delete files older than one week }}} |
Every Morning checks
PACSSCOS
S2K, Event monitor and CDMS sim. diskspaces
ssh -X oper@pacsscos df
PACS1 - PACS5 (at present only for PACS3 & PACS4)
Check PCSS version.
ssh -X pcss@pacsx cat software/pcss/RELEASE
if not up-to-date:
cd software ./fetchpcss
PACS 5
Is TmIngestion still running ?
ssh -X oper@pacs5 ps -elf | grep TmIngestion
TC History and OOL Ingestion running ? (at present not because of deadlock)
> ssh -X oper@pacs5 > ps -elf | grep thfser > ps -elf | grep tchi > ps -elf | grep orf > ps -elf | grep ool
Check TmSourcePackets arrive and get propagated
ssh -X oper@pacs5 psetup db2tty -d pacs_fm_ilt_2 | grep TmsourcePackets ssh -X oper@pacs1 (at present not) psetup db2tty -d pacs_fm_ilt_2_prop | grep TmsourcePackets
Check disk space and space of the operational Database
ssh -X dbsa@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_2
if less than 20% free:
oscp -i ls -als /home/db/versant/pacs_fm_ilt_2 (not during tests) addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_2/extraspace_ -s 2047M pacs_fm_ilt_2
Database Backup done ?
ssh -X dbsa@pacs5 ls /home/db/dbBck
PACS5
Check Disk space and space of the propagated database (at present not)
ssh -X oper@pacs5 df psetup dbtool -space -volume -all pacs_fm_ilt_1_prop
Check daily and hourly TmPacketRecorder files
ssh -X oper@pacs1 ps -elf | grep PacketRecorder
Check whether Hardware support Monitor runs
ssh -X oper@pacs1 cat /home/jakob/FMMonitor.txt
If not :
ssh jakob@pacs1 psetup cd monitor ./startMonitor >& /dev/null & ps -elf | grep writeHk
Monday Morning checks
PACS2
QLA running ?
PACS-SV
Check disk space
ssh oper@pacs-sv df -h .
To recover disk space try to delete "old" TM/TC packet files.
ssh oper@pacs-sv cd packetArchive find . -mtime +10 | xargs rm # to delete files older than 10 days # or find . -mtime +7| xargs rm # to delete files older than one week