Every Morning checks

PACSSCOS

S2K, Event monitor and CDMS sim. diskspaces

ssh -X oper@pacsscos
df

PACS1 - PACS5 (at present only for PACS3 & PACS4)

Check PCSS version.

ssh -X pcss@pacsx
cat software/pcss/RELEASE

if not actuell:

cd software
./fetchpcss

PACS 5

Is TmIngestion still running ?

  ssh -X oper@pacs5
  ps -elf | grep TmIngestion 

TC History and OOL Ingestion running ?

> ssh -X oper@pacs5
> ps -elf | grep thfser
> ps -elf | grep tchi
> ps -elf | grep orf
> ps -elf | grep ool

Check TmSourcePackets arrive and get propagated

  ssh -X oper@pacs5
  psetup
  db2tty -d pacs_fm_ilt_2 | grep TmsourcePackets 
  ssh -X oper@pacs1
  psetup
  db2tty -d pacs_fm_ilt_2_prop | grep TmsourcePackets 

Check disk space and space of the operational Database

ssh -X dbsa@pacs5
df
psetup
dbtool -space -volume -all pacs_fm_ilt_2

if less than 20% free:

oscp -i
ls -als /home/db/versant/pacs_fm_ilt_2
(not during tests)
addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_2/extraspace_ -s 2047M pacs_fm_ilt_2

preferable add 4x 2 GB - 2 GB quicker acess - 8 GB should last one week
total of DB should not be more than 20 GB 

Database Backup done ?

ssh -X dbsa@pacs5
ls /home/db/dbBck

PACS1

Check Disk space and space of the propagated database

ssh -X oper@pacs1
df
psetup
dbtool -space -volume -all pacs_fm_ilt_1_prop

Check daily and hourly TmPacketRecorder files

  ssh -X oper@pacs1
  ps -elf | grep PacketRecorder 

Check whether Hardware support Monitor runs

ssh -X oper@pacs1
cat /home/jakob/FMMonitor.txt

If not :

ssh -X jakob@pacs1
cd monitor 
jylaunch writeHkNrt.py 

PACS2

Monday Morning checks

QLA running ?

PACS-SV

Check disk space

ssh oper@pacs-sv
df -h .

To recover disk space try to delete "old" TM/TC packet files.

ssh oper@pacs-sv
cd packetArchive

find . -mtime +10 | xargs rm        # to delete files older than 10 days
# or
find . -mtime +7| xargs rm          # to delete files older than one week