Differences between revisions 6 and 13 (spanning 7 versions)
Revision 6 as of 2006-11-02 15:42:55
Size: 1345
Comment:
Revision 13 as of 2006-12-06 14:22:54
Size: 2321
Editor: Ost
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:

== PACSSCOS ==

S2K, Event monitor and CDMS sim. diskspaces
{{{
ssh -X oper@pacsscos
df
}}}

== PACS1 - PACS5 (at present only for PACS3 & PACS4) ==
Check PCSS version.
{{{
ssh -X pcss@pacsx
cat software/pcss/RELEASE
}}}

if not up-to-date:
{{{
cd software
./fetchpcss
}}}
Line 10: Line 31:
=== TC History and OOL Ingestion running ? === === TC History and OOL Ingestion running ?  (at present not because of deadlock) ===
Line 23: Line 44:
  db2tty -d pacs_fm_ilt_1 | grep TmsourcePackets
  ssh -X oper@pacs1
  db2tty -d pacs_fm_ilt_2 | grep TmsourcePackets

ssh -X oper@pacs1                                     (at present not)
Line 26: Line 48:
  db2tty -d pacs_fm_ilt_1_prop | grep TmsourcePackets   db2tty -d pacs_fm_ilt_2_prop | grep TmsourcePackets
Line 31: Line 53:
ssh -X oper@pacs5 ssh -X dbsa@pacs5
Line 34: Line 56:
dbtool -space -volume -all pacs_fm_ilt_1 dbtool -space -volume -all pacs_fm_ilt_2
Line 36: Line 58:

if less than 20% free:
{{{
oscp -i
ls -als /home/db/versant/pacs_fm_ilt_2
(not during tests)
addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_2/extraspace_ -s 2047M pacs_fm_ilt_2
}}}
Line 44: Line 75:
== PACS1 == == PACS5 ==
Line 46: Line 77:
=== Check Disk space and space of the propagated database === === Check Disk space and space of the propagated database (at present not) ===
Line 48: Line 79:
ssh -X oper@pacs1 ssh -X oper@pacs5
Line 57: Line 88:
  ps -elf | grep TmPacketRecorder   ps -elf | grep PacketRecorder
Line 63: Line 94:
cat /home/jakob/TMMonitor.txt cat /home/jakob/FMMonitor.txt
Line 73: Line 104:

= Monday Morning checks =
Line 75: Line 109:
=== QLA running ? ===
Line 76: Line 111:
= Monday Morning checks =
=== QLA running ? ===
= PACS-SV =

== Check disk space ==
{{{
ssh oper@pacs-sv
df -h .
}}}
To recover disk space try to delete "old" TM/TC packet files.
{{{
ssh oper@pacs-sv
cd packetArchive

find . -mtime +10 | xargs rm # to delete files older than 10 days
# or
find . -mtime +7| xargs rm # to delete files older than one week
}}}

Every Morning checks

PACSSCOS

S2K, Event monitor and CDMS sim. diskspaces

ssh -X oper@pacsscos
df                           

PACS1 - PACS5 (at present only for PACS3 & PACS4)

Check PCSS version.

ssh -X pcss@pacsx
cat software/pcss/RELEASE

if not up-to-date:

cd software
./fetchpcss

PACS 5

Is TmIngestion still running ?

  ssh -X oper@pacs5
  ps -elf | grep TmIngestion 

TC History and OOL Ingestion running ? (at present not because of deadlock)

> ssh -X oper@pacs5
> ps -elf | grep thfser
> ps -elf | grep tchi
> ps -elf | grep orf
> ps -elf | grep ool

Check TmSourcePackets arrive and get propagated

  ssh -X oper@pacs5
  psetup
  db2tty -d pacs_fm_ilt_2 | grep TmsourcePackets 

  ssh -X oper@pacs1                                     (at present not) 
  psetup
  db2tty -d pacs_fm_ilt_2_prop | grep TmsourcePackets 

Check disk space and space of the operational Database

ssh -X dbsa@pacs5
df
psetup
dbtool -space -volume -all pacs_fm_ilt_2

if less than 20% free:

oscp -i
ls -als /home/db/versant/pacs_fm_ilt_2
(not during tests)
addvol -n extraspace_ -p /home/db/versant/pacs_fm_ilt_2/extraspace_ -s 2047M pacs_fm_ilt_2

Database Backup done ?

ssh -X dbsa@pacs5
ls /home/db/dbBck

PACS5

Check Disk space and space of the propagated database (at present not)

ssh -X oper@pacs5
df
psetup
dbtool -space -volume -all pacs_fm_ilt_1_prop

Check daily and hourly TmPacketRecorder files

  ssh -X oper@pacs1
  ps -elf | grep PacketRecorder 

Check whether Hardware support Monitor runs

ssh -X oper@pacs1
cat /home/jakob/FMMonitor.txt

If not :

ssh -X jakob@pacs1
cd monitor 
jylaunch writeHkNrt.py 

Monday Morning checks

PACS2

QLA running ?

PACS-SV

Check disk space

ssh oper@pacs-sv
df -h .

To recover disk space try to delete "old" TM/TC packet files.

ssh oper@pacs-sv
cd packetArchive

find . -mtime +10 | xargs rm        # to delete files older than 10 days
# or
find . -mtime +7| xargs rm          # to delete files older than one week

Herschel: PACS/FM_ILT_Procedures/RoutineChecks (last edited 2009-07-15 14:32:38 by localhost)