oracle11g巡检RAC数据库

# 环境

操作系统

$ uname -a

Linux jinhu-db02 2.6.32-696.el6.x86_64 #1 SMP Tue Mar 21

19:29:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/redhat-release

CentOS release 6.9 (Final)

数据库

BANNER

--------------------------------------------------------------------------------

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 -

64bit Production

PL/SQL Release 11.2.0.4.0 - Production

CORE 11.2.0.4.0 Production

TNS for Linux: Version 11.2.0.4.0 - Production

NLSRTL Version 11.2.0.4.0 - Production

一.巡检RAC数据库

1.1列出数据库

[grid@jinhu-db01 ~]$ srvctl config database

orcl

[grid@jinhu-db02 ~]$ srvctl config database

orcl

1.2列出数据库的实例

[grid@jinhu-db01 ~]$ srvctl status database -d orcl

Instance oracl11 is running on node jinhu-db01

Instance oracl12 is running on node jinhu-db02

[grid@jinhu-db02 ~]$ srvctl status database -d orcl

Instance oracl11 is running on node jinhu-db01

Instance oracl12 is running on node jinhu-db02

1.3数据库的配置

[grid@jinhu-db01 ~]$ srvctl config database -d orcl -a

Database unique name: orcl

Database name: orcl

Oracle home: /u01/app/oracle/product/11.2.0/db_1

Oracle user: oracle

Spfile: +RACDATA/orcl/spfileoracl1.ora

Domain:

Start options: open

Stop options: immediate

Database role: PRIMARY

Management policy: AUTOMATIC

Server pools: orcl

Database instances: oracl11,oracl12

Disk Groups: RACDATA,DATA

Mount point paths:

Services:

Type: RAC

Database is enabled

Database is administrator managed

[grid@jinhu-db02 ~]$ srvctl config database -d orcl -a

Database unique name: orcl

Database name: orcl

Oracle home: /u01/app/oracle/product/11.2.0/db_1

Oracle user: oracle

Spfile: +RACDATA/orcl/spfileoracl1.ora

Domain:

Start options: open

Stop options: immediate

Database role: PRIMARY

Management policy: AUTOMATIC

Server pools: orcl

Database instances: oracl11,oracl12

Disk Groups: RACDATA,DATA

Mount point paths:

Services:

Type: RAC

Database is enabled

Database is administrator managed

二.巡检Grid

2.1集群名称

[grid@jinhu-db01 ~]$ cemutlo -n

rac-cluster

[grid@jinhu-db02 ~]$ cemutlo -n

rac-cluster

2.2检查集群栈状态

[grid@jinhu-db01 ~]$ crsctl check cluster -all

**************************************************************

jinhu-db01:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

jinhu-db02:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

[grid@jinhu-db02 ~]$ crsctl check cluster -all

**************************************************************

jinhu-db01:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

jinhu-db02:

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

**************************************************************

2.3 集群的资源

[grid@jinhu-db01 ~]$ crsctl status res -t

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.DATA.dg

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.FRA.dg

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.LISTENER.lsnr

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.RACDATA.dg

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.asm

ONLINE ONLINE jinhu-db01 Started

ONLINE ONLINE jinhu-db02 Started

ora.gsd

OFFLINE

OFFLINE jinhu-db01

OFFLINE

OFFLINE jinhu-db02

ora.net1.network

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.ons

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

1 ONLINE ONLINE jinhu-db01

ora.cvu

1 ONLINE ONLINE jinhu-db01

ora.jinhu-db01.vip

1 ONLINE ONLINE jinhu-db01

ora.jinhu-db02.vip

1 ONLINE ONLINE jinhu-db02

ora.oc4j

1 ONLINE ONLINE jinhu-db01

ora.orcl.db

1 ONLINE ONLINE jinhu-db01 Open

2 ONLINE ONLINE jinhu-db02 Open

ora.scan1.vip

1 ONLINE ONLINE jinhu-db01

[grid@jinhu-db02 ~]$ crsctl status res -t

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.DATA.dg

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.FRA.dg

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.LISTENER.lsnr

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.RACDATA.dg

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.asm

ONLINE ONLINE jinhu-db01 Started

ONLINE ONLINE jinhu-db02 Started

ora.gsd

OFFLINE

OFFLINE jinhu-db01

OFFLINE

OFFLINE jinhu-db02

ora.net1.network

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

ora.ons

ONLINE ONLINE jinhu-db01

ONLINE ONLINE jinhu-db02

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

1 ONLINE ONLINE jinhu-db01

ora.cvu

1 ONLINE ONLINE jinhu-db01

ora.jinhu-db01.vip

1 ONLINE ONLINE jinhu-db01

ora.jinhu-db02.vip

1 ONLINE ONLINE jinhu-db02

ora.oc4j

1 ONLINE ONLINE jinhu-db01

ora.orcl.db

1 ONLINE ONLINE jinhu-db01 Open

2 ONLINE ONLINE jinhu-db02 Open

ora.scan1.vip

1 ONLINE ONLINE jinhu-db01

主机jinhu-db01的更加详细的资源

[grid@jinhu-db01 ~]$ crsctl status res -t -init

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

1 ONLINE ONLINE jinhu-db01 Started

ora.cluster_interconnect.haip

1 ONLINE ONLINE jinhu-db01

ora.crf

1 ONLINE ONLINE jinhu-db01

ora.crsd

1 ONLINE ONLINE jinhu-db01

ora.cssd

1 ONLINE ONLINE jinhu-db01

ora.cssdmonitor

1 ONLINE ONLINE jinhu-db01

ora.ctssd

1 ONLINE ONLINE jinhu-db01 OBSERVER

ora.diskmon

1 OFFLINE OFFLINE

ora.evmd

1 ONLINE ONLINE jinhu-db01

ora.gipcd

1 ONLINE ONLINE jinhu-db01

ora.gpnpd

1 ONLINE ONLINE jinhu-db01

ora.mdnsd

1 ONLINE ONLINE jinhu-db01

主机jinhu-db02的更加详细的资源

[grid@jinhu-db02 ~]$ crsctl status res -t -init

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

1 ONLINE ONLINE jinhu-db02 Started

ora.cluster_interconnect.haip

1 ONLINE ONLINE jinhu-db02

ora.crf

1 ONLINE ONLINE jinhu-db02

ora.crsd

1 ONLINE ONLINE jinhu-db02

ora.cssd

1 ONLINE ONLINE jinhu-db02

ora.cssdmonitor

1 ONLINE ONLINE jinhu-db02

ora.ctssd

1 ONLINE ONLINE jinhu-db02 OBSERVER

ora.diskmon

1 OFFLINE OFFLINE

ora.evmd

1 ONLINE ONLINE jinhu-db02

ora.gipcd

1 ONLINE ONLINE jinhu-db02

ora.gpnpd

1 ONLINE ONLINE jinhu-db02

ora.mdnsd

1 ONLINE ONLINE jinhu-db02

2.4检查节点应用

[grid@jinhu-db01 ~]$ srvctl status nodeapps

VIP jinhu-db01-vip is enabled

VIP jinhu-db01-vip is running on node: jinhu-db01

VIP jinhu-db02-vip is enabled

VIP jinhu-db02-vip is running on node: jinhu-db02

Network is enabled

Network is running on node: jinhu-db01

Network is running on node: jinhu-db02

GSD is disabled

GSD is not running on node: jinhu-db01

GSD is not running on node: jinhu-db02

ONS is enabled

ONS daemon is running on node: jinhu-db01

ONS daemon is running on node: jinhu-db02

[grid@jinhu-db02 ~]$ srvctl status nodeapps

VIP jinhu-db01-vip is enabled

VIP jinhu-db01-vip is running on node: jinhu-db01

VIP jinhu-db02-vip is enabled

VIP jinhu-db02-vip is running on node: jinhu-db02

Network is enabled

Network is running on node: jinhu-db01

Network is running on node: jinhu-db02

GSD is disabled

GSD is not running on node: jinhu-db01

GSD is not running on node: jinhu-db02

ONS is enabled

ONS daemon is running on node: jinhu-db01

ONS daemon is running on node: jinhu-db02

2.5 检查SCAN

检查scan-ip地址的配置

[grid@jinhu-db01 ~]$ srvctl config scan

SCAN name: racscan, Network:

1/192.168.20.0/255.255.255.0/eth0

SCAN VIP name: scan1, IP: /racscan/192.168.20.30

[grid@jinhu-db02 ~]$ srvctl config scan

SCAN name: racscan, Network:

1/192.168.20.0/255.255.255.0/eth0

SCAN VIP name: scan1, IP: /racscan/192.168.20.30

检查scan-ip地址的实际分布及状态

[grid@jinhu-db01 ~]$ srvctl status scan

SCAN VIP scan1 is enabled

SCAN VIP scan1 is running on node

jinhu-db01

[grid@jinhu-db02 ~]$ srvctl status scan

SCAN VIP scan1 is enabled

SCAN VIP scan1 is running on node jinhu-db01

检查scan监听配置

[grid@jinhu-db01 ~]$ srvctl config scan_listener

SCAN Listener LISTENER_SCAN1 exists. Port:

TCP:1521

[grid@jinhu-db02 ~]$ srvctl config scan_listener

SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521

检查scan监听状态

[grid@jinhu-db01 ~]$ srvctl status scan_listener

SCAN Listener LISTENER_SCAN1 is enabled

SCAN listener LISTENER_SCAN1 is running on node

jinhu-db01

[grid@jinhu-db02 ~]$ srvctl status scan_listener

SCAN Listener LISTENER_SCAN1 is enabled

SCAN listener LISTENER_SCAN1 is running on node

jinhu-db01

2.6 检查VIP和监听

检查VIP的配置情况

[grid@jinhu-db01 ~]$ srvctl config vip -n jinhu-db01

VIP exists:

/jinhu-db01-vip/192.168.20.11/192.168.20.0/255.255.255.0/eth0,

hosting node jinhu-db01

[grid@jinhu-db02 ~]$ srvctl config vip -n jinhu-db02

VIP exists:

/jinhu-db02-vip/192.168.20.21/192.168.20.0/255.255.255.0/eth0,

hosting node jinhu-db02

检查VIP的状态

[grid@jinhu-db01 ~]$ srvctl status nodeapps

VIP jinhu-db01-vip is enabled

VIP jinhu-db01-vip is running on node: jinhu-db01

VIP jinhu-db02-vip is enabled

VIP jinhu-db02-vip is running on node: jinhu-db02

Network is enabled

Network is running on node: jinhu-db01

Network is running on node: jinhu-db02

GSD is disabled

GSD is not running on node: jinhu-db01

GSD is not running on node: jinhu-db02

ONS is enabled

ONS daemon is running on node: jinhu-db01

ONS daemon is running on node: jinhu-db02

或

[grid@jinhu-db01 ~]$ srvctl status vip -n jinhu-db01

VIP jinhu-db01-vip is enabled

VIP jinhu-db01-vip is running on node:

jinhu-db01

[grid@jinhu-db02 ~]$ srvctl status vip -n jinhu-db02

VIP jinhu-db02-vip is enabled

VIP jinhu-db02-vip is running on node: jinhu-db02

检查本地监听配置：

[grid@jinhu-db01 ~]$ srvctl config listener -a

Name: LISTENER

Network: 1, Owner: grid

Home:

/u01/app/11.2.0/grid on node(s)

jinhu-db02,jinhu-db01

End points: TCP:1521

[grid@jinhu-db02 ~]$ srvctl config listener -a

Name: LISTENER

Network: 1, Owner: grid

Home:

/u01/app/11.2.0/grid on node(s)

jinhu-db02,jinhu-db01

End points: TCP:1521

检查本地监听状态：

[grid@jinhu-db01 ~]$ srvctl status listener

Listener LISTENER is enabled

Listener LISTENER is running on node(s):

jinhu-db01,jinhu-db02

[grid@jinhu-db02 ~]$ srvctl status listener

Listener LISTENER is enabled

Listener LISTENER is running on node(s):

jinhu-db01,jinhu-db02

2.7 检查ASM

检查ASM配置

[grid@jinhu-db01 ~]$ srvctl config asm -a

ASM home: /u01/app/11.2.0/grid

ASM listener: LISTENER

ASM is enabled.

[grid@jinhu-db02 ~]$ srvctl config asm -a

ASM home: /u01/app/11.2.0/grid

ASM listener: LISTENER

ASM is enabled.

检查ASM状态

[grid@jinhu-db01 ~]$ srvctl status asm -a

ASM is running on jinhu-db01,jinhu-db02

ASM is enabled.

[grid@jinhu-db02 ~]$ srvctl status asm -a

ASM is running on jinhu-db01,jinhu-db02

ASM is enabled.

检查磁盘组

[grid@jinhu-db01 ~]$ srvctl status diskgroup -g DATA

Disk Group DATA is running on jinhu-db01,jinhu-db02

[grid@jinhu-db01 ~]$ srvctl status diskgroup -g FRA

Disk Group FRA is running on jinhu-db01,jinhu-db02

[grid@jinhu-db01 ~]$ srvctl status diskgroup -g RACDATA

Disk Group RACDATA is running on jinhu-db01,jinhu-db02

[oracle@jinhu-db02 ~]$ srvctl status diskgroup -g DATA

Disk Group DATA is running on jinhu-db01,jinhu-db02

[oracle@jinhu-db02 ~]$ srvctl status diskgroup -g FRA

Disk Group FRA is running on jinhu-db01,jinhu-db02

[oracle@jinhu-db02 ~]$ srvctl status diskgroup -g

RACDATA

Disk Group RACDATA is running on jinhu-db01,jinhu-db02

查看ASM磁盘及与asm 磁盘对应关系

[oracle@jinhu-db02 ~]$ sqlplus /nolog

SQL> SELECT group_number,name,path FROM

v$asm_disk_stat;

GROUP_NUMBER NAME  PATH

------------ ------------------------------

------------------------------

2 FRA_0000

/dev/mapper/data3

3 RACDATA_0000

/dev/mapper/data2

1 DATA_0000

/dev/mapper/data1

SQL> SELECT name,path FROM v$asm_disk;

NAME  PATH

------------------------------

------------------------------

FRA_0000  /dev/mapper/data3

RACDATA_0000  /dev/mapper/data2

DATA_0000  /dev/mapper/data1

2.8检查集群节点间的时钟同步

检查节点jinhu-db01的时间同步

[grid@jinhu-db01 ~]$ cluvfy comp clocksync -verbose

Verifying Clock Synchronization across the cluster

nodes

Checking if Clusterware is installed on all nodes...

Check of Clusterware install passed

Checking if CTSS Resource is running on all nodes...

Check: CTSS Resource running on all nodes

Node Name Status

------------------------------------ ------------------------

jinhu-db01 passed

Result: CTSS resource check passed

Querying CTSS for time offset on all nodes...

Result: Query of CTSS for time offset passed

Check CTSS state started...

Check: CTSS state

Node Name State

------------------------------------ ------------------------

jinhu-db01 Observer

CTSS is in Observer state. Switching over to clock

synchronization checks using NTP

Starting Clock synchronization checks using Network Time

Protocol(NTP)...

NTP Configuration file check started...

The NTP configuration file "/etc/ntp.conf" is available on all

nodes

NTP Configuration file check passed

Checking daemon liveness...

Check: Liveness for "ntpd"

Node Name Running?

------------------------------------ ------------------------

jinhu-db01 no

Result: Liveness check failed for "ntpd"

PRVF-5494 : The NTP Daemon or Service was not alive on all

nodes

PRVF-5415 : Check to see if NTP daemon or service is running

failed

Result: Clock synchronization check using Network Time

Protocol(NTP) failed

PRVF-9652 : Cluster Time Synchronization Services check

failed

Verification of Clock Synchronization across the cluster nodes

was unsuccessful on all the specified nodes.

检查节点jinhu-db02的时间同步

[grid@jinhu-db02 ~]$ cluvfy comp clocksync -verbose

Verifying Clock Synchronization across the cluster

nodes

Checking if Clusterware is installed on all nodes...

Check of Clusterware install passed

Checking if CTSS Resource is running on all nodes...

Check: CTSS Resource running on all nodes

Node Name Status

------------------------------------ ------------------------

jinhu-db02 passed

Result: CTSS resource check passed

Querying CTSS for time offset on all nodes...

Result: Query of CTSS for time offset passed

Check CTSS state started...

Check: CTSS state

Node Name State

------------------------------------ ------------------------

jinhu-db02 Observer

CTSS is in Observer state. Switching over to clock

synchronization checks using NTP

Starting Clock synchronization checks using Network Time

Protocol(NTP)...

NTP Configuration file check started...

The NTP configuration file "/etc/ntp.conf" is available on all

nodes

NTP Configuration file check passed

Checking daemon liveness...

Check: Liveness for "ntpd"

Node Name Running?

------------------------------------ ------------------------

jinhu-db02 no

Result: Liveness check failed for "ntpd"

PRVF-5494 : The NTP Daemon or Service was not alive on all

nodes

PRVF-5415 : Check to see if NTP daemon or service is running

failed

Result: Clock synchronization check using Network Time

Protocol(NTP) failed

PRVF-9652 : Cluster Time Synchronization Services check

failed

Verification of Clock Synchronization across the cluster nodes

was unsuccessful on all the specified nodes.

!!The End!!