Oracle 11g RAC 错误集锦之---VIP 故障

浏览数：57 / 时间：2015年06月12日

Oracle 11g RAC 错误集锦之---VIP 故障

故障现象：Oracle RAC VIP 故障

系统环境：

操作系统：Redhat EL55
集群软件：CRS 11G
数据库软件：Oracle 11g（11.2.0.1）

网络配置：

[root@syw1 bin]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1                localhost
192.168.16.101       syw1
10.10.10.101       syw1-priv
192.168.16.201       syw1-vip
192.168.16.102      syw2
10.10.10.102       syw2-priv
192.168.16.202     syw2-vip
192.168.16.110     rac_scan
[root@syw1 bin]#

故障现象：

由于在安装grid 时，在节点2 /etc/hosts文件配置错误，导致在CRS 资源里确少vip 和scan
[root@syw2 bin]# crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....BASE.dg ora....up.type ONLINE    ONLINE    syw1
ora....ER.lsnr ora....er.type ONLINE    ONLINE    syw1
ora....VOTE.dg ora....up.type ONLINE    ONLINE    syw1
ora.RCY.dg     ora....up.type ONLINE    ONLINE    syw1
ora.asm        ora.asm.type   ONLINE    ONLINE    syw1
ora.eons       ora.eons.type  ONLINE    ONLINE    syw1
ora.gsd        ora.gsd.type   ONLINE    ONLINE    syw1
ora....network ora....rk.type ONLINE    ONLINE    syw1
ora.ons        ora.ons.type   ONLINE    ONLINE    syw1
ora....ry.acfs ora....fs.type ONLINE    ONLINE    syw1
ora....SM2.asm application    ONLINE    ONLINE    syw1
ora....W1.lsnr application    ONLINE    ONLINE    syw1
ora.syw1.gsd   application    ONLINE    ONLINE    syw1
ora.syw1.ons   application    ONLINE    ONLINE    syw1
ora.syw1.vip   ora....t1.type ONLINE    ONLINE    syw1
ora....SM1.asm application    ONLINE    ONLINE    syw2
ora....W2.lsnr application    OFFLINE   OFFLINE
ora.syw2.gsd   application    ONLINE    ONLINE    syw2
ora.syw2.ons   application    ONLINE    ONLINE    syw2

解决方法：

第一步：添加vip service

[root@syw2 bin]# srvctl add -h

The SRVCTL add command adds the configuration and the Oracle Clusterware application to the OCR for the cluster database, named instances, named services, or for the named nodes.

Usage: srvctl add nodeapps { { -n <node_name> -A <name|ip>/<netmask>/[if1[|if2...]] } | { -S <subnet>/<netmask>/[if1[|if2...]] } } [-p <portnum>] [-m <multicast-ip-address>] [-e <eons-listen-port>] [-l <ons-local-port>] [-r <ons-remote-port>] [-t <host>[:<port>][,<host>[:<port>]...]] [-v]
Usage: srvctl add vip -n <node_name> -k <network_number> -A <name|ip>/<netmask>/[if1[|if2...]] [-v]

[root@syw2 bin]# srvctl add nodeapps -A 192.168.16.202/255.255.255.0/eth0 -n syw2
[root@syw2 bin]#

[root@syw2 bin]# crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....BASE.dg ora....up.type ONLINE    ONLINE    syw1
ora....ER.lsnr ora....er.type ONLINE    ONLINE    syw1
ora....VOTE.dg ora....up.type ONLINE    ONLINE    syw1
ora.RCY.dg     ora....up.type ONLINE    ONLINE    syw1
ora.asm        ora.asm.type   ONLINE    ONLINE    syw1
ora.eons       ora.eons.type  ONLINE    ONLINE    syw1
ora.gsd        ora.gsd.type   ONLINE    ONLINE    syw1
ora....network ora....rk.type ONLINE    ONLINE    syw1
ora.ons        ora.ons.type   ONLINE    ONLINE    syw1
ora....ry.acfs ora....fs.type ONLINE    ONLINE    syw1
ora....SM2.asm application    ONLINE    ONLINE    syw1
ora....W1.lsnr application    ONLINE    ONLINE    syw1
ora.syw1.gsd   application    ONLINE    ONLINE    syw1
ora.syw1.ons   application    ONLINE    ONLINE    syw1
ora.syw1.vip   ora....t1.type ONLINE    ONLINE    syw1
ora....SM1.asm application    ONLINE    ONLINE    syw2
ora....W2.lsnr application    OFFLINE   OFFLINE
ora.syw2.gsd   application    ONLINE    ONLINE    syw2
ora.syw2.ons   application    ONLINE    ONLINE    syw2
ora.syw2.vip   ora....t1.type OFFLINE   OFFLINE

启动vip service:

[root@syw2 bin]# srvctl start vip -n syw2
[root@syw2 bin]#

第二步：添加scan ip service

Usage: srvctl add scan -n <scan_name> [-k <network_number> [-S <subnet>/<netmask>[/if1[|if2|...]]]]
Usage: srvctl add scan_listener [-l <lsnr_name_prefix>] [-s] [-p [TCP:]<port>[/IPC:<key>][/NMP:<pipe_name>][/TCPS:<s_port>] [/SDP:<port>]]
[root@syw2 bin]# srvctl add scan -n rac_scan
[root@syw2 bin]# srvctl enable scan
PRCC-1010 : scan1 was already enabled

[root@syw2 bin]# crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....BASE.dg ora....up.type ONLINE    ONLINE    syw1
ora....ER.lsnr ora....er.type ONLINE    ONLINE    syw1
ora....VOTE.dg ora....up.type ONLINE    ONLINE    syw1
ora.RCY.dg     ora....up.type ONLINE    ONLINE    syw1
ora.asm        ora.asm.type   ONLINE    ONLINE    syw1
ora.eons       ora.eons.type  ONLINE    ONLINE    syw1
ora.gsd        ora.gsd.type   ONLINE    ONLINE    syw1
ora....network ora....rk.type ONLINE    ONLINE    syw1
ora.ons        ora.ons.type   ONLINE    ONLINE    syw1
ora....ry.acfs ora....fs.type ONLINE    ONLINE    syw1
ora.scan1.vip  ora....ip.type OFFLINE   OFFLINE
ora....SM2.asm application    ONLINE    ONLINE    syw1
ora....W1.lsnr application    ONLINE    ONLINE    syw1
ora.syw1.gsd   application    ONLINE    ONLINE    syw1
ora.syw1.ons   application    ONLINE    ONLINE    syw1
ora.syw1.vip   ora....t1.type ONLINE    ONLINE    syw1
ora....SM1.asm application    ONLINE    ONLINE    syw2
ora....W2.lsnr application    OFFLINE   OFFLINE
ora.syw2.gsd   application    ONLINE    ONLINE    syw2
ora.syw2.ons   application    ONLINE    ONLINE    syw2
ora.syw2.vip   ora....t1.type ONLINE    ONLINE    syw2
[root@syw2 bin]# crs_stat |grep scan
NAME=ora.scan1.vip
TYPE=ora.scan_vip.type
[root@syw2 bin]# crs_stat |grep lsn
NAME=ora.LISTENER.lsnr
NAME=ora.syw1.LISTENER_SYW1.lsnr
NAME=ora.syw2.LISTENER_SYW2.lsnr

启动scan ip service:

[root@syw2 bin]# srvctl start scan

第三步：添加scan listener

[root@syw2 bin]# srvctl add scan_listener -h

Adds a SCAN listener listening on the given endpoints.

Usage: srvctl add scan_listener [-l <lsnr_name_prefix>] [-s] [-p [TCP:]<port>[/IPC:<key>][/NMP:<pipe_name>][/TCPS:<s_port>] [/SDP:<port>]]
    -l <lsnr_prefix>         SCAN listener name prefix
    -s                       Skip the checking of ports
    -p [TCP:]<port>[/IPC:<key>][/NMP:<pipe_name>][/TCPS:<s_port>] [/SDP:<port>]           SCAN Listener endpoints
    -h                       Print usage
[root@syw2 bin]# srvctl add scan_listener

启动scan listener service:

[root@syw2 bin]# srvctl start scan_listener
[root@syw2 bin]#

[root@syw2 bin]# crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....BASE.dg ora....up.type ONLINE    ONLINE    syw1
ora....ER.lsnr ora....er.type ONLINE    ONLINE    syw1
ora....N1.lsnr ora....er.type ONLINE    ONLINE    syw1
ora....VOTE.dg ora....up.type ONLINE    ONLINE    syw1
ora.RCY.dg     ora....up.type ONLINE    ONLINE    syw1
ora.asm        ora.asm.type   ONLINE    ONLINE    syw1
ora.eons       ora.eons.type  ONLINE    ONLINE    syw1
ora.gsd        ora.gsd.type   ONLINE    ONLINE    syw1
ora....network ora....rk.type ONLINE    ONLINE    syw1
ora.ons        ora.ons.type   ONLINE    ONLINE    syw1
ora....ry.acfs ora....fs.type ONLINE    ONLINE    syw1
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    syw1
ora....SM2.asm application    ONLINE    ONLINE    syw1
ora....W1.lsnr application    ONLINE    ONLINE    syw1
ora.syw1.gsd   application    ONLINE    ONLINE    syw1
ora.syw1.ons   application    ONLINE    ONLINE    syw1
ora.syw1.vip   ora....t1.type ONLINE    ONLINE    syw1
ora....SM1.asm application    ONLINE    ONLINE    syw2
ora....W2.lsnr application    OFFLINE   OFFLINE
ora.syw2.gsd   application    ONLINE    ONLINE    syw2
ora.syw2.ons   application    ONLINE    ONLINE    syw2
ora.syw2.vip   ora....t1.type ONLINE    ONLINE    syw2
[root@syw2 bin]# crs_stat|grep lsn
NAME=ora.LISTENER.lsnr
NAME=ora.LISTENER_SCAN1.lsnr
NAME=ora.syw1.LISTENER_SYW1.lsnr
NAME=ora.syw2.LISTENER_SYW2.lsnr

启动local listener：
[root@syw2 bin]# crs_start -f ora.syw2.LISTENER_SYW2.lsnr
Attempting to start `ora.LISTENER.lsnr` on member `syw2`
Start of `ora.LISTENER.lsnr` on member `syw2` succeeded.

[root@syw2 bin]# crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....BASE.dg ora....up.type ONLINE    ONLINE    syw1
ora....ER.lsnr ora....er.type ONLINE    ONLINE    syw1
ora....N1.lsnr ora....er.type ONLINE    ONLINE    syw1
ora....VOTE.dg ora....up.type ONLINE    ONLINE    syw1
ora.RCY.dg     ora....up.type ONLINE    ONLINE    syw1
ora.asm        ora.asm.type   ONLINE    ONLINE    syw1
ora.eons       ora.eons.type  ONLINE    ONLINE    syw1
ora.gsd        ora.gsd.type   ONLINE    ONLINE    syw1
ora....network ora....rk.type ONLINE    ONLINE    syw1
ora.ons        ora.ons.type   ONLINE    ONLINE    syw1
ora....ry.acfs ora....fs.type ONLINE    ONLINE    syw1
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    syw1
ora....SM2.asm application    ONLINE    ONLINE    syw1
ora....W1.lsnr application    ONLINE    ONLINE    syw1
ora.syw1.gsd   application    ONLINE    ONLINE    syw1
ora.syw1.ons   application    ONLINE    ONLINE    syw1
ora.syw1.vip   ora....t1.type ONLINE    ONLINE    syw1
ora....SM1.asm application    ONLINE    ONLINE    syw2
ora....W2.lsnr application    ONLINE    ONLINE    syw2
ora.syw2.gsd   application    ONLINE    ONLINE    syw2
ora.syw2.ons   application    ONLINE    ONLINE    syw2
ora.syw2.vip   ora....t1.type ONLINE    ONLINE    syw2

查看vip ip 启动信息：

[root@syw2 bin]# ip add sh
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 08:00:27:e3:1e:a4 brd ff:ff:ff:ff:ff:ff
inet 192.168.16.102/24 brd 192.168.16.255 scope global eth0
    inet 192.168.16.202/24 brd 192.168.16.255 scope global secondary eth0:1
    inet6 fe80::a00:27ff:fee3:1ea4/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 08:00:27:8d:49:38 brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.102/24 brd 10.10.10.255 scope global eth1
    inet6 fe80::a00:27ff:fe8d:4938/64 scope link
       valid_lft forever preferred_lft forever
4: sit0: <NOARP> mtu 1480 qdisc noop
    link/sit 0.0.0.0 brd 0.0.0.0
[root@syw2 bin]#