安装Oracle11gR2 - RAC

RAC安装文档

2015

年01月06日

1

目录

一 环境准备 ..................................................................................................................................... 3

1.1安装环境：......................................................................................................................... 3 1.2 禁用防火墙和SELNUX（所有节点） .......................................................................... 3 1.3 修改/etc/hosts 文件（所有节点） .................................................................................. 3 1.4 添加组和用户（所有节点） ............................................................................................ 4 1.5建目录（所有节点） ......................................................................................................... 5 1.6修改系统参数（所有节点） ............................................................................................. 6 1.7 配置时间同步（所有节点） ............................................................................................ 6 1.8安装依赖包（所有节点） ................................................................................................. 8 1.9配置用户环境变量（所有节点） ..................................................................................... 8

1.9.1 grid 用户 ............................................................................................................ 8 1.9.2 oracle 用户 ........................................................................................................ 9 1.10使用UDEV 进行持久化（两个节点） ...................................................................... 10 二 安装CRS .................................................................................................................................. 11

2.1 安装Grid infrastructure ............................................................................................... 11 2.2 检查CRS状态 ................................................................................................................ 27

2.2.1 资源组状态:....................................................................................................... 27 2.2.2 检查集群节点:................................................................................................... 28 2.2.3 检查 Oracle 集群注册表 (OCR):................................................................... 28 2.2.4 检查表决磁盘:................................................................................................... 29 2.2.5 检查CRS状态..................................................................................................... 29

三 安装Oracle 软件（不建库） ................................................................................................ 30

2

一 环境准备

1.1安装环境：

本机32位VMware Workstation

操作系统OracleLinux-R6-U3-Server-x86_64-dvd 安装介质： p10404530_112030_Linux-x86-64_3of7.zip p10404530_112030_Linux-x86-64_1of7.zip p10404530_112030_Linux-x86-64_2of7.zip IP分配： 主机名 node1 node2 sacnIP 安装目录： Oracle软件 grid 软件 /s01/oracle/app/oracle/product/11.2.0/dbhome_1 /g01/11ggrid/app/11.2.0/grid 主机IP 172.24.240.72 172.24.240.73 172.24.240.87 PRIV 10.0.0.4 10.0.0.5 VIP 172.24.240.82 172.24.240.83 集群基础包 数据库基础包 数据库基础包 1.2 禁用防火墙和SELNUX（所有节点）

关闭防火墙：

Service iptables status Service iptables stop chkconfig iptables off chkconfig iptables –list

[root@oracle1 ~]# cat /etc/selinux/config

# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced.

# permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled

# SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted

1.3 修改/etc/hosts 文件（所有节点）

3

Last login: Tue Jan 6 08:51:31 2015 from 10.24.111.91 [root@node1 ~]# cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

172.24.240.72 node1 node1.oracle.com 172.24.240.82 node1-vip

172.24.240.73 node2 node2.oracle.com 172.24.240.83 node2-vip

172.24.240.87 node-cluster node-cluster-scan

10.0.0.4 node1-priv 10.0.0.5 node2-priv [root@node1 ~]#

Last login: Sun Jan 4 15:55:55 2015 from 10.24.111.91 [root@node2 ~]# cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

172.24.240.72 node1 node1.oracle.com 172.24.240.82 node1-vip

172.24.240.73 node2 node2.oracle.com 172.24.240.83 node2-vip

172.24.240.87 node-cluster node-cluster-scan

10.0.0.4 node1-priv 10.0.0.5 node2-priv [root@node2 ~]#

1.4 添加组和用户（所有节点）

node1添加用户和组

groupadd -g 5000 asmadmin groupadd -g 5001 asmdba groupadd -g 5002 asmoper

4

groupadd -g 6000 oinstall groupadd -g 6001 dba groupadd -g 6002 oper

useradd -g oinstall -G asmadmin,asmdba,asmoper grid useradd -g oinstall -G dba,asmdba oracle

node2添加用户和组

groupadd -g 5000 asmadmin groupadd -g 5001 asmdba groupadd -g 5002 asmoper groupadd -g 6000 oinstall groupadd -g 6001 dba groupadd -g 6002 oper

useradd -g oinstall -G asmadmin,asmdba,asmoper grid useradd -g oinstall -G dba,asmdba oracle

设置oracle和grid用户密码（所有节点）

passwd oracle passwd grid

1.5建目录（所有节点）

node1建立目录 mkdir /s01 mkdir /g01

chown oracle:oinstall /s01 chown grid:oinstall /g01 mkdir /g01/orabase mkdir /s01/orabase

chown oracle:oinstall /s01/orabase chown grid:oinstall /g01/orabase

node2建立目录 mkdir /s01 mkdir /g01

chown oracle:oinstall /s01

5

chown grid:oinstall /g01 mkdir /g01/orabase mkdir /s01/orabase

chown oracle:oinstall /s01/orabase chown grid:oinstall /g01/orabase

1.6修改系统参数（所有节点）

[root@node1 ~]# vi /etc/security/limits.conf

#ORACLE SETTING

grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536

[root@node2 app]# vi /etc/security/limits.conf #ORACLE SETTING

grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536

1.7 配置时间同步（所有节点）

在11gR2 中，RAC 在安装的时候，时间同步可以用两种方式来实现： （1） NTP （2） CTSS。

当安装程序发现 NTP 协议处于非活动状态时，安装集群时间同步服务将

以活动模式自动进行安装并通过所有节点的时间。如果发现配置了 NTP，则以

6

观察者模式启动集群时间同步服务，Oracle Clusterware 不会在集群中进行活 动的时间同步。

Oracle 集群时间同步服务 (ctssd) 旨在为那些 Oracle RAC 数据库 无法访问 NTP 服务的组织提供服务。目前建议配置NTP。

本文档采用CTSS

使用集群时间同步服务在集群中提供同步服务，需要卸载网络时间协 议 (NTP) 及其配置。

要停用 NTP 服务，必须停止当前的 ntpd 服务，从初始化序列中禁用 该服务，并删除 ntp.conf 文件。

要在Linux 上完成这些步骤，以 root 用户身份在两个 Oracle RAC 节 点上运行以下命令：

[root@node1 ~]# /sbin/service ntpd stop

关闭 ntpd： [失败] [root@node1 ~]# chkconfig ntpd off

[root@node1 ~]# mv /etc/ntp.conf /etc/ntp.conf.original [root@ node1 ~]# chkconfig ntpd --list

ntpd 0:关闭 1:关闭 2:关闭 3:关闭 4:关闭 5:关闭 6:关闭 [root@node1 ~]# rm /var/run/ntpd.pid

rm: 无法删除\没有那个文件或目录

[root@ node2 ~]# /sbin/service ntpd stop

关闭 ntpd： [失败] [root@ node2 ~]# chkconfig ntpd off

[root@ node2 ~]# mv /etc/ntp.conf /etc/ntp.conf.original [root@ node2 ~]# chkconfig ntpd --list

ntpd 0:关闭 1:关闭 2:关闭 3:关闭 4:关闭 5:关闭 6:关闭 [root@ node2 ~]# rm /var/run/ntpd.pid

rm: 无法删除\没有那个文件或目录

在安装集群软件完成后，要确认 ctssd 处于活动状态，用grid 所有者输入以下命令：

[grid@node1 ~]$ crsctl check ctss

CRS-4701: The Cluster Time Synchronization Service is in Active mode. CRS-4702: Offset (in msec): 0

[grid@node2 ~]$ crsctl check ctss

CRS-4701: The Cluster Time Synchronization Service is in Active mode. CRS-4702: Offset (in msec): 0

7

1.8安装依赖包（所有节点）

将系统镜像文件OracleLinux-R6-U3-Server-x86_64-dvd.iso挂载。执行以下命令：

[root@node1 ~]# cd /etc/yum.repos.d

[root@node1 yum.repos.d]#mv public-yum-ol6.repo public-yum-ol6.repo.bak

[root@node1 yum.repos.d]# touch public-yum-ol6.repo [root@node1 yum.repos.d]# vi public-yum-ol6.repo 设置其内容为：

[oel6]

name = Enterprise Linux 6.3 DVD

baseurl=file:///media/\gpgcheck=0 enabled=1

然后执行安装命令进行相关依赖软件的安装： [root@node1 ~]# yum install

oracle-rdbms-server-11gR2-preinstall-1.0-6.el6 [root@node1 ~]# yum install xterm

1.9配置用户环境变量（所有节点）

1.9.1 grid 用户

node1配置环境变量 [root@node1 ~]# su - grid

[grid@node1 ~]$ vi .bash_profile

# .bash_profile

# Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi

# User specific environment and startup programs

export GRID_HOME=/g01/11ggrid/app/11.2.0/grid export ORACLE_HOME=/g01/11ggrid/app/11.2.0/grid

export PATH=$GRID_HOME/bin:$GRID_HOME/OPatch:/sbin:/bin:/usr/sbin:/usr/bin

8

export ORACLE_SID=+ASM1

export LD_LIBRARY_PATH=$GRID_HOME/lib:$GRID_HOME/lib32

export ORACLE_BASE=/g01/orabase

export ORA_NLS10=$ORACLE_HOME/nls/data

export NLS_LANG=\~

node2配置环境变量 [root@node2 ~]# su grid

[grid@node2 ~]$ vi .bash_profile

# .bash_profile

# Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi

# User specific environment and startup programs

export GRID_HOME=/g01/11ggrid/app/11.2.0/grid export ORACLE_HOME=/g01/11ggrid/app/11.2.0/grid

export PATH=$GRID_HOME/bin:$GRID_HOME/OPatch:/sbin:/bin:/usr/sbin:/usr/bin

export ORACLE_SID=+ASM2

export LD_LIBRARY_PATH=$GRID_HOME/lib:$GRID_HOME/lib32

export ORACLE_BASE=/g01/orabase

export ORA_NLS10=$ORACLE_HOME/nls/data

export NLS_LANG=\ ~ ~ ~ ~

1.9.2 oracle 用户

node1配置环境变量

[oracle@node1 ~]$ cat .bash_profile # .bash_profile

# Get the aliases and functions

9

if [ -f ~/.bashrc ]; then . ~/.bashrc fi

# User specific environment and startup programs

export ORACLE_HOME=/s01/oracle/app/oracle/product/11.2.0/dbhome_1

export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:/sbin:/bin:/usr/sbin:/usr/bin

export ORACLE_SID=ORCL1

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32

export ORACLE_BASE=/s01/orabase

export ORA_NLS10=$ORACLE_HOME/nls/data

export NLS_LANG=\

node2配置环境变量

[oracle@node2 ~]$ cat .bash_profile # .bash_profile

# Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi

# User specific environment and startup programs

export ORACLE_HOME=/s01/oracle/app/oracle/product/11.2.0/dbhome_1

export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:/sbin:/bin:/usr/sbin:/usr/bin

export ORACLE_SID=ORCL2

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32

export ORACLE_BASE=/s01/orabase

export ORA_NLS10=$ORACLE_HOME/nls/data

export NLS_LANG=\

1.10使用UDEV 进行持久化（两个节点）

配置共享存储 for i in b ;

10

do

echo \--replace-whitespace --device=/dev/\\$name\\\--replace-whitespace --device=/dev/sd$i`\\\

GROUP=\\\ >> /etc/udev/rules.d/99-oracle-asmdevices.rules done

/sbin/start_udev

[grid@node2 ~]$ ls -l /dev/asm*

brw-rw---- 1 grid asmadmin 8, 16 Jan 6 11:18 /dev/asm-diskb

二 安装CRS

2.1 安装Grid infrastructure

解压p10404530_112030_Linux-x86-64_3of7.zip到/g01/grid目录下，

Unzip p10404530_112030_Linux-x86-64_3of7.zip –d /g01/grid 使用Xstart软件连接到oracle1。

[grid@node1 grid]$ ./runInstaller

11

12

13

14

15

16

17

18

19

在node1和node2运行/tmp/CVU_11.2.0.3.0_grid/runfixup.sh脚本

[root@node2 ~]# /tmp/CVU_11.2.0.3.0_grid/runfixup.sh

Response file being used is :/tmp/CVU_11.2.0.3.0_grid/fixup.response Enable file being used is :/tmp/CVU_11.2.0.3.0_grid/fixup.enable Log file location: /tmp/CVU_11.2.0.3.0_grid/orarun.log

Installing Package /tmp/CVU_11.2.0.3.0_grid//cvuqdisk-1.0.9-1.rpm

Preparing... ########################################### [100%] 1:cvuqdisk ########################################### [100%] [root@node2 ~]#

20

21

22

分别在node1和node2运行g01/oraInventory/orainstRoot.sh和/g01/11ggrid/app/11.2.0/grid/root.sh脚本

[root@node1 ~]# /g01/oraInventory/orainstRoot.sh Changing permissions of /g01/oraInventory. Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /g01/oraInventory to oinstall. The execution of the script is complete.

[root@node1 ~]# /g01/11ggrid/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g

The following environment variables are set as: ORACLE_OWNER= grid

ORACLE_HOME= /g01/11ggrid/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ...

23

Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /g01/11ggrid/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory

User ignored Prerequisites during installation OLR initialization - successful root wallet root wallet cert root cert export peer wallet

profile reader wallet pa wallet

peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert

peer root cert TP

profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP

profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert

Adding Clusterware entries to upstart

CRS-2672: Attempting to start 'ora.mdnsd' on 'node1' CRS-2676: Start of 'ora.mdnsd' on 'node1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'node1' CRS-2676: Start of 'ora.gpnpd' on 'node1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node1' CRS-2672: Attempting to start 'ora.gipcd' on 'node1'

CRS-2676: Start of 'ora.cssdmonitor' on 'node1' succeeded CRS-2676: Start of 'ora.gipcd' on 'node1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'node1' CRS-2672: Attempting to start 'ora.diskmon' on 'node1' CRS-2676: Start of 'ora.diskmon' on 'node1' succeeded

24

CRS-2676: Start of 'ora.cssd' on 'node1' succeeded

ASM created and started successfully.

Disk Group DATADG created successfully.

clscfg: -install mode specified

Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful.

CRS-4256: Updating the profile

Successful addition of voting disk 622a76615f464f50bfb1c7e8d719a895. Successfully replaced voting disk group with +DATADG. CRS-4256: Updating the profile

CRS-4266: Voting file(s) successfully replaced

## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- ---------

1. ONLINE 622a76615f464f50bfb1c7e8d719a895 (/dev/asm-diskb) [DATADG] Located 1 voting disk(s).

CRS-2672: Attempting to start 'ora.asm' on 'node1' CRS-2676: Start of 'ora.asm' on 'node1' succeeded

CRS-2672: Attempting to start 'ora.DATADG.dg' on 'node1' CRS-2676: Start of 'ora.DATADG.dg' on 'node1' succeeded Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@node1 ~]#

25

CRS安装成功。

26

[root@node1 ~]# crsctl check crs

CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [root@node1 ~]#

2.2 检查CRS状态

2.2.1 资源组状态:

[root@node1 ~]# crsctl status res -t

--------------------------------------------------------------------------------

NAME TARGET STATE STATE_DETAILS

-------------------------------------------------------------------------------- Local Resources

-------------------------------------------------------------------------------- ora.DATADG.dg

ONLINE ONLINE node1 ONLINE ONLINE node2 ora.LISTENER.lsnr

ONLINE ONLINE node1 ONLINE ONLINE node2 ora.asm

ONLINE ONLINE node1 ONLINE ONLINE node2 ora.gsd

OFFLINE OFFLINE node1 OFFLINE OFFLINE node2 ora.net1.network

ONLINE ONLINE node1 ONLINE ONLINE node2 ora.ons

ONLINE ONLINE node1 ONLINE ONLINE node2 -------------------------------------------------------------------------------- Cluster Resources

-------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr

1 ONLINE ONLINE node1 ora.cvu

1 ONLINE ONLINE node1 ora.node1.vip

1 ONLINE ONLINE node1 ora.node2.vip

SERVER Started Started 27

1 ONLINE ONLINE node2 ora.oc4j

1 ONLINE ONLINE node1 ora.scan1.vip

1 ONLINE ONLINE node1 [root@node1 ~]#

2.2.2 检查集群节点:

[root@node1 ~]# olsnodes -l node1

[root@node1 ~]#

[grid@node2 ~]$ olsnodes -l node2

[grid@node2 ~]$

2.2.3 检查 Oracle 集群注册表 (OCR):

[root@node1 ~]# ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 2368 Available space (kbytes) : 259752

ID : 1943929675 Device/File Name : +DATADG

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded [root@node2 ~]# ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 2368 Available space (kbytes) : 259752

ID : 1943929675 Device/File Name : +DATADG

Device/File integrity check succeeded

28

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

2.2.4 检查表决磁盘:

[root@node1 ~]# crsctl query css votedisk

## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- ---------

1. ONLINE 622a76615f464f50bfb1c7e8d719a895 (/dev/asm-diskb) [DATADG] Located 1 voting disk(s). [root@node1 ~]#

[root@node2 ~]# crsctl query css votedisk

## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- ---------

1. ONLINE 622a76615f464f50bfb1c7e8d719a895 (/dev/asm-diskb) [DATADG] Located 1 voting disk(s). [root@node1 ~]#

2.2.5 检查CRS状态

[root@node1 ~]# crs_stat -t

Name Type Target State Host ------------------------------------------------------------

ora.DATADG.dg ora....up.type ONLINE ONLINE node1 ora....ER.lsnr ora....er.type ONLINE ONLINE node1 ora....N1.lsnr ora....er.type ONLINE ONLINE node1

ora.asm ora.asm.type ONLINE ONLINE node1 ora.cvu ora.cvu.type ONLINE ONLINE node1 ora.gsd ora.gsd.type OFFLINE OFFLINE ora....network ora....rk.type ONLINE ONLINE node1 ora....SM1.asm application ONLINE ONLINE node1 ora....E1.lsnr application ONLINE ONLINE node1 ora.node1.gsd application OFFLINE OFFLINE ora.node1.ons application ONLINE ONLINE node1 ora.node1.vip ora....t1.type ONLINE ONLINE node1

29

ora....SM2.asm application ONLINE ONLINE node2 ora....E2.lsnr application ONLINE ONLINE node2 ora.node2.gsd application OFFLINE OFFLINE ora.node2.ons application ONLINE ONLINE node2 ora.node2.vip ora....t1.type ONLINE ONLINE node2 ora.oc4j ora.oc4j.type ONLINE ONLINE node1 ora.ons ora.ons.type ONLINE ONLINE node1 ora.scan1.vip ora....ip.type ONLINE ONLINE node1 [root@node1 ~]#

三 安装Oracle 软件（不建库）

[oracle@node1 database]$ ./runInstaller

30

36

在两个节点分别执行

/s01/oracle/app/oracle/product/11.2.0/dbhome_1/root.sh脚本。 [root@node1 ~]# /s01/oracle/app/oracle/product/11.2.0/dbhome_1/root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as: ORACLE_OWNER= oracle

ORACLE_HOME= /s01/oracle/app/oracle/product/11.2.0/dbhome_1

Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of \The contents of \The contents of \

Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script.

Now product-specific root actions will be performed. Finished product-specific root actions. [root@node1 ~]#

37

本文来源：https://www.bwwdw.com/article/ef43.html