在vmware vsphere esxi上安装oracle RAC

更新时间:2023-10-31 18:20:01 阅读量: 综合文库 文档下载

说明:文章内容仅供预览,部分内容可能不全。下载后的文档,内容与下面显示的完全一致。下载之前请确认下面内容是否您想要的,是否完整无缺。

Oracle Databases on VMware

RAC 安装手册

Contents

1. Introduction ......................................................... 错误!未定义书签。 2. Infrastructure Environment .................................. 错误!未定义书签。

2.1 Hardware ......................................................................................... 错误!未定义书签。 2.2 Software Environment ..................................................................... 错误!未定义书签。 2.3 VMware Virtual Machines ............................................................... 错误!未定义书签。 2.4 Virtual Disk Layout .......................................................................... 错误!未定义书签。 2.5 Oracle RAC Database Environment .............................................................................. 6 2.6 Network Configuration .................................................................... 错误!未定义书签。 2.7 Physical Architecture ...................................................................... 错误!未定义书签。 2.8 Logical Architecture ........................................................................ 错误!未定义书签。

3. Overview of Deployment Steps ........................... 错误!未定义书签。 4. Configure NTP Client on All ESX Hosts .............. 错误!未定义书签。 5. Prepare a Virtual Machine for the First RAC Node错误!未定义书签。

5.1 Create a Virtual Machine ................................................................ 错误!未定义书签。 5.2 Add Two NICs ................................................................................. 错误!未定义书签。 5.3 Add CRS and Voting Disk ............................................................... 错误!未定义书签。 5.4 Install the Guest Operating System ................................................ 错误!未定义书签。 5.5 Install VMware Tools .................................................................................................... 19 5.6 Timekeeping with the Linux Guest Operating System .................... 错误!未定义书签。

6. Prepare Subsequent Virtual Machines/RAC Nodes错误!未定义书签。

6.1 Clone the First Virtual Machine/RAC Node ................................................................. 28 6.2 Configure CSR and Voting Disks .................................................... 错误!未定义书签。 6.3 Format the Added Disks Using fdisk ............................................................................ 32

7. Install Oracle Grid Infrastructure ..................................................... 33 8. Verify the Grid Infrastructure Installation ......................................... 35 9. Add DATA and REDO Disks to Virtual Machine RAC Nodes .......... 37 10. Create DATA and REDO ASM Disk Groups as Grid User ............ 41 11. Install and Create the Oracle Database 11g R2 RAC ................... 42 12. References ....................................................... 错误!未定义书签。

? 2014 VMware, Inc. All rights reserved.

Page 2 of 42

? 2014 VMware, Inc. All rights reserved.

Page 3 of 42

1. 介绍

本文通过一个案例简要介绍 Oracle Real Application Clusters (RAC)在VMware vSphere平台上的安装过程,帮助Oracle 数据库管理员在VMware 平台上规划和实施Oracle RAC环境.在VMware环境下,数据库管理员通常需要和VMware的管理员以及存储厂商的管理员合作,以顺利部署一个Oracle RAC环境. 在虚拟化平台上部署Oracle RAC在许多方面与纯物理环境上的部署很类似,比如: ? ?

在虚拟机创建完成之后,安装操作系统和Oracle RAC软件的过程与物理环境下基本一致 存储的规划,包括用于数据存放的LUN数量的规划和多路径的设计与物理环境也基本一致.

?

差别在于我们可以充分依靠虚拟化软件, 简化许多工作, 从而加快安装进度,减少错误的发生,比如: ? ? ?

由于安装过程的细节与软件版本(尤其是Linux版本和Oracle版本)有较强的关联性,本文只能尽量把安装步骤描述清楚.读者在自己进行安装的时候,还需要参考相应版本的官方安装文档. 先将本次安装中用到的主要软件和版本罗列如下: ? ? ? ? ?

VMware ESXi?5.1

VMware vCenterServer?5.1 vSphere Client 5.1

Oracle Database 11g 11.2.0.3. Oracle Enterprise Linux x86_64 6.3.

利用事先准备好的模板, 多次安装同版本的系统和数据库,同时加强标准化 利用快照技术, 很方便的进行测试和回退

利用虚拟机复制技术,简化多个RAC节点的系统安装、打补丁、配置工作

? 2014 VMware, Inc. All rights reserved.

Page 4 of 42

2. 软硬件环境

2.1 硬件

组件 ESXiHosts

数量 2

配置 Cisco UCS B440

4x 10-core CPUs, 256GB RAM, and 4 NICs

Storage

Ethernet Switches FC Switches Host Based Adapters

1 2 1 4

NetApp F3240双路控制器 Cisco FI6248 Cisco Nexus 7000

每个刀片两个,与网卡共享流量

2.2 软件环境

组件 vSphere

VMware vCenter?

Oracle Enterprise Linux x86_64

版本 5.1 5.1 6.3

作用 Hypervisor hosting all virtual machines 管理vSphere

Guest operatingsystemforvirtual machines running Oracle Database server

Oracle Database 11g (with Oracle RAC and Oracle Grid Infrastructure) Enterprise Edition Microsoft Windows Server

11.2.0.3

2008 R2 用于vCenter Server

2.3 虚拟机配置

虚拟机用途 Oracle RAC nodes

数量 2

配置 20vCPU, 120GB RAM,

Oracle Enterprise Linux x86_64 6.3

2 vCPUs, 8GB RAM, Windows Server 2008 R2

vCenter

1

2.4 共享存储配置

? 2014 VMware, Inc. All rights reserved.

Page 5 of 42

Datastore Name Virtual Disk on ESX Linux路径 虚拟路径 驱动类型 VMDK Size(GB) 150

用途 VMDATASTORE VMDK –

Hard disk 1

/dev/sda SCSI 0:0 LSI Logic

Oracle Enterprise Linux 5.5 OS and Oracle binaries CRS and voting disk

CRS1

VMDK – Hard disk 2 VMDK – Hard disk 3 VMDK – Hard disk 4 VMDK – Hard disk 5 VMDK – Hard disk 6 VMDK – Hard disk 7

Shared disk SCSI 1:0 Paravirtual 20

/dev/sdb1

Shared disk

SCSI 1:1

Paravirtual

900

REDO and Archieve

REDO1

/dev/sdc1

Shared disk

SCSI 2:0

Paravirtual

1000

RAC database DATA RAC database DATA RAC database DATA RAC database DATA

DATA01

/dev/sdd1

Shared disk

SCSI 2:1

Paravirtual

1000

DATA02

/dev/sde1

Shared disk

SCSI 3:0

Paravirtual

1000

DATA03

/dev/sdf1

Shared disk

SCSI 3:1

Paravirtual

1000

DATA04

/dev/sdg1

2.5 Oracle RAC 数据库环境

Node Name RAC1 RAC2

ASM Instance +ASM1 +ASM2

RAC Instance RAC1 RAC2

Database RAC RAC

ASM Storage ASM file system ASM file system

2.6 网络配置

按照Oracle RAC的实施惯例,每个RAC节点都提供一个Public网口用于承载业务,并在其上捆绑虚拟IP地址 (VIP)用于HA切换..

各RAC节点之间还需要一个私有网络进行节点之间的通信。

在物理环境中,通常需要考虑链路的冗余以消除单点鼓掌。而在虚拟化环境中,冗余问题由底层虚拟化软件处理,所以对于每个RAC节点,只是简单地需要两个网口就可以工作,大大简化了配置和安装工作.

? 2014 VMware, Inc. All rights reserved.

Page 6 of 42

我们这次测试的网络配置如下三: ? ? ?

UCS底层软件配置出三个物理网口(均为10Gb)– vmnic0和 vmnic1用于Oracle Public and Oracle

??

Private网络, vmnic2用于vSpherevMotion工作.

两个分布式虚拟交换机– Oracle Public (VM Net) and Oracle Private (Oracle HB). 每个RAC节点配置3个IP地址– 以节点1 (RAC1)为例: o Onepublic static IP address (eth0):

162.16.3.101

o Oneprivate static IP address (eth1): 192.168.2.30 o Onevirtual static IP address(VIP):

162.16.3.111(由CRS软件配置和管理)

还有三个虚拟IP用于节点间负载均衡(SCAN): 162.16.3.200,162.16.3.201,162.16.3.202(由CRS配置和管理)

物理网口 vmnic0 连接到分布式交换机 VM Net,vmnic1连接到分布式交换机Oracle HB,vMotion使用标准交换机vSwitch3.这样区分概念上比较清晰,也可以只建一个分布式网络,用不同的端口组区分数据网络和心跳网。

Figure 1.Network Configuration

Oracle RAC – Private

? 2014 VMware, Inc. All rights reserved.

Page 7 of 42

? 2014 VMware, Inc. All rights reserved.

Page 8 of 42

2.8 逻辑结构图

Figure 2. Logical Architecture of Two-Node Oracle RAC on vSphere

? 2014 VMware, Inc. All rights reserved.

Page 9 of 42

3. 部署流程概览

部署流程大体按照下面的流程图进行.

Figure 3. Deployment Steps for Four-Node Oracle RAC on vSphere with VMFS

? 2014 VMware, Inc. All rights reserved.

Page 10 of 42

4. 为第一个RAC节点准备虚拟机环境

本节安装步骤简介如下: ? ? ?

使用vSphere Client创建第一个虚拟机。添加公网和心跳网卡,挑选适当的LUN创建仲裁盘CRS (VMDK模式)。

安装Redhat Linux操作系统,安装VMware tools,按照Oracle RAC要求安装相应补丁. 在Linux中配置NTP 客户端.

4.1 创建第一个虚拟机

1. 使用vSphere Client登录vCenter.

2. 选中一台主机,在Getting Started页面单击Create a new virtual machine.

3. In the Create New Virtual Machinewizard, select Typical for the Configuration, and click Next. 4. On the Name and Location page, enter RAC1 for the Name, and click Next. 5. On the Datastore page, select VMDATASTORE, and click Next.

6. On the Guest Operating System page, select Linux from the Guest Operating System list. For the

Version, select Oracle Linux 6 (64-bit) from the drop-down menu. Click Next. 7. On the Create a Disk page, set the Virtual Disk Size to 20GB, and click Next.

8. On the Ready to Complete page, select Edit the virtual machine settings before completion.

Click Continue. 9. On the Virtual Machine Properties page for theRAC1 virtual machine, select 8CPUs and

128GBMemory. 10. Click Finish.

4.2 增加两个虚拟网卡

现在需要增加两块网卡. 一块用于公共数据网络, 另一块用于心跳网络.添加这两块网卡的过程基本相同,差别仅在于添加公网网卡时选择标签:VM Net而添加心跳网卡时选择Oracle HB. 添加公网网卡的步骤: 1. Select a virtual machine.

2. Click Edit settings to display the Virtual Machine Properties pane. 3. Click Add.

4. Select Ethernet Adaptor and click Next to display the Add Hardware wizard.

5. On the Network connection page, select VMXNET 3 from the Adapter Type drop-down menu. 6. In the Network Connection section of the Network connection page, select Named network with

specified label. 7. In the Named network with specified label drop-down menu, select VM Net.

8. In the Device Status section of the Network connection page, select Connect at power on. 9. On the Ready to Complete page, click Finish, and then click OK. 添加心跳网卡的步骤:

1. Repeat steps 1 through 6 from the preceding procedure.

? 2014 VMware, Inc. All rights reserved.

Page 11 of 42

2. In the Named network with specified label drop-down menu, select Oracle HB. 3. Continue with steps 8 and 9 from the preceding procedure.

4.3 添加仲裁盘

Oracle RAC需要至少一个共享的仲裁盘存放clusterware信息和节点投票信息。存储管理员需要事先为仲裁盘分配好单独的LUN,然后我们需要创建单独的SCSI控制器并设为Paravirtual驱动.SCSI 总线sharing policy 设为none.

下面描述创建共享存储的步骤. 为虚拟机创建20GB datastore:

1. In vSphereClient, select a virtual machine.

2. Right-click the name of the virtual machine and select Edit Settings to display the Virtual Machine

Properties pane. 3. Click Addto display the Add Hardware wizard.

4. In the Add Hardware wizard select Hard Disk for the Device Type, and click Next.

5. In the Disk section of the Select a Disk page, select Create a new virtual disk. Set the disk size to

20GB, select the Support clustering features such as Fault Tolerance check box, and select CRS1 to specify a datastore. Also, for vSphere 5, in the Disk Provisioning section select Thick Provision Eager Zeroed. Click Next. 6. On the Compatibility Mode page, click Next.

7. On the Advanced Options page, in the Virtual Device Node section, select SCSI (1:0). In

theMode section, selectIndependent. Click Next. 8. On the Ready to Complete page, click Finish. 9. Click OK.

Add configuration parameters for each Oracle RAC virtual machine as described in Disabling simultaneous write protection provided by VMFS using the multi-writer flag(http://kb.vmware.com/kb/1034165). 手工修改虚拟机参数

1. In vSphereClient, select a virtual machine.

2. Right-click the name of the virtual machine and select Edit Settings to display the Virtual Machine

Properties pane. 3. Click the Options tab, click Advanced, and click General. 4. Click Configuration Parameters.

5. On the Configuration Parameters page, add rows and entries for the shared disks in the virtual

machine's configuration parameters as shown in the following screenshot. 6. Repeat steps 1 through 5 for each of the remaining virtual machines.

? 2014 VMware, Inc. All rights reserved.

Page 12 of 42

? 2014 VMware, Inc. All rights reserved.

Page 13 of 42

You can view the virtual machine propertiesfor the RAC1 virtual machinein vCenteras in the following screenshot.

web client 显示如下图:

? 2014 VMware, Inc. All rights reserved.

Page 14 of 42

? 2014 VMware, Inc. All rights reserved.

Page 15 of 42

4.4 安装操作系统

将OEL6.4安装介质ISO文件上传到datastore,然后进行安装。

Note: This document is specific to Linux guest operating system. Oracle RAC 11.2.0.2 and above

versions can be deployed and supported on the list ofguest operation systems usingVMware multi-writer flag(http://kb.vmware.com/kb/1008027).

在虚拟机上安装Linux操作系统

1. In vSphereClient, select a virtual machine. 2. Click Edit settings.

3. In the Hardware tab of the Virtual Machine Properties pane, selectCD/DVD Drive 1. 4. In the Device Status section of this pane, select the Connect at power on check box.

5. In the Device Type section of this pane, select Datastore ISO File and enter the path to your Linux

ISO image file. 6. In the Virtual Device Node section of this pane, select CD/DVD Drive 1 from the drop-down

menu. 7. Click OK.

8. Power on the virtual machine.

9. Install Linux with the following settings.

Custom partitioning scheme:

/dev/sda1 ext3 150GB mounted on /

Installed packages – use the default package selection, plus the following: ? ? ?

kernel-uek.x86_64 kmod-oracleasm

oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64 (download from

http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html.) ? ?

oracleasm-support-2.1.3-1.el5.x86_64 systat-7.0.2-3.el5.x86_64

10. Use below script to install remaining packages:

yuminstall binutils \\ compat-libcap1 \\

compat-libstdc++-33 \\

compat-libstdc++-33.i686 \\ gcc.x86_64 \\

gcc-c++.x86_64 \\ glibc.x86_64 \\ glibc.i686 \\

glibc-devel.x86_64 \\ glibc-devel.i686 \\ unixODBC.i686 \\ unixODBC.x86_64 \\

unixODBC-devel.x86_64 \\ unixODBC-devel.i686 \\ libgcc.x86_64 \\ libgcc.i686 \\

? 2014 VMware, Inc. All rights reserved.

Page 16 of 42

libstdc++.x86_64 \\ libstdc++.i686 \\

libstdc++-devel.x86_64 \\ libstdc++-devel.i686 \\

compat-libstdc++-33.x86_64 \\ compat-libstdc++-33.i686 \\ libaio.x86_64 \\ libaio.i686 \\

libaio-devel.x86_64 \\ libaio-devel.i686 \\ sysstat.x86_64 \\ ksh \\

libXtst-devel.x86_64 \\ libXtst.i686 \\

compat-db.x86_64 \\

xorg-x11-utils.x86_64 \\ xorg-x11-xinit.x86_64 \\ make

? 2014 VMware, Inc. All rights reserved.

Page 17 of 42

11. View the installed packages with this command:

run command “rpm –q

12. Confirm that the following packages have been installed:

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

binutils-2.20.51.0.2-5.11.el6 (x86_64) compat-libcap1-1.10-1 (x86_64)

compat-libstdc++-33-3.2.3-69.el6 (x86_64) compat-libstdc++-33-3.2.3-69.el6.i686 gcc-4.4.4-13.el6 (x86_64) gcc-c++-4.4.4-13.el6 (x86_64) glibc-2.12-1.7.el6 (i686) glibc-2.12-1.7.el6 (x86_64) glibc-devel-2.12-1.7.el6 (x86_64) glibc-devel-2.12-1.7.el6.i686 ksh

libgcc-4.4.4-13.el6 (i686) libgcc-4.4.4-13.el6 (x86_64) libstdc++-4.4.4-13.el6 (x86_64) libstdc++-4.4.4-13.el6.i686

libstdc++-devel-4.4.4-13.el6 (x86_64) libstdc++-devel-4.4.4-13.el6.i686

? ? ? ? ? ? ? ? ? ? ?

libaio-0.3.107-10.el6 (x86_64) libaio-0.3.107-10.el6.i686

libaio-devel-0.3.107-10.el6 (x86_64) libaio-devel-0.3.107-10.el6.i686 make-3.81-19.el6

sysstat-9.0.4-11.el6 (x86_64)

unixODBC-2.2.14-11.el6 (x86_64) or later unixODBC-2.2.14-11.el6.i686 or later unixODBC-devel-2.2.14-11.el6 (x86_64) or later

unixODBC-devel-2.2.14-11.el6.i686 or later

? 2014 VMware, Inc. All rights reserved.

Page 18 of 42

13. 最后安装下面两个包:

? ?

oracle-rdbms-server-11gR2-preinstall (从YUM安装,这个包会配置Oracle RAC所需系统参数和环境变量)

cvuqdisk (Oracle安装介质中提供这个包,用于安装前检查)

14. 检查配置文件 /etc/sysctl.conf

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

fs.file-max = 6815744 (512 x processes for example 6815744 for 13312 processes) net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576

kernel.shmall = physical RAM size / pagesizeFor most systems, this will be the value 2097152. See Note: 301830.1 for more information. eg:

kernel.shmall=4160749568(62GB) kernel.shnmax= 34359738368(32GB)

kernel.shmmax = RAM times 0.5 (or higher at customer's discretion - seeNote:567506.1)

/etc/profile

ulimit –u 16384 ulimit –n 65535

15. 依照oracle传统,创建clusterware主用户 “grid”:

[root@racnode1 ~]# groupadd -g 1000 oinstall [root@racnode1 ~]# groupadd -g 1200 asmadmin [root@racnode1 ~]# groupadd -g 1201 asmdba [root@racnode1 ~]# groupadd -g 1202 asmoper

[root@racnode1 ~]# useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -d /home/grid -s /bin/bash -c \

[root@racnode1 ~]# id griduid=1100(grid) gid=1000(oinstall)

groups=1000(oinstall), 1200(asmadmin), 1201(asmdba),1202(asmoper) 设置 grid 帐户的口令:

[root@racnode1 ~]# passwd grid

Changing password for user grid. New UNIX password:xxxxxxxxxxx

Retype new UNIX password:xxxxxxxxxxx

passwd: all authentication tokens updated successfully.

为grid用户创建初始化脚本(/home/grid/.bash_profile):

# --------------------------------------------------- # .bash_profile

# --------------------------------------------------- # OS User: grid

# Application: Oracle Grid Infrastructure # Version: Oracle 11g release 2

? 2014 VMware, Inc. All rights reserved.

Page 19 of 42

# ---------------------------------------------------

# Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi

alias ls=\

# --------------------------------------------------- # ORACLE_SID

# --------------------------------------------------- # Specifies the Oracle system identifier (SID)

# for the Automatic Storage Management (ASM)instance # running on this node.

# Each RAC node must have a unique ORACLE_SID. # (i.e. +ASM1, +ASM2,...)

# --------------------------------------------------- ORACLE_SID=+ASM1; export ORACLE_SID

# --------------------------------------------------- # JAVA_HOME

# --------------------------------------------------- # Specifies the directory of the Java SDK and Runtime # Environment.

# --------------------------------------------------- JAVA_HOME=/usr/local/java; export JAVA_HOME

# --------------------------------------------------- # ORACLE_BASE

# --------------------------------------------------- # Specifies the base of the Oracle directory structure # for Optimal Flexible Architecture (OFA) compliant # installations. The Oracle base directory for the # grid installation owner is the location where

# diagnostic and administrative logs, and other logs # associated with Oracle ASM and Oracle Clusterware # are stored.

# --------------------------------------------------- ORACLE_BASE=/u01/app/grid; export ORACLE_BASE

# --------------------------------------------------- # ORACLE_HOME

# --------------------------------------------------- # Specifies the directory containing the Oracle # Grid Infrastructure software. For grid

# infrastructure for a cluster installations, the Grid # home must not be placed under one of the Oracle base # directories, or under Oracle home directories of # Oracle Database installation owners, or in the home # directory of an installation owner. During

# installation, ownership of the path to the Grid # home is changed to root. This change causes # permission errors for other installations.

# --------------------------------------------------- ORACLE_HOME=/u01/app/11.2.0/grid; export ORACLE_HOME

# --------------------------------------------------- # ORACLE_PATH

# --------------------------------------------------- # Specifies the search path for files used by Oracle # applications such as SQL*Plus. If the full path to # the file is not specified, or if the file is not # in the current directory, the Oracle application

? 2014 VMware, Inc. All rights reserved.

Page 20 of 42

9. 以grid用户创建DATA和REDO磁盘组

为安装数据库做准备

1. 使用oracleasm创建4个数据盘DATA1到DATA4,使用的设备为/dev/sdd1到/dev/sdg1。用设备

/dev/sdc1创建REDO磁盘REDO1. 2. 以grid用户登录节点1(RAC1),启动ASM配置助手(asmca),创建两个磁盘组DATADG (选择所有

DATA标签盘DATA1到DATA4) 和REDODG(选择磁盘 REDO1). 3. In the ASM Configuration Assistant, on the Create Disk Group page, enter DATADG for the Disk

Group Name. In the Redundancy section, select External (None). Click OK, and click OK again. 4. On the Create Disk Group page, in the Select Member Disks section, select ORCL:DATA1,

ORCL:DATA2, ORCL:DATA3, and ORCL:DATA4. Set the size of each to 1000GB. Click OK. 5. On the Create Disk Group page, enter REDODG for the Disk Group Name. In the Redundancy

section, select External (None). Click OK, and click OK again. 6. On the Create Disk Group page, in the Select Member Disks section, select ORCL:REDO1, ,. Set

the size of each to 900GB. Click OK.

? 2014 VMware, Inc. All rights reserved.

Page 41 of 42

10. 安装 RAC数据库

To install and create the Oracle Database 11gR2 RAC

1. Login into RAC1 as oracle user and launch runInstallerfromthe Oracle software location to install

the Oracle binaries. 2. In the Oracle Installer wizard, on the Configure Security Updates page, leave the Email field blank,

and do not select I wish to receive security updates via My Oracle Support. Click Next. 3. On the Installation Option page, select Install database software only.Click Next.

4. On the Grid Options page, select Real Application Clusters database installation. Select the four

nodes RAC1, RAC2.Click Next. 5. On the Product Languages page, select English.Click Next.

6. On the Database Edition page, select Enterprise Edition (4.29GB).Click Next. 7. On the Installation Location page, enter /u01/app/oracle for Oracle Base, and

/u01/app/oracle/product/11.2.0/dbhome_1 for Software Location.Click Next. 8. On the Operating System Groups page, select dba for the Database Administrator (OSDBA)

Group and oinstall for the Database Operator (OSOPER) Group. Click Next. 9. Review the configuration on the Summary page, and click Finish.

10. After the installation is complete, open a terminal window, log in as root, and run the following script:

/u01/app/oracle/product/11.2.0/dbhome_1/root.sh Repeat for each of the four nodes, and click OK. 11. On the Finish page, click Close.

12. Create the Oracle RAC database using Oracle Database Configuration Assistant (DBCA).

? 2014 VMware, Inc. All rights reserved.

Page 42 of 42

# uses ORACLE_PATH to locate the file.

# This variable is used by SQL*Plus, Forms and Menu. # ---------------------------------------------------

ORACLE_PATH=/u01/app/oracle/common/oracle/sql; export ORACLE_PATH

# --------------------------------------------------- # SQLPATH

# --------------------------------------------------- # Specifies the directory or list of directories that # SQL*Plus searches for a login.sql file.

# --------------------------------------------------- # SQLPATH=/u01/app/common/oracle/sql; export SQLPATH

# --------------------------------------------------- # ORACLE_TERM

# --------------------------------------------------- # Defines a terminal definition. If not set, it # defaults to the value of your TERM environment # variable. Used by all character mode products.

# --------------------------------------------------- ORACLE_TERM=xterm; export ORACLE_TERM

# --------------------------------------------------- # NLS_DATE_FORMAT

# --------------------------------------------------- # Specifies the default date format to use with the # TO_CHAR and TO_DATE functions. The default value of # this parameter is determined by NLS_TERRITORY. The # value of this parameter can be any valid date

# format mask, and the value must be surrounded by # double quotation marks. For example:

# # NLS_DATE_FORMAT = \

# # ---------------------------------------------------

NLS_DATE_FORMAT=\

# --------------------------------------------------- # TNS_ADMIN

# --------------------------------------------------- # Specifies the directory containing the Oracle Net # Services configuration files like listener.ora, # tnsnames.ora, and sqlnet.ora.

# --------------------------------------------------- TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

# --------------------------------------------------- # ORA_NLS11

# --------------------------------------------------- # Specifies the directory where the language,

# territory, character set, and linguistic definition # files are stored.

# --------------------------------------------------- ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

# --------------------------------------------------- # PATH

# --------------------------------------------------- # Used by the shell to locate executable programs; # must include the $ORACLE_HOME/bin directory.

# ---------------------------------------------------

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin PATH=${PATH}:/u01/app/common/oracle/bin export PATH

? 2014 VMware, Inc. All rights reserved.

Page 21 of 42

# --------------------------------------------------- # LD_LIBRARY_PATH

# --------------------------------------------------- # Specifies the list of directories that the shared # library loader searches to locate shared object # libraries at runtime.

# --------------------------------------------------- LD_LIBRARY_PATH=$ORACLE_HOME/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib export LD_LIBRARY_PATH

# --------------------------------------------------- # CLASSPATH

# --------------------------------------------------- # Specifies the directory or list of directories that # contain compiled Java classes.

# --------------------------------------------------- CLASSPATH=$ORACLE_HOME/JRE

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib export CLASSPATH

# --------------------------------------------------- # THREADS_FLAG

# --------------------------------------------------- # All the tools in the JDK use green threads as a # default. To specify that native threads should be # used, set the THREADS_FLAG environment variable to # \# threads by setting THREADS_FLAG to the value # \

# --------------------------------------------------- THREADS_FLAG=native; export THREADS_FLAG

# --------------------------------------------------- # TEMP, TMP, and TMPDIR

# --------------------------------------------------- # Specify the default directories for temporary # files; if set, tools that create temporary files # create them in one of these directories.

# --------------------------------------------------- export TEMP=/tmp export TMPDIR=/tmp

# --------------------------------------------------- # UMASK

# --------------------------------------------------- # Set the default file mode creation mask

# (umask) to 022 to ensure that the user performing # the Oracle software installation creates files # with 644 permissions.

# --------------------------------------------------- umask 022

16. 创建oracle账号 “oracle”:

[root@racnode1 ~]# groupadd -g 1300 dba [root@racnode1 ~]# groupadd -g 1301 oper

[root@racnode1 ~]# useradd -m -u 1101 -g oinstall -G dba,oper,asmdba -d /home/oracle -s /bin/bash -c \

? 2014 VMware, Inc. All rights reserved.

Page 22 of 42

[root@racnode1 ~]# id oracle uid=1101(oracle) gid=1000(oinstall)

groups=1000(oinstall),1201(asmdba),1300(dba),1301(oper)

然后为oracle用户创建登录环境(/home/oracle/.bash_profile)

# --------------------------------------------------- # .bash_profile

# --------------------------------------------------- # OS User: oracle

# Application: Oracle Database Software Owner # Version: Oracle 11g release 2

# ---------------------------------------------------

# Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi

alias ls=\

# --------------------------------------------------- # ORACLE_SID

# --------------------------------------------------- # Specifies the Oracle system identifier (SID) for # the Oracle instance running on this node. # Each RAC node must have a unique ORACLE_SID. # (i.e. racdb1, racdb2,...)

# --------------------------------------------------- ORACLE_SID=racdb1; export ORACLE_SID

# --------------------------------------------------- # ORACLE_UNQNAME

# --------------------------------------------------- # In previous releases of Oracle Database, you were # required to set environment variables for

# ORACLE_HOME and ORACLE_SID to start, stop, and # check the status of Enterprise Manager. With

# Oracle Database 11g release 2 (11.2) and later, you # need to set the environment variables ORACLE_HOME # and ORACLE_UNQNAME to use Enterprise Manager. # Set ORACLE_UNQNAME equal to the database unique # name.

# --------------------------------------------------- ORACLE_UNQNAME=racdb; export ORACLE_UNQNAME

# --------------------------------------------------- # JAVA_HOME

# --------------------------------------------------- # Specifies the directory of the Java SDK and Runtime # Environment.

# --------------------------------------------------- JAVA_HOME=/usr/local/java; export JAVA_HOME

# --------------------------------------------------- # ORACLE_BASE

# --------------------------------------------------- # Specifies the base of the Oracle directory structure # for Optimal Flexible Architecture (OFA) compliant # database software installations.

# --------------------------------------------------- ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

? 2014 VMware, Inc. All rights reserved.

Page 23 of 42

# --------------------------------------------------- # ORACLE_HOME

# --------------------------------------------------- # Specifies the directory containing the Oracle # Database software.

# ---------------------------------------------------

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1; export ORACLE_HOME

# --------------------------------------------------- # ORACLE_PATH

# --------------------------------------------------- # Specifies the search path for files used by Oracle # applications such as SQL*Plus. If the full path to # the file is not specified, or if the file is not # in the current directory, the Oracle application # uses ORACLE_PATH to locate the file.

# This variable is used by SQL*Plus, Forms and Menu. # ---------------------------------------------------

ORACLE_PATH=/u01/app/common/oracle/sql; export ORACLE_PATH

# --------------------------------------------------- # SQLPATH

# --------------------------------------------------- # Specifies the directory or list of directories that # SQL*Plus searches for a login.sql file.

# --------------------------------------------------- # SQLPATH=/u01/app/common/oracle/sql; export SQLPATH

# --------------------------------------------------- # ORACLE_TERM

# --------------------------------------------------- # Defines a terminal definition. If not set, it # defaults to the value of your TERM environment # variable. Used by all character mode products.

# --------------------------------------------------- ORACLE_TERM=xterm; export ORACLE_TERM

# --------------------------------------------------- # NLS_DATE_FORMAT

# --------------------------------------------------- # Specifies the default date format to use with the # TO_CHAR and TO_DATE functions. The default value of # this parameter is determined by NLS_TERRITORY. The # value of this parameter can be any valid date

# format mask, and the value must be surrounded by # double quotation marks. For example:

# # NLS_DATE_FORMAT = \

# # ---------------------------------------------------

NLS_DATE_FORMAT=\

# --------------------------------------------------- # TNS_ADMIN

# --------------------------------------------------- # Specifies the directory containing the Oracle Net # Services configuration files like listener.ora, # tnsnames.ora, and sqlnet.ora.

# --------------------------------------------------- TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

# --------------------------------------------------- # ORA_NLS11

# --------------------------------------------------- # Specifies the directory where the language,

? 2014 VMware, Inc. All rights reserved.

Page 24 of 42

# territory, character set, and linguistic definition # files are stored.

# --------------------------------------------------- ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

# --------------------------------------------------- # PATH

# --------------------------------------------------- # Used by the shell to locate executable programs; # must include the $ORACLE_HOME/bin directory.

# ---------------------------------------------------

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin PATH=${PATH}:/u01/app/common/oracle/bin export PATH

# --------------------------------------------------- # LD_LIBRARY_PATH

# --------------------------------------------------- # Specifies the list of directories that the shared # library loader searches to locate shared object # libraries at runtime.

# --------------------------------------------------- LD_LIBRARY_PATH=$ORACLE_HOME/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib export LD_LIBRARY_PATH

# --------------------------------------------------- # CLASSPATH

# --------------------------------------------------- # Specifies the directory or list of directories that # contain compiled Java classes.

# --------------------------------------------------- CLASSPATH=$ORACLE_HOME/JRE

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib export CLASSPATH

# --------------------------------------------------- # THREADS_FLAG

# --------------------------------------------------- # All the tools in the JDK use green threads as a # default. To specify that native threads should be # used, set the THREADS_FLAG environment variable to # \# threads by setting THREADS_FLAG to the value # \

# --------------------------------------------------- THREADS_FLAG=native; export THREADS_FLAG

# --------------------------------------------------- # TEMP, TMP, and TMPDIR

# --------------------------------------------------- # Specify the default directories for temporary # files; if set, tools that create temporary files # create them in one of these directories.

# --------------------------------------------------- export TEMP=/tmp export TMPDIR=/tmp

# --------------------------------------------------- # UMASK

? 2014 VMware, Inc. All rights reserved.

Page 25 of 42

# --------------------------------------------------- # Set the default file mode creation mask

# (umask) to 022 to ensure that the user performing # the Oracle software installation creates files # with 644 permissions.

# --------------------------------------------------- umask 022

17. 创建RAC安装所需目录结构:

# mkdir -p /u01/app/11.2.0/grid

# chowngrid:oinstall /u01/app/11.2.0/grid # chmod -R 775 /u01/app/11.2.0/grid # mkdir -p /u01/app/oracle

# chown -R oracle:oinstall /u01/app/oracle # chmod -R 775 /u01/app/oracle

4.5 安装 VMware Tools

VMware Tools 是VMware为虚拟机操作系统提供的工具包,用于提升虚拟机性能和方便对虚拟机的监控和管理。VMware Tools 必须在操作系统内安装。其安装步骤可以参考以下知识库文档:Installing VMware Tools in a Linux virtual machine using Red Hat Package Manager (RPM)(http://kb.vmware.com/kb/1018392). To install VMware Tools

1. Confirm that the Linux virtual machine is powered on.

2. In vSphereClient, right-click the virtual machine and selectGuest>Install VMware Tools. 3. In the Linux guest OS,create a mount point and run the following command:

mkdir /mnt/cdrom

4. Mount the CD-ROM with the following command:

mount /dev/cdrom /mnt/cdrom

5. Install VMware Tools using RPM, with the following command:

rpm -ivh /mnt/cdrom/VMwareTools-8.3.2-257589.rpm

As an example, “8.3.2-257589” is the version of VMware Tools used here. 6. Configure VMware Tools, by running the following command:

/usr/bin/vmware-config-tools.pl 7. Unmountthe CD-ROM with the following command:

umount /mnt/cdrom

8. Click VM in the virtual machine menu, and click Guest>End VMware Tools Install.

? 2014 VMware, Inc. All rights reserved.

Page 26 of 42

4.6 时钟同步

按照VMware官方文档Timekeeping best practices for Linux guests(http://kb.vmware.com/kb/1006427) 来配置在Linux上配置NTP,同时停掉虚拟机中VMware tools与ESX主机的时钟同步. 有两种方法停止VMware tools的时钟同步进程:

? Set tools.syncTime = \ in the configuration file (.vmx file) of the virtual machine. ? Run the vmware-guestd --cmd \ command in the guest

operating system.

编辑/etc/sysconfig/ntpd 增加 -x 选项,如下所示:

# Drop root to id 'ntp:ntp' by default.

OPTIONS=\

/etc/ntp.conf示例:

tinker panic 0

driftfile /var/lib/ntp/drift statisticsloopstats statsdir /var/log/ntp/

logfile /var/log/ntp/ntpd.log

filegenpeerstats file peers type day link enable filegenloopstats file loops type day link enable

server 162.16.3.33

? 2014 VMware, Inc. All rights reserved.

Page 27 of 42

5. 准备其他虚拟机和RAC节点

现在第一个RAC节点(RAC1)所用的虚拟机就创建好了,现在可以用这个虚拟机克隆出其他RAC节点所用的虚拟机,如果前面的步骤都执行正确,现在就可以把这个虚拟机制成模板,留作以后使用。大概步骤如下: ? ? ? ?

In vCenter, clone RAC1.

在新创建的虚拟机里删除RAC1所带的共享磁盘。

重新添加仲裁盘,仲裁盘同样需要开启multi-writer开关。 修改客户机IP地址.

5.1 虚机克隆

1. RAC1停机

2. In vSphereClient, select the virtual machine, RAC1.

3. Right-click the name of the virtual machine and select Clone.

4. In the Clone Virtual Machine wizardon the Name and Location page, enter RAC2 for the Name.

Click Next. 5. On the Host / Cluster page, select a host or cluster. Click Next.

6. On the Specific Host page, select a unique ESX host (that is, one not running a RAC node virtual

machine). Click Next. 7. On the Datastore page, select a VMFS datastore. Click Next.

8. On the Disk Format page, select Same format as source. Click Next.

9. On the Guest Customization page, select customize using the customization wizard. 10. config IP address for eth0 and eth1

? 2014 VMware, Inc. All rights reserved.

Page 28 of 42

11. On the Ready to Complete page, click Finish.

? 2014 VMware, Inc. All rights reserved.

Page 29 of 42

5.2 配置仲裁盘

虚拟机克隆之后,指向仲裁盘的路径会失效,所以需要先在虚机配置里删除这个硬盘重建,同时把SCSI控制器类型改为paravirtual. 按照下述步骤配置RAC2: 删除共享盘

1. In vSphereClient, select a virtual machine.

2. Right-click the name of the virtual machine and select Edit Settings to display the Virtual Machine

Properties pane. 3. Select hard disk 2 and click Remove. In the Removal Options section, select Remove from virtual

machine and click OK. 再新建一次

1. In vSphereClient, select a virtual machine.

2. Right-click the name of a virtual machine and select Edit Settings to display the Virtual Machine

Properties pane. 3. Click Addto display the Add Hardware wizard.

4. In the Add Hardware wizard select Hard Disk for the Device Type, and click Next.

5. In the Disk section of the Select a Disk page, select Use an existing virtual disk and click Next. 6. On the Select Existing Disk page, enter the path to the first VMDK of the virtual machine (selected

in step 1) used for the CRS and voting datastores. Click Next. 7. On the Advanced Options page, in the Virtual Device Node section, select SCSI (1:0). In

theMode section, select Independent and Persistent. Click Next. 8. On the Ready to Complete page, click Finish. 9. Click OK.

? 2014 VMware, Inc. All rights reserved.

Page 30 of 42

修改SCSI控制器类型为paravirtual

1. In vSphereClient, select the virtual machine that was selected in the preceding step 1.

2. Right-click the name of the virtual machine and select Edit Settings to display the Virtual Machine

Properties pane. 3. In the device list, select SCSI controller 1.

4. In the SCSI Controller Type section, click Change Type. 5. Select VMware Paravirtual. 6. Click OK, and click OK again.

? 2014 VMware, Inc. All rights reserved.

Page 31 of 42

5.3 使用fdisk格式化仲裁盘:

必须在一个节点上对共享盘进行分区和格式化。在Linux上可以使用系统自带的fdisk工具进行 。步骤如下:

仲裁盘分区:

1. Login as root on node1. 2. Enter the command:

fdisk /dev/sdb

3. Enter n to create a new partition. 4. Enter p to create a primary partition. 5. Enter 1 to create the first partition.

6. Press Enter to take the default value of 1 for the first cylinder. 7. Press Enter to take the default value of 20480 for the last cylinder. 8. Enter w to write the new partition table to disk and exit fdisk.

The output from the fdisk command should like somewhat like the following:

root@vmorarac1 ~# fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n Command action e extended

p primary partition (1-4) p

Partition number (1-4): 1

First cylinder (1-20480, default 1): Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-20480, default 20480): Using default value 20480 Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table. Syncing disks. ? 2014 VMware, Inc. All rights reserved.

Page 32 of 42

6. 安装Oracle Grid软件

现在我们有两个虚机,可以开始安装grid环境了. 安装Oracle Grid Infrastructure

1. 用oracleasm在/dev/sdb1创建griddisk CRS1. 2. 将Oracle RAC安装包(11.2.0.3)上传到RAC1并解压 3. 切换到grid用户启动Oracle Grid安装程序 ./runInstaller

4. 在第一页安装选项页,选择Install and Configure Infrastructure for a Cluster. Click Next. 5. On the Installation Type page, select Advanced Installation and click Next. 6. On the Product Languages page, select English and click Next.

7. On the Grid Plug and Play page, enter rac for the Cluster Name,

rac-scan.localdomainfor the SCAN Name and 1521 for the SCAN Port. Click Next. 8. On the Cluster Node Information page, click Add to enter aHostname and Virtual IP Name.

Repeat the process for each of the four hosts, as follows:

Hostname RAC1 RAC2

9. Click Next.

10. On the Network Interface Usage page, enter the following information for the Interface Name,

Subnet, and Interface Type:

Interface Name eth0 eth1

11. Click Next.

12. On the Storage Option page, select Automatic Storage Management (ASM), and click Next. 13. On the Create ASM Disk Group page, enter CRS for the Disk Group Name, and select Normal for

Redundancy. 14. In the Add Disks section of the Create ASM Disk Group page, select Candidate Disks. Make the

following entries:

Disk Path ORCL:CRSDISK1

15. Select the check box to the left of each of these entries, and click Next.

? 2014 VMware, Inc. All rights reserved.

Page 33 of 42

Size (in MB) 20479

Status Candidate

Subnet 162.16.3.0 192.168.2.0

Interface Type Public Private

Virtual IP Name RAC1-vip RAC2-vip

16. On the ASM Password page, select Use same passwords for these accounts and enter the

password “oracle” in the Specify Password and Confirm Password fields. Click Next. 17. On the Failure Isolation page, select Do not use Intelligent Platform Management Interface

(IPMI). Click Next. 18. On the Operating System Groups page, make the following entries:

Field ASM Database Administrator (OSDBA) Group

ASM Instance Administration Operator (OSOPER) Group ASM Instance Administrator (OSASM) Group

19. Click Next.

20. On the Installation Location page, enter /u01/app/grid for the Oracle Base, and

/u01/app/11.2.0/grid for the Software Location. Click Next

21. On the Create Inventory page, enter /u01/app/orainventory for the Inventory Directory. Click

Next. 22. Wait while the setup application conducts the prerequisite checks and installs Grid on each node.

After setup is complete, execute the following configuration scripts on the four nodes

VMORARAC1,VMORARAC2. These scripts must be executed in a terminal window as root.

/u01/app/orainventory/orinstRoot.sh /u01/app/11.2.0/grid/root.sh

Note: don’t run root.sh scripts on 2 nodes at the same time, or else It will fail. 23. Click OK, and click Finish.

Selection asmdba asmoper asmadmin

? 2014 VMware, Inc. All rights reserved.

Page 34 of 42

7. 验证Grid安装的正确性

The following figures show how to run various RAC commands in the guest OS to check the status of the cluster.

Figure 4. Check the CRS Status on the First Node as Grid User

Figure 5.Check for Cluster Nodes

? 2014 VMware, Inc. All rights reserved.

Page 35 of 42

本文来源:https://www.bwwdw.com/article/qpd2.html

Top