RHCS+ORACLE10G安装

RHCS+ORACLE安装

一、简介

二、RHCS原理

三、RHCS安装 1、 安装环境

2、 所要安装的RPM包 3、 共享磁盘的配置 4、 时间同步的设置 5、 设置/etc/hosts 6、 配置RHCS

四、ORACLE安装 1、 安装前的准备

2、 Oracle 安装与测试 3、 各节点的配置 4、 完成集群和起动服务

五、需要注意

六、RHCS及ORACLE的管理

一、简介

Linux集群主要分成三大类( 高可用集群， 负载均衡集群，科学计算集群)

高可用集群( High Availability Cluster) 负载均衡集群(Load Balance Cluster)

科学计算集群(High Performance Computing Cluster)

================================================ 这里只介绍高可用集群

高可用集群(High Availability Cluster)

常见的就是2个节点做成的HA集群，有很多通俗的不科学的名称，比如\双机热备\\双机互备\双机\

高可用集群解决的是保障用户的应用程序持续对外提供服务的能力。 (请注意高可用集群既不是用来保护业务数据的，保护的是用户的业务程序对外不间断提供服务，把因软件/硬件/人为造成的故障对业务的影响降低到最小程度)

RedHat Linux平台下的高可用集群是指:

RedHat 公司 的 开放源代码RedHat Cluster Suite, 简称RHCS . (产品介绍http://www.redhat.com/solutions/clustersuite/)

RHCS主要包括用于High Availability的Red Hat Cluster Manager和用于Load Balance的Linux Virtual Server。这里我们只介绍Red Hat Cluster Manager。

RHCS的获得:

RHCS的获得有三个途径:

a. 从redhat 的官方销售渠道购买，得到的是RedHat RHCS产品的二进制代码(光盘或直接从RHN-RedHat Network下载)，源代码，以及支持和升级服务.

b. 从redhat 的官方站点下载SRPM包，编译并安装. 无原厂商业支持和升级服务. 这里下

载

(ftp://ftp.redhat.com/pub/redhat/linux/enterprise/4/en/RHCS

和

ftp://ftp.redhat.com/pub/redhat/linux/enterprise/3/en/RHCS)

c. 从centOS官方站点下载RHCS的二进制代码包并安装，无原厂商业支持和升级服务. 这里下载(http://mirror.centos.org/centos/4/csgfs/, 请阅读此目录中的README文件)

由于我先择了centOS4.4做为测试平台，所以这里选择了c这种方法获得RHCS二进制安装包。

二、RHCS原理

目前常用的集群环境框架

常用服务(如oracle,mysql,httpd等)

|

|

集群软件（rhcs,clusterware, Novell Cluster Service等）

|

|

操作系统（Red Hat,debian,Suse等）

这里我们用到的是：oracle?rhcs?Red Hat。

- RHCS实际上是一个套件，除了现在提到的高可用集群软件，还包含了一个基于LVS的负载均衡软件.

- RHCS包含的集群软件，是GPL协议下开放源代码的集群系统.

- RHCS目前的版本有 RHCS v3和 v4,分别对应于RedHat 企业版Linux的 3.0家族和4.0家族.

- RHCS是一个独立的产品，不包括在RedHat Enterprise Linux当中，无论是购买还是下载，都是单独存在的.

RedHat Cluster Suite 中包括在的套件

该集群项目在许多不同的领域涉及到发展，包括：

*CCS-集群配置系统，以管理c luster.conf档案

*集群套房部署工具-图形化工具，部署集群中多个机器 * CLVM -集群扩展到LVM2可扩展逻辑盘管理工具 * CMAN -集群管理器

* Conga -图形界面-集群管理器 * DLM-分布式锁管理器

*Fence-输入/输出系统围栏系统

*GFS * -共享磁盘簇文件系统（全球文件系统）

* GFS2 * -共享磁盘簇文件系统（全球文件系统2 ） * GNBD -内核模块,多机网络共享块设备管理器

* GULM -基于服务器的集群冗余锁定管理器（替代CMAN和DLM） * Openais -开源集群基础设施

*Magma-集群/锁定库用于g ulm和CMAN/DLM 的迁移

* RGmanager -资源组管理器，用来监督、启动和停止应用、服务和资源 *system-config-cluster-图形化工具来管理集群中的多个机器

三、RHCS安装

1、安装环境

硬件环境 ：VM server 1.0 cpu i386 内存512 操作系统 ：RedHat AS 4 update 4 内核版本 ：2.6.9-42.ELsmp

结构：

主机名 扮演角色

webdb1 Client1 (192.168.5.11) webdb2 client2 (192.168.5.12) shared storge 拓扑图：

client (one) client(tow)

2、所要安装的RPM包

所需安装包如下： ccs-1.0.2-0.i686.rpm ccs-devel-1.0.2-0.i686.rpm cman-1.0.2-0.i686.rpm cman-devel-1.0.2-0.i686.rpm

cman-kernel-smp-2.6.9-39.8.i686.rpm cman-kernheaders-2.6.9-39.8.i686.rpm dlm-1.0.0-5.i686.rpm dlm-devel-1.0.0-5.i686.rpm

dlm-kernel-smp-2.6.9-37.9.i686.rpm dlm-kernheaders-2.6.9-37.9.i686.rpm fence-1.32.6-0.i686.rpm GFS-6.1.2-0.i386.rpm

GFS-kernel-smp-2.6.9-42.2.i686.rpm GFS-kernheaders-2.6.9-42.2.i686.rpm gnbd-1.0.1-1.i686.rpm

gnbd-kernel-smp-2.6.9-9.14.i686.rpm gnbd-kernheaders-2.6.9-9.14.i686.rpm gnome-python2-2.6.0-3.i386.rpm

gnome-python2-bonobo-2.6.0-3.i386.rpm gnome-python2-canvas-2.6.0-3.i386.rpm gulm-1.0.4-0.i686.rpm iddev-2.0.0-3.i686.rpm iddev-devel-2.0.0-3.i686.rpm

kernel-smp-2.6.9-22.0.1.EL.i686.rpm kernel-smp-devel-2.6.9-22.0.1.EL.i686.rpm lvm2-cluster-2.01.14-1.0.RHEL4.i386.rpm

storge(sdb1)

magma-1.0.1-4.i686.rpm magma-devel-1.0.1-4.i686.rpm magma-plugins-1.0.2-0.i386.rpm perl-Net-Telnet-3.03-3.noarch.rpm piranha-0.8.1-1.i386.rpm pyorbit-2.0.1-1.i386.rpm rgmanager-1.9.39-0.i386.rpm

system-config-cluster-1.0.16-1.0.noarch.rpm

3、共享磁盘的配置

用vmware-vdiskmanager 创建 pre-allocated并且是lsi contoller的硬盘 8GB 一个 for oracle。

分别到 webdb1和webdb2的目录，打开webdb1.vmx 和webdb2.vmx, 在最后空白处添加这几段内容

scsi1.present = \scsi1.virtualDev = \scsi1.sharedBus = \

这段是打开 scsi1上的使用，并且设置成virtual, controller设置成lsilogic

然后添加

scsi1:1.present = \

scsi1:1.mode = \scsi1:1.filename = \disk.locking = \

diskLib.dataCacheMaxSize = \

diskLib.dataCacheMaxReadAheadSize = \ diskLib.DataCacheMinReadAheadSize = \diskLib.dataCachePageSize = \ diskLib.maxUnsyncedWrites = \

4、时间同步的设置

5、设置/etc/hosts

分别在webdb1,webdb2上面加入如下：其中10开头的IP地址做心跳。 192.168.5.11 192.168.5.12 192.168.5.13

webdb1 webdb2 webdb3

分别在webdb1,webdb2的/etc/crontab中加入：

0,30 * * * * /usr/sbin/ntpdate –u xxx.xxx.xxx.xxx 其中后面的指时间服务器

10.0.0.1 10.0.0.2

6、配置RHCS

webdb1 webdb2

6.1 仅在webdb1上以root用户运行:system-config-cluster & 出现如下图所示。

第一次运行，由于还没有/etc/cluster/cluster.conf这个文件，所有会出现下面这个

提示框，点击”Create New Configuration”,创建一个新的/etc/cluster/cluster.conf文件。如果删除/etc/cluster/cluster.conf文件，再次运行system-config-cluster还会出现这个提示框。

6.2 做完上面步骤，就会出现下面对话框。这里选择“DLM”。

6.3 点击“Cluster”，在点击“Edit Cluster Properties”

(图一)

6.4 编辑集群的名字

6.5 添加节点

先点左右”cluster nodes” 再点右下角 ”Add a Cluster Node” 给节点命名

6.6 配置fence

在(图一)点击Fence devices出现下图：

在下拉菜单中选择Manual Fencing，再填上name ora_fence.

6.7 为每个节点配置Fence设备

先点击左边 “webdb1” ,再点击右下角 “Manage Fencing For This Node”

6.8 点击”Add a New Fence Level”

6.9 先点击 “Fence-Level-1” 再点击 “Add a New Fence to this Level”

6.10 选择ora_fence，再选择OK

6.11同理配置webdb2在Fence设备 至此我们rhcs就配置完成了。

注：传统的RHCS配置还包括Failover Domains, Resources, Services.因我们用到的是一个叫oracledb.sh的脚本来完成集群配置，所以这里就没有配置它们。

四、oracle安装

1. 安装前的准备，分别在webdb1,webdb2上进行配置。

1.1 stop all cluster daemons and services

1.2 Create an Oracle user. The installation will not work if run as root.

# useradd oracle # passwd oracle

The oracle UIDs and GIDs must be the same on all cluster nodes. Makesure they match by looking at /etc/passwd and /etc/groups. If the UID/GIDs do not match, Oracle will not run. The password should be the same for the Oracle user on all nodes as well.

1.3 Edit /etc/sysctl.conf and add these lines:

kernel.sem = 250 32000 100 128 kernel.shmmax = 536870912 fs.file-max = 65536

net.core.rmem_default = 262144 net.core.rmem_max = 262144 net.core.wmem_default = 262144 net.core.wmem_max = 262144

net.ipv4.ip_local_port_range = 1024 65000 Run:

# sysctl -f /etc/sysctl.conf

Repeat for all nodes in the cluster.

1.4 Install these RPMs if you do not have them already: compat-libstdc33-3.2.3 libaio-0.3.* glibc-devel-2.3.4

There may be more which are not on this list. REFER TO THE ORACLE INSTALLATION DOCUMENTATION FOR MORE INFORMATION Ensure all of the development tools (gcc, binutils, etc.) installed as well.

1.5 Create the Oracle mountpoint(s); e.g.:

# mkdir /mnt/oracle

1.6 Install the oracledb.sh resource agent in to /usr/share/cluster

2. Oracle 安装与测试

The following steps should be performed on one node only.

2.1 Choose the partition(s) you want to use for Oracle and format them using the file system of your choice. In my example, I used /dev/sdb7 (a partition on a shared Winchester OpenRAID array) formatted with the ext3 file system.

# mke2fs -j /dev/sdb1

2.2 Add the virtual IP you intend to use for the system and reconfigure your hostname to the FQDN matching the IP address. As with all virtual IPs, the virtual IP used for Oracle must be in the same subnet as an existing IP address on the system. In my case, I chose 192.168.1.20 (my main eth0 IP is 192.168.1.40).

# ip addr add 192.168.5.13/24 dev eth0 # hostname webdb3

2.3Mount the target Oracle partition(s). I used only /dev/sdb7, and I chose /mnt/oracle as the target for Oracle. Change the ownership of the root mount point(s) or the subdirectory in which you intend to install Oracle and/or its data.

# mount -t ext3 /dev/sdb1 /mnt/oracle # chown oracle.oracle /mnt/oracle

2.4 Run the Oracle installer. I am not a DBA, and I do not know anything about what problems you might hit during install. I just accepted the defaults (except for the Oracle Home and Oracle Inventory directories) and used \database password. If you have any trouble with the installation, refer to the Oracle documentation.

The Oracle installer must be run as the Oracle user. Note: You must use the OUI_HOSTNAME=xxx command line option!

# su - oracle

$ cd /mnt/oracle_install

$ ./runInstaller OUI_HOSTNAME=webdb3

Here are some screenshots to give you an idea of how I installed it.

Install everything on shared storage - even the oraInventory parts

I used /mnt/oracle/home as my Oracle Home Directory

Defaults.

Defaults.

Defaults.

Defaults.

Defaults.

Defaults.

Defaults.

Specify passwords for administration accounts.

Make sure these match your OS group names.

Ready to install.

Installation complete.

2.5 Test the database by logging in to the Database's Enterprise

Management Console to make sure everything is working. You will need to refer to the URL you were given by the Oracle installer. In my case, it was:

http://webdb3:1158/em

Login: SYS

Password: (your password) Connect As: SYSDBA

Create a cluster service the resource agent. It should look something like the following example. You should use a text editor to do this for now, as there is no GUI support:

2.6 Run a test-stop the service using rg_test (RHCS4U2 and later; Never run this command

when the cluster is running!):

type=\

[root@cyan ~]# rg_test test /etc/cluster/cluster.conf stop service oracle10g

Running in test mode. Stopping oracle10g...

Restarting /usr/share/cluster/oracledb.sh as oracle.

Stopping Oracle EM DB Console: [ OK ] Stopping iSQL*Plus: [ OK ] Stopping Oracle Database: [ OK ] Stopping Oracle Listener: [ OK ] Waiting for all Oracle processes to exit: unmounting /mnt/oracle Stop of oracle10g complete

2.7 Set your hostname back to what it should be. If everything is set up correctly, the oracledb.sh script has already generated a hostname.so file which fools Oracle into using the virtual hostname.

# hostname cyan.foo.test.com

[ OK ]

Removing IPv4 address 192.168.1.20/22 from eth0

2.8 Start the service using rg_test (Never run this command when the cluster is running!):

[root@cyan home]# rg_test test /etc/cluster/cluster.conf start service oracle10g

Running in test mode. Starting oracle10g...

mounting /dev/sdb7 on /mnt/oracle mount -t ext3 /dev/sdb7 /mnt/oracle Link for eth0: Detected

Adding IPv4 address 192.168.1.20/22 to eth0

Sending gratuitous ARP: 192.168.1.20 00:02:55:54:28:6c brd ff:ff:ff:ff...

Restarting /usr/share/cluster/oracledb.sh as oracle.

Starting Oracle Database: [ OK ] Starting Oracle Listener: [ OK ] Starting iSQL*Plus: [ OK ] Starting Oracle EM DB Console: [ OK ]

Start of oracle10g complete

2.10 Repeat step 5 for sanity checking / testing. If you can not log in to the EM Database Console, file a Bugzilla.

2.11 repeat step 7 to stop the Oracle database.

3. 各节点的配置

Perform the following steps on all nodes in the cluster:

3.1 Mount the target Oracle partition(s). I used only /dev/sdb7, and I chose /mnt/oracle as the target for Oracle. During installation, Oracle asked you to run a script which set up symbolic links and/or copy files. We must Run this script on each node as the root user:

# mount -t ext3 /dev/sdb1 /mnt/oracle # /mnt/oracle/home/root.sh # umount /mnt/oracle

3.2 Copy /etc/oratab and /etc/oraInst.loc from the install node on which to /etc using scp or another utility .

3.3 Since the cluster daemons are still not running, copy

/etc/cluster/cluster.conf from the install node in to /etc/cluster using scp.

3.4 At your option, you may wish to perform basic sanity checks on the Oracle installation. See Part 2, steps 8-10; the procedure is the same. DO NOT START THE ORACLE SERVICE ON MORE THAN ONE NODE. 4. 完成集群和起动服务

Hacks in oracledb.sh

4.1 Automatically switches to the correct UID at the right time. RGmanager runs as root, while Oracle must run as the oracle user.

4.2 Automatically creates and sets LD_PRELOAD environment variables so that Oracle will always get the right information from the gethostname(2) call. This means that you can not start or stop the clustered Oracle instance outside of the oracledb.sh resource agent/script without also setting the LD_PRELOAD environment variable!, i.e.:

# export LD_PRELOAD=$ORACLE_HOME/hostname.so

This library is not created until the oracle.sh script is run for the first time.

4.3 The oracledb.sh script is designed to be used outside of RHCS4 as a regular init script, too. Change the environment variables, and it should work.

五、需要注意

1、如果使用了防火墙，则： 如下端口需要被打开:

41966 rgmanager/clurgmgrd tcp 41967 rgmanager/clurgmgrd tcp 41968 rgmanager/clurgmgrd tcp 41969 rgmanager/clurgmgrd tcp 50006 ccsd tcp 50007 ccsd udp 50008 ccsd tcp 50009 ccsd tcp 21064 dlm tcp

6809 (RHEL4 and under) cman (RHEL4 and under) udp 5405 (RHEL5 and above) openais (RHEL5 and above) udp 14567 gnbd tcp

六、RHCS及ORACLE的管理

1、RHCS的启动与停止顺序

service rgmanager stop service gfs stop service clvmd stop service fenced stop service cman stop service ccsd stop

service ccsd start service cman start service fenced start service clvmd start service gfs start

service rgmanager start

2、常用命令

#clustat –l 结点与服务状态

#clusvadm –r 各结点之间的手动切换

3、根据使用过程的经验，再加入…….

本文来源：https://www.bwwdw.com/article/g49.html