corosync+pacemaker实现web服务高可用

43tfwed

一：实验环境

节点	OS	IP	SAN_IP	VIP
node1	rhel 6.5	192.168.10.11	172.16.1.1	192.168.10.100
node2	rhel 6.5	192.168.10.12	172.16.1.2
san	rhel 6.5		172.16.1.3

注：

1.corosync和pacemaker的概念这里就不说了，网上有很多资料
2.其中两节点IP地址已按上图设置好
3.已连接好san (映射本地盘符为/dev/sdb)
4.两节点已配置相互ssh信任，并已做了时间同步

二：安装相关软件（节点1和2都安装）
1.安装corosync、pacemaker
[iyunv@node1 ~]# for i in 1 2; do ssh node$i yum -y install corosync* pacemaker* ; done
2.安装crmsh
到下面地址下载crmsh、pssh、python-pssh
http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/


本次下载的各版本为：
crmsh-2.1-1.6.x86_64.rpm
pssh-2.3.1-4.1.x86_64.rpm
python-pssh-2.3.1-4.1.x86_64.rpm

安装：
[iyunv@node1 ~]# for i in 1 2; do ssh node$i yum -y --nogpgcheck localinstall /root/*.rpm; done

3.安装apache
[iyunv@node1 ~]# for i in 1 2; do ssh node$i yum -y install httpd; done
[iyunv@node1 ~]# for i in 1 2; do ssh node$i chkconfig httpd off; done

三：配置corosync
1.[iyunv@node1 ~]# cd /etc/corosync/
2.[iyunv@node1 corosync]# cp corosync.conf.example corosync.conf
3. 完成后的配置文件如下所示：
[iyunv@node1 corosync]# cat corosync.conf
# Please read the corosync.conf.5 manualpage
compatibility: whitetank

totem {
       version: 2
       secauth: off
       threads: 0
       interface {
                ringnumber: 0
                bindnetaddr: 192.168.10.0     //在哪个网段上进行多播，根据实际情况修改
                mcastaddr: 226.94.1.1
                mcastport: 5405
                ttl: 1
       }
}

logging {
       fileline: off
       to_stderr: no
       to_logfile: yes
       to_syslog: no
       logfile: /var/log/cluster/corosync.log   //日志所在位置
       debug: off
       timestamp: on
       logger_subsys {
                subsys: AMF
                debug: off
       }
}

amf {
       mode: disabled
}

#
# 以下为添加部分
service {
       ver: 0
       name: pacemaker          //启动corosync时，同时启动pacemaker
}

aisexec {
       user: root
       group: root
}
4.复制配置文件到node2上
[iyunv@node1 corosync]# scp corosync.conf node2:/etc/corosync/
5.启动corosync服务
[iyunv@node1 ~]# /etc/init.d/corosync start
Starting Corosync Cluster Engine(corosync):               [  OK  ]
[iyunv@node1 ~]# ssh node2 "/etc/init.d/corosync start"
Starting Corosync Cluster Engine(corosync): [  OK  ]
6.设置corosync随机启动
[iyunv@node1 ~]# for i in 1 2; do ssh node$i chkconfig corosync on; done

四：集群服务配置
1.查看目前的集群状态
[iyunv@node1 ~]# crm status
Last updated: Tue Jun 23 15:28:58 2015
Last change: Tue Jun 23 15:23:58 2015 via crmd on node1
Stack: classic openais (with plugin)
Current DC: node1 - partition with quorum
Version: 1.1.10-14.el6-368c726
2 Nodes configured, 2 expected votes
0 Resources configured

Online: [ node1 node2 ]

由以上可知，节点1和2都在线，还未配置任何资源

2.设置集群属性
[iyunv@node1 ~]# crm configure
crm(live)configure# property stonith-enabled=false   //禁用stonith设备
crm(live)configure# property no-quorum-policy=ignore //达不到法定票数的策略为忽略
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node node1
node node2
property cib-bootstrap-options: \
       dc-version=1.1.10-14.el6-368c726 \
       cluster-infrastructure="classic openais (with plugin)" \
       expected-quorum-votes=2 \
       stonith-enabled=false \
       no-quorum-policy=ignore

3.添加文件系统（Filesystem）资源
crm(live)configure# primitive webstore ocf:heartbeat:Filesystem params \
  > device=/dev/sdb1 directory=/var/www/html fstype=xfs \
  > op start timeout=60 \
  > op stop timeout=60
crm(live)configure# verify
先不要提交，接着设置资源wetstore最优先运行在node1节点上
crm(live)configure# location webstore_perfer_node1 webstore 50: node1
crm(live)configure# verify
现在提交
crm(live)configure# commit
返回到上一级，查看目前集群状态
crm(live)configure# cd
crm(live)# status
Last updated: Tue Jun 23 15:55:03 2015
Last change: Tue Jun 23 15:54:14 2015 via cibadmin on node1
Stack: classic openais (with plugin)
Current DC: node1 - partition with quorum
Version: 1.1.10-14.el6-368c726
2 Nodes configured, 2 expected votes
1 Resources configured

Online: [ node1 node2 ]

webstore       (ocf::heartbeat:Filesystem):    Started node1

由以上可知，webstore目前运行在node1上

4.添加httpd服务资源并设置httpd服务必须和webstore在一起，webstore必须先启动后，httpd服务才能启动
crm(live)configure# primitive httpd lsb:httpd
crm(live)configure# colocation httpd_with_httpd inf: httpd webstore
crm(live)configure# order webstore_before_httpd Mandatory: webstore:start httpd
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# cd
crm(live)# status
Last updated: Tue Jun 23 15:58:53 2015
Last change: Tue Jun 23 15:58:46 2015 via cibadmin on node1
Stack: classic openais (with plugin)
Current DC: node1 - partition with quorum
Version: 1.1.10-14.el6-368c726
2 Nodes configured, 2 expected votes
2 Resources configured

Online: [ node1 node2 ]

webstore      (ocf::heartbeat:Filesystem):   Started node1
httpd (lsb:httpd):    Started node1

5.添加虚拟IP资源，并设置虚拟IP必须和httpd服务在一起，httpd服务启动后，才能启动虚拟IP
crm(live)configure# primitive webip ocf:heartbeat:IPaddr params \
  > ip=192.168.10.100 nic=eth0
crm(live)configure# colocation webip_with_httpd inf: webip httpd
crm(live)configure# order httpd_before_webip Mandatory: httpd webip
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# cd
crm(live)# status
Last updated: Tue Jun 23 16:02:03 2015
Last change: Tue Jun 23 16:01:54 2015 via cibadmin on node1
Stack: classic openais (with plugin)
Current DC: node1 - partition with quorum
Version: 1.1.10-14.el6-368c726
2 Nodes configured, 2 expected votes
3 Resources configured

Online: [ node1 node2 ]

webstore      (ocf::heartbeat:Filesystem):   Started node1
httpd (lsb:httpd):    Started node1
webip (ocf::heartbeat:IPaddr):       Started node1


五：高可用测试
1.使node1离线后，查看集群状态
[iyunv@node1 ~]# crm node standby
[iyunv@node1 ~]# crm status
Last updated: Tue Jun 23 16:05:40 2015
Last change: Tue Jun 23 16:05:37 2015 viacrm_attribute on node1
Stack: classic openais (with plugin)
Current DC: node1 - partition with quorum
Version: 1.1.10-14.el6-368c726
2 Nodes configured, 2 expected votes
3 Resources configured

Node node1: standby
Online: [ node2 ]

webstore      (ocf::heartbeat:Filesystem):   Started node2
httpd (lsb:httpd):    Started node2
webip (ocf::heartbeat:IPaddr):       Started node2

由以上可知，资源切换到了node2上

2.使node1重新上线
[iyunv@node1 ~]# crm node online
[iyunv@node1 ~]# crm status
Last updated: Tue Jun 23 16:06:43 2015
Last change: Tue Jun 23 16:06:40 2015 viacrm_attribute on node1
Stack: classic openais (with plugin)
Current DC: node1 - partition with quorum
Version: 1.1.10-14.el6-368c726
2 Nodes configured, 2 expected votes
3 Resources configured

Online: [ node1 node2 ]

webstore      (ocf::heartbeat:Filesystem):   Started node1
httpd (lsb:httpd):    Started node1
webip (ocf::heartbeat:IPaddr):       Started node1

由以上可知，资源又回到了node1，这和我们设置的优先运行在node1上相符

至此一个简单的web高可用配置完成

beson2000

谢谢分享