Heartbeat+Ldirectord+LVS+FreeNas实现HA和LB群集

frtd

heartbeat的四个主要层次：
1、messaging layer 用来实现HA群集中成员存活情况的探测
2、ccm 成员管理，维护HA成员的一致性
3、crm 群集资源管理，用来声明要接管的资源
4、resources agent 资源代理，控制要代理的一些服务

ldirectord 用来提供heartbeat与LVS之间的结合和对后方LB群集中Server的健康探测

LVS Linux虚拟服务，用来管控后方LB群集中的成员，实现客户端服务请求的定向

FreeNas 提供网络附加存储服务，保持LB群集中所提供服务的数据一致性

大致思路：

LVS将其权力移交至ldirectord服务，而ldirectord实际上由heartbeat掌控，后方LB群集成员的健康探测也由ldirectord服务完成。

安装步骤：
LVSà[url=]ldirectord[/url]àheartbeatà安装配置FreeNasà配置LB群集中的Server

拓扑：

步骤：(Director1-->Director2-->FreeNas-->Server1-->Server2-->测试)
Director1：
[iyunv@Director1 ~]# vim /etc/hosts
添加两行解析记录：
192.168.2.1 Director1.a.com
192.168.2.2 Director2.a.com

为了便于安装软件包，使用本地yum
[iyunv@Director1 ~]# vim/etc/yum.repos.d/rhel-debuginfo.repo
1 [rhel-Server]
2 name=Red Hat Enterprise Linux Server
3 baseurl=file:///mnt/cdrom/Server
4 enabled=1
5 gpgcheck=1
6 gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
7 [rhel-Cluster]
8 name=Red Hat Enterprise Linux Cluster
9 baseurl=file:///mnt/cdrom/Cluster
10 enabled=1
11 gpgcheck=1
12 gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release

挂载光盘
[iyunv@Director1 ~]# mkdir /mnt/cdrom
[iyunv@Director1 ~]# mount /dev/cdrom/mnt/cdrom/
mount: block device /dev/cdrom iswrite-protected, mounting read-only

安装ipvsadm
[iyunv@Director1 ~]# yum install ipvsadm –y

确保ipvsadm服务关闭并且不随系统启动自启动
[iyunv@Director1 ~]# service ipvsadm stop
Clearing the current IPVS table: [ OK ]
[iyunv@Director1 ~]# chkconfig ipvsadm off

上传安装ldirectord和heartbeat所需的软件包


安装所有软件包
[iyunv@Director1 heartbeat]# yumlocalinstall *.rpm --nogpgcheck –y

设置heartbeat接管ldirectord服务
拷贝几个需要的配置文件
[iyunv@Director1 heartbeat]# cd/usr/share/doc/heartbeat-2.1.4/
[iyunv@Director1 heartbeat-2.1.4]#cp ha.cf haresources authkeys /etc/ha.d/
[iyunv@Director1 heartbeat-2.1.4]#cd /etc/ha.d/

编辑ha.cf文件来声明HA群集的成员以及心跳探测的方法
[iyunv@Director1 ha.d]# vim ha.cf
91 bcast eth1 # Linux /////在eth1口发送广播来进行Director之间的心跳探测

声明HA群集内的成员
211 node Director1.a.com
212 node Director2.a.com

设置Director加入HA群集的验证方式
[iyunv@Director1 ha.d]# chmod 600 authkeys
[iyunv@Director1 ha.d]# vim authkeys
23 auth 3 /////使用第三种验证方式
24#1 crc
25#2 sha1 HI!
26 3 md5 Director /////启用这种验证方式

[iyunv@Director1 ha.d]# vim haresources /////设置heartbeat的资源清单
45 Director1.a.com 192.168.2.100/24/eth0 ldirectord::ldirectord.cf /////使Director1成为主Director

[iyunv@Director1 ha.d]# cp /usr/share/doc/heartbeat-ldirectord-2.1.4/ldirectord.cf ./
设置LVS移交权力至ldirectord服务

[iyunv@Director1 ha.d]# vim ldirectord.cf
21 quiescent=no /////不使用静默，探测不到后方Server就把Server踢出ipvsadm的定向规则
22
23 #Sample for an http virtual service
24 virtual=192.168.2.100:80
25 real=192.168.2.3:80 gate
26 real=192.168.2.4:80 gate
27 service=http
28 request=".test.html" /////后方Server上的探测页面，ldirectord探测不到该页面即视为Server失效
29 receive="ok" /////探测页面的内容
30 #virtualhost=some.domain.com.au
31 scheduler=rr /////LVS选择的算法
32 #persistent=600
33 #netmask=255.255.255.255
34 #protocol=tcp
35 #checktype=negotiate
36 #checkport=80
37 #request="index.html"
38 #receive="Test Page"


确保ldirectord服务关闭并且不随系统启动自启动
[iyunv@Director1 ha.d]# service ldirectord stop
Stopping ldirectord... success
[iyunv@Director1 ha.d]# chkconfig ldirectord off

启动heartbeat服务并设置其随系统启动而自启动
[iyunv@Director1 ha.d]# service heartbeat start
Starting High-Availability services:
2013/10/08_20:40:27 INFO: Resource isstopped
[ OK ]
[iyunv@Director1 ha.d]# chkconfig heartbeat on


Director2：
[iyunv@Director2 ~]# vim /etc/hosts
添加两行解析记录：
192.168.2.1 Director1.a.com
192.168.2.2 Director2.a.com

为了便于安装软件包，使用本地yum
[iyunv@Director2 ~]# vim/etc/yum.repos.d/rhel-debuginfo.repo
1 [rhel-Server]
2 name=Red Hat Enterprise Linux Server
3 baseurl=file:///mnt/cdrom/Server
4 enabled=1
5 gpgcheck=1
6 gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
7 [rhel-Cluster]
8 name=Red Hat Enterprise Linux Cluster
9 baseurl=file:///mnt/cdrom/Cluster
10 enabled=1
11 gpgcheck=1
12 gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release

挂载光盘
[iyunv@Director2 ~]# mkdir /mnt/cdrom
[iyunv@Director2 ~]# mount /dev/cdrom/mnt/cdrom/
mount: block device /dev/cdrom iswrite-protected, mounting read-only

安装ipvsadm
[iyunv@Director2 ~]# yum install ipvsadm –y

确保ipvsadm服务关闭并且不随系统启动自启动
[iyunv@Director2 ~]# service ipvsadm stop
Clearing the current IPVS table: [ OK ]
[iyunv@Director2 ~]# chkconfig ipvsadm off

上传安装ldirectord和heartbeat所需的软件包


安装所有软件包
[iyunv@Director2 heartbeat]# yumlocalinstall *.rpm --nogpgcheck –y

设置heartbeat接管ldirectord服务并且将LVS权力移交至ldirectord服务
拷贝几个需要的配置文件
在Director1上
[iyunv@Director1 ha.d]# scp ha.cf haresources authkeys ldirectord.cfDirector2.a.com:/etc/ha.d

确保ldirectord服务关闭并且不随系统启动自启动
[iyunv@Director2 ~]# service ldirectord stop
Stopping ldirectord... success
[iyunv@Director2 ~]# chkconfig ldirectordoff

启动heartbeat服务并设置其随系统启动而自启动
[iyunv@Director2 ~]# service heartbeat start
Starting High-Availability services:
2013/10/08_20:41:17 INFO: Resource isstopped
[ OK ]
[iyunv@Director2 ~]# chkconfig heartbeat on


FreeNas：
安装FreeNas并设置共享目录


启动虚拟机


在这里要注意，安装FreeNas系统的硬盘不能作为数据存储盘




此时取出系统镜像
下面为系统成功启动后的界面


可以登录web界面进行管理了


将第二块，大小为10G的盘作为数据存储盘


开启NFS服务，并将刚才设置好的盘共享出来


最后在共享出来的目录中创建两个文件，一个站点的默认页面，一个ldirectord的探测页面，这个目录在后面的服务器上要挂载到web站点的主目录



Server1：
设置参数使Server1不宣告自己的IP
[iyunv@Server1 ~]# echo"net.ipv4.conf.all.arp_announce = 2" >>/etc/sysctl.conf
[iyunv@Server1 ~]# [url=]echo"net.ipv4.conf.all.arp_ignore = 1" >>/etc/sysctl.conf[/url]
[iyunv@Server1 ~]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route =0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
net.ipv4.conf.all.arp_announce= 2
net.ipv4.conf.all.arp_ignore= 1

要在lo:0接口设置VIP，保证Server能与Director通信即可，不需要将网关指向Director。
[iyunv@Server1 ~]# setup





要注意子网掩码为“255.255.255.255”
[iyunv@Server1 ~]# service network restart

安装Apache
[iyunv@Server1 ~]# mkdir /mnt/cdrom
[iyunv@Server1 ~]# mount /dev/cdrom/mnt/cdrom/
mount: block device /dev/cdrom iswrite-protected, mounting read-only
[iyunv@Server1 ~]# rpm -ivh/mnt/cdrom/Server/httpd-2.2.3-31.el5.i386.rpm
warning: /mnt/cdrom/Server/httpd-2.2.3-31.el5.i386.rpm:Header V3 DSA signature: NOKEY, key ID 37017186
Preparing... ########################################### [100%]
1:httpd ########################################### [100%]
[iyunv@Server1 ~]# service httpd start
Starting httpd: [ OK ]
[iyunv@Server1 ~]# chkconfig httpd on

挂载后方FreeNas共享的目录
[iyunv@Server1 ~]# showmount -e 192.168.3.3
Export list for 192.168.3.3:
/mnt/web (everyone)
[iyunv@Server1 ~]# mount192.168.3.3:/mnt/web /var/www/html/ /////临时挂载

实现永久挂载
[iyunv@Server1 ~]# vim /etc/fstab
192.168.3.3:/mnt/web /var/www/html nfs defaults,soft,intr 0 0 /////重启系统后生效

Server2上的操作与Server1相同，不再列出


测试：
正常状态下：




模拟Director1成为备份设备：
[iyunv@Director1 ~]# cd /usr/lib/heartbeat
[iyunv@Director1 heartbeat]# ./hb_standby
2013/10/09_11:03:42 Going standby [all].



模拟Dirctor1恢复正常：
[iyunv@Director1 heartbeat]# ./hb_takeover




模拟Server1失效：




模拟Server1恢复正常：



这样就实现了高可用性和负载均衡群集。

Mei笑D小妞

关羽五绺长髯，风度翩翩，手提青龙偃月刀，江湖人送绰号——刀郎。

0755mx

真是 收益 匪浅

搜ijsio

谢谢楼主，共同发展

fateame

不在课堂上沉睡，就在酒桌上埋醉。

lb5645284

即使哭了我也要笑、即使笑了我也要继续。

狼狼

男人永远是把牵挂当作麻烦；女人永远是把关心当作猜忌