【HA】Heartbeat高可用集群配置

231weq

    主：hh.huangmingming.cn 192.168.1.31            从：yo.huangmingming.cn 192.168.1.250
        主和从hosts文件配置：

        192.168.1.31 hh hh.huangmingming.cn
        192.168.1.250 yo yo.huangmingming.cn
一、安装epel扩展源

[iyunv@yo ~]# wget http://mirrors.sohu.com/fedora-e ... ease-6-8.noarch.rpm
[iyunv@hh ~]# yum list |grep heartbeat
heartbeat.x86_64                            3.0.4-2.el6                  epel   
heartbeat-devel.i686                        3.0.4-2.el6                  epel   
heartbeat-devel.x86_64                      3.0.4-2.el6                  epel   
heartbeat-libs.i686                         3.0.4-2.el6                  epel   
heartbeat-libs.x86_64                       3.0.4-2.el6                  epel
二、安装heartbeat、libnet（主从都需要安装）
[iyunv@hh ~]# yum -y install heartbeat
[iyunv@hh ~]# yum -y install libnet
[iyunv@hh ~]# yum -y install nginx  （安装nginx做测试）

三、配置heartbeat（在主上配置）
1、拷贝配置文件authkeys、ha.cf、haresources到目录/etc/ha.d/下
[iyunv@hh ~]# cd /usr/share/doc/heartbeat-3.0.4/
[iyunv@hh heartbeat-3.0.4]# ls
apphbd.cf  authkeys  AUTHORS  ChangeLog  COPYING  COPYING.LGPL  ha.cf  haresources  README
[iyunv@hh heartbeat-3.0.4]# cp authkeys ha.cf haresources /etc/ha.d/

2、配置authkeys文件
[iyunv@hh ~]# cd /etc/ha.d/
[iyunv@hh ha.d]# vim authkeys
auth 3
#1 crc
#2 sha1 HI!
3 md5 Hello!
[iyunv@hh ha.d]# chmod 600 authkeys   （authkeys文件的权限要注意改成600）

3、配置haresources文件
[iyunv@hh ha.d]# vim haresources
hh.huangmingming.cn     192.168.1.13/24/eth0:0 nginx  （指定集群IP）

4、配置ha.cf文件
[iyunv@hh ha.d]# vim ha.cf
debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility     local0
keepalive 2
deadtime 30
warntime 10
initdead 60
udpport 694
ucast eth0 192.168.1.250
auto_failback on
node hh.huangmingming.cn
node yo.huangmingming.cn
ping 192.168.1.1
respawn hacluster /usr/lib64/heartbeat/ipfail

5、把这三个文件发送到yo（从机器上）
[iyunv@hh ha.d]# scp authkeys haresources ha.cf yo:/etc/ha.d/

6、启动heartbeat服务，先主后从
[iyunv@hh ha.d]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO:  Resource is stopped
Done.

四、启动产生的错误
1、<--respawn hacluster /usr/lib64/heartbeat/ipfail （注意当前系统是64位还是32位的否则会产生以下错误）--/>
[iyunv@hh ha.d]# /etc/init.d/heartbeat start
heartbeat: udpport setting must precede media statementsheartbeat[4227]: 2016/01/01_05:04:26 ERROR: Client child command [/usr/lib/heartbeat/ipfail] is not executable
heartbeat[4227]: 2016/01/01_05:04:26 ERROR: Heartbeat not started: configuration error.
heartbeat[4227]: 2016/01/01_05:04:26 ERROR: Configuration error, heartbeat not started.

2、<--以下两个错误都是因为主机名配置不对造成的，在heartbeat配置中指定主机名的地方要与当前主机名对应，否则启不来--/>

[iyunv@hh ha.d]# /etc/init.d/heartbeat start
heartbeat: udpport setting must precede media statementsheartbeat: baudrate setting must precede media statementsheartbeat[28195]: 2015/10/29_21:31:14 info: Pacemaker support: false
heartbeat[28195]: 2015/10/29_21:31:14 ERROR: Current node [hh.huangmingming.com] not in configuration!
heartbeat[28195]: 2015/10/29_21:31:14 info: By default, cluster nodes are named by `uname -n` and must be declared with a 'node' directive in the ha.cf file.
heartbeat[28195]: 2015/10/29_21:31:14 info: See also: http://linux-ha.org/wiki/Ha.cf#node_directive
heartbeat[28195]: 2015/10/29_21:31:14 WARN: Logging daemon is disabled --enabling logging daemon is recommended
heartbeat[28195]: 2015/10/29_21:31:14 ERROR: Configuration error, heartbeat not started.

[iyunv@hh ha.d]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO:  Resource is stopped
Heartbeat failure [rc=6]. Failed.
heartbeat: udpport setting must precede media statementsheartbeat: baudrate setting must precede media statementsheartbeat[30724]: 2015/10/29_21:56:29 info: Pacemaker support: false
heartbeat[30724]: 2015/10/29_21:56:29 WARN: Logging daemon is disabled --enabling logging daemon is recommended
heartbeat[30724]: 2015/10/29_21:56:29 info: **************************
heartbeat[30724]: 2015/10/29_21:56:29 info: Configuration validated. Starting heartbeat 3.0.4
heartbeat[30724]: 2015/10/29_21:56:29 ERROR: Bad nodename in /etc/ha.d//haresources [hh]
heartbeat[30724]: 2015/10/29_21:56:29 ERROR: Configuration error, heartbeat not started.

五、故障模拟测试，如主down掉，看会不会切换到从继续提供服务
1、正常情况下
[iyunv@hh ha.d]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO:  Resource is stopped
Done.
[iyunv@hh ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:97:EE:BF  
          inet addr:192.168.1.31  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe97:eebf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:37359 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22139 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:36402808 (34.7 MiB)  TX bytes:2377053 (2.2 MiB)

eth0:0    Link encap:Ethernet  HWaddr 00:0C:29:97:EE:BF  
          inet addr:192.168.1.13  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:1    Link encap:Ethernet  HWaddr 00:0C:29:97:EE:BF  
          inet addr:192.168.1.144  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:28 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2727 (2.6 KiB)  TX bytes:2727 (2.6 KiB)

[iyunv@hh ha.d]# ps aux |grep nginx    （主）
root      31367  0.0  0.1  96488  1728 ?        Ss   21:58   0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     31368  0.0  0.2  96876  2500 ?        S    21:58   0:00 nginx: worker process                  
root      31643  0.0  0.0 103252   824 pts/0    S+   22:16   0:00 grep nginx

[iyunv@yo ha.d]# netstat -tnlp |grep nginx   （从）
[iyunv@yo ha.d]# ps aux |grep nginx
root       5217  0.0  0.0 103256   828 pts/0    S+   06:11   0:00 grep nginx

2、创建测试页面，并在正常情况下测试访问，此时由主提供服务
[iyunv@hh ha.d]# echo "AAAAAAAAAAAAAAAAhh192.168.1.31" >/usr/share/nginx/html/index.html
[iyunv@yo ha.d]# echo "AAAAAAAAAAAAAAAAyo192.168.1.250" >/usr/share/nginx/html/index.html


3、主服务节点down掉之后（测试）
[iyunv@hh ~]# iptables -A INPUT -p icmp -j DROP
[iyunv@hh ~]# tail /var/log/ha-log   （查看日志信息）
ResourceManager(default)[32095]:2015/10/29_22:24:20 info: Running /etc/ha.d/resource.d/IPaddr 192.168.1.13/24/eth0:0 start
IPaddr(IPaddr_192.168.1.13)[32256]:2015/10/29_22:24:20 INFO: Adding inet address 192.168.1.13/24 with broadcast address 192.168.1.255 to device eth0 (with label eth0:0)
IPaddr(IPaddr_192.168.1.13)[32256]:2015/10/29_22:24:20 INFO: Bringing device eth0 up
IPaddr(IPaddr_192.168.1.13)[32256]:2015/10/29_22:24:20 INFO: /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p /var/run/resource-agents/send_arp-192.168.1.13 eth0 192.168.1.13 auto not_used not_used
/usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.1.13)[32230]:2015/10/29_22:24:20 INFO:  Success
ResourceManager(default)[32095]:2015/10/29_22:24:20 info: Running /etc/init.d/nginx  start
Oct 29 22:24:20 hh.huangmingming.cn ipfail: [30881]: info: NS: We are still alive!
Oct 29 22:24:20 hh.huangmingming.cn ipfail: [30881]: info: Link Status update: Link yo.huangmingming.cn/eth0 now has status dead
Oct 29 22:24:22 hh.huangmingming.cn ipfail: [30881]: info: Asking other side for ping node count.
Oct 29 22:24:22 hh.huangmingming.cn ipfail: [30881]: info: Checking remote count of ping nodes.

4、在从（yo）上查看有没有自动启动nginx，并在客户端访问，此时由从提供服务
[iyunv@yo ha.d]# ps aux |grep nginx
root       5534  0.0  0.1  96496  1972 ?        Ss   06:20   0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx      5535  0.0  0.2  96884  2960 ?        S    06:20   0:00 nginx: worker process                  
root       5546  0.0  0.0 103256   828 pts/0    S+   06:24   0:00 grep nginx
[iyunv@yo ha.d]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:8B:40:4A  
          inet addr:192.168.1.250  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe8b:404a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:36106 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21435 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:39751462 (37.9 MiB)  TX bytes:2077943 (1.9 MiB)

eth0:0    Link encap:Ethernet  HWaddr 00:0C:29:8B:40:4A  
          inet addr:192.168.1.13  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:31 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3005 (2.9 KiB)  TX bytes:3005 (2.9 KiB)

删除防火墙再测试
[iyunv@hh ~]# iptables -nvL
[iyunv@hh ~]# iptables -D INPUT -p icmp -j DROP