基于heartbeat v1配置mysql和httpd的高可用双主模型
一、配置高可用集群的前提：（以两节点的heartbeat为例） ⑴时间必须保持同步 ⑵节点之间必须用名称互相通信 建议使用/etc/hosts，而不要用DNS集群中使用的主机名为`uname -n`表示的主机名； ⑶ping node（仅偶数节点才需要） ⑷ssh密钥认证进行无障碍通信；
二、heartbeat v1的配置 程序主配置文件：ha.cf 认证密钥：authkeys, 其权限必须为组和其它无权访问； 资源配置文件：haresources
/usr/share/doc/heartbeat-VERSION 目录中有此三个文件的模板，可将其复制到/etc/ha.d/目录下三、ha.cf文件部分参数详解 logfile /var/log/ha-log#指定heartbaet的日志存放位置 keepalive 2#指定心跳信息间隔时间为2秒 deadtime 30#指定备用节点在30秒内没有收到主节点的心跳信息后，则立即接管主节点的服务资源 warntime 10#指定心跳延迟的时间为十秒。当10秒钟内备份节点不能接收到主节点的心跳信号时，就会往日志中写入一个警告日志，但此时不会切换服务 initdead 120#在某些系统上，系统启动或重启之后需要经过一段时间网络才能正常工作，该选项用于解决这种情况产生的时间间隔。取值至少为deadtime的两倍。 udpport 694 #694为默认使用的端口号。 baud 19200#设置串行通信的波特率 #bcast eth0 # Linux#以广播方式通过eth0传递心跳信息 #mcast eth0 225.0.0.1 694 1 0 #以组播方式通过eth0传递心跳信息，一般在备用节点不止一台时使用。Bcast、ucast和mcast分别代表广播、单播和多播，是组织心跳的三种方式，任选其一即可。 #ucast eth0 192.168.1.2#以单播方式通过eth0传递心跳信息，后面跟的IP地址应为双机对方的IP地址 auto_failback on#用来定义当主节点恢复后，是否将服务自动切回，heartbeat的两台主机分别为主节点和备节点。主节点在正常情况下占用资源并运行所有的服务，遇到故障时把资源交给备节点并由备节点运行服务。在该选项设为on的情况下，一旦主节点恢复运行，则自动获取资源并取代备节点，如果该选项设置为off，那么当主节点恢复后，将变为备节点，而原来的备节点成为主节点 #stonith baytech /etc/ha.d/conf/stonith.baytech #watchdog /dev/watchdog#该选项是可选配置，是通过Heartbeat来监控系统的运行状态。使用该特性，需要在内核中载入"softdog"内核模块，用来生成实际的设备文件，如果系统中没有这个内核模块，就需要指定此模块，重新编译内核。编译完成输入"insmod softdog"加载该模块。然后输入"grep misc /proc/devices"(应为10)，输入"cat /proc/misc |grep watchdog"(应为130)。最后，生成设备文件："mknod /dev/watchdog c 10 130" 。即可使用此功能 node node1.magedu.com node2.magedu.com#要做高可用的节点名称，可以通过命令“uname –n”查看。 ping 192.168.12.237#ping节点地址，ping节点选择的越好，HA集群就越强壮，可以选择固定的路由器作为ping节点，但是最好不要选择集群中的成员作为ping节点，ping节点仅仅用来测试网络连接
ping_group group1 192.168.12.120 192.168.12.237#ping组 apiauth pingdgid=haclient uid=hacluster respawn hacluster /usr/local/ha/lib/heartbeat/pingd -m 100 -d 5s #该选项为可选配置，列出与heartbeat一起启动和关闭的进程，该进程一般是和heartbeat集成的插件，这些进程遇到故障可以自动重新启动。最常用的进程是pingd，此进程用于检测和监控网卡状态，需要配合ping语句指定的ping node来检测网络的连通性。其中hacluster表示启动pingd进程的身份。 #下面的配置是关键，也就是激活crm管理，开始使用v2 style格式 # crm respawn#还可以使用crm on/yes的写法，但这样写的话，如果后面的cib.xml配置有问题会导致heartbeat直接重启该服务器，所以，测试时建议使用respawn的写法 #下面是对传输的数据进行压缩，是可选项 compression bz2 compression_threshold 2
四、案例：基于heartbeat v1配置mysql和httpd的高可用双主模型，二者使用nfs共享数据1、实验环境： node4: 192.168.30.14，mysqld主节点，httpd备节点 node5: 192.168.30.15, httpd主节点，mysql备节点 node3: 192.168.30.20，nfs mysql高可用所需的资源： ip: 192.168.30.100 mysqld nfs:/mydata httpd高可用所需的资源： ip: 192.168.30.101 httpd nfs:/web2、准备工作 ⑴让节点之间的时间同步，并能使用名称进行无障碍通信 ntpdate 202.120.2.101 vim /etc/hosts ssh-keygen -t rsa
ssh-copy-id -i .ssh/id_rsa.pub root@node5
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
# ntpdate 202.120.2.101
13 Apr 23:08:47 ntpdate: the NTP socket is in use, exiting
# date
Wed Apr 13 23:09:25 CST 2016
# crontab -e
*/10 * * * * /usr/sbin/ntpdate 202.120.2.101
# vim /etc/hosts #编辑本地hosts文件
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.30.10 node1
192.168.30.20 node2
192.168.30.13 node3
192.168.30.14 node4
192.168.30.15 node5
# scp /etc/hosts root@node5:/etc/
The authenticity of host 'node5 (192.168.30.15)' can't be established.
RSA key fingerprint is a3:d3:a0:9d:f0:3b:3e:53:4e:ee:61:87:b9:3a:1c:8c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node5,192.168.30.15' (RSA) to the list of known hosts.
root@node5's password:
hosts 100%262 0.3KB/s 00:00
# ping node5
PING node5 (192.168.30.15) 56(84) bytes of data.
64 bytes from node5 (192.168.30.15): icmp_seq=1 ttl=64 time=0.419 ms
64 bytes from node5 (192.168.30.15): icmp_seq=2 ttl=64 time=0.706 ms
^C
--- node5 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1888ms
rtt min/avg/max/mdev = 0.419/0.562/0.706/0.145 ms
# ssh-keygen -t rsa #生成密码对
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
bc:b8:e6:78:6d:51:91:30:4d:d4:dd:50:c0:18:f1:28 root@node4
The key's randomart image is:
+--[ RSA 2048]----+
| o=ooo*o=.|
| .+ ooo .|
| E.. .|
| ... |
| S. |
| ... |
| .... |
| .o.o |
| .+o. |
+-----------------+
# ssh-copy-id -i .ssh/id_rsa.pub root@node5 #将公钥信息导入对方节点的认证文件中
The authenticity of host 'node5 (192.168.30.15)' can't be established.
RSA key fingerprint is a3:d3:a0:9d:f0:3b:3e:53:4e:ee:61:87:b9:3a:1c:8c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node5,192.168.30.15' (RSA) to the list of known hosts.
root@node5's password:
Now try logging into the machine, with "ssh 'root@node5'", and check in:
.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
# ssh root@node5 hostname #连接对方节点不需要输入密码了
node5
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
#在另一个节点上执行同样的步骤
# ntpdate 202.120.2.101
...
# crontab -e
*/10 * * * * /usr/sbin/ntpdate 202.120.2.101
# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.30.10 node1
192.168.30.20 node2
192.168.30.13 node3
192.168.30.14 node4
192.168.30.15 node5
# ssh-keygen -t rsa
...
# ssh-copy-id -i .ssh/id_rsa.pub root@node4
...
# ssh root@node4 hostname
node4
⑵提供一个nfs服务器，共享两个目录，一个给mysql服务，一个给httpd服务 vim /etc/exports
/mydata 192.168.30.0/24(rw,no_root_squash)
/web 192.168.30.0/24(rw)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# mkdir -p /mydata/{data,binlogs} /web
# vim /web/index.html
hello
# ls /mydata
binlogsdata
# useradd -r mysql
# id mysql
uid=27(mysql) gid=27(mysql) groups=27(mysql)
# useradd -r apache
# id apache
uid=48(apache) gid=48(apache) groups=48(apache)
# chown -R mysql.mysql /mydata
# setfacl -R -m u:apache:rwx /web
# vim /etc/exports
/mydata 192.168.30.0/24(rw,no_root_squash)
/web 192.168.30.0/24(rw)
# service rpcbind status
rpcbind (pid1337) is running...
# service nfs start
Starting NFS services:
Starting NFS quotas:
Starting NFS mountd:
Starting NFS daemon:
Starting RPC idmapd:
⑶在两个节点上安装好要做高可用的服务程序
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# useradd -u 27 -r mysql
# useradd -u 48 -r apache
# yum -y install mysql-server httpd
...
# chkconfig mysqld off
# chkconfig httpd off
# vim /etc/my.cnf
datadir=/mydata/data
socket=/var/lib/mysql/mysql.sock
user=mysql
log-bin=/mydata/binlogs/mysql-bin
innodb_file_per_table=ON
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
skip-name-resolve
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
# scp /etc/my.cnf root@node5:/etc/
my.cnf 100%308 0.3KB/s
# mkdir /mydata
# showmount -e 192.168.30.13
Export list for 192.168.30.13:
/web 192.168.30.0/24
/mydata 192.168.30.0/24
# mount -t nfs 192.168.30.13:/mydata /mydata
# service mysqld start
Initializing MySQL database:Installing MySQL system tables...
OK
Filling help tables...
OK
# mysql
...
mysql> grant all on *.* to root@'192.168.30.%' identified by 'magedu';
Query OK, 0 rows affected (0.04 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)
mysql> \q
Bye
# cd /mydata
# ls
binlogsdata
# ls data
ibdata1ib_logfile0ib_logfile1mysqltest
# ls binlogs
mysql-bin.000001mysql-bin.000002mysql-bin.000003mysql-bin.index
# cd
# service mysqld stop
Stopping mysqld:
# umount /mydata
另一节点执行类似步骤，只是不需要再次执行mysql初始化
1
2
3
4
# vim /etc/exports #当mysql初始化完毕后，就可将no_root_squash选项去掉
/mydata 192.168.30.0/24(rw)
/web 192.168.30.0/24(rw)
⑷在每个节点上安装heartbeat并配置好资源 本例中安装heartbeat v2，v2兼容v1，可使用haresources作为配置接口。 说明： ①heartbeat-pils不要使用yum安装，否则会被自动更新成cluter-glue，而cluster-glue跟heartbeat v2不兼容 ②/usr/share/doc/heartbeat-VERSION 目录中有ha.cf、haresources和authkey的模板，可将其复制到/etc/ha.d/目录下： cp /usr/share/doc/heartbeat-2.1.4/{authkeys,ha.cf,haresources} /etc/ha.d/
③/etc/ha.d/resource.d目录中是一些资源代理 IPADDR：使用ifconfig命令配置ip IPADDR2：使用ip addr命令配置ip（需要使用ip addr show命令查看） ④/usr/lib64/heartbeat目录中是一些功能脚本 hb_standby：把当前节点变成备节点 hb_takeover：接管资源 ha_propagate：将ha.cf和authkeys复制到其它节点，会自动保持权限 send_arp：任何时候把地址抢过来就要通知前端路由更新arp缓存 haresources2cib.py：将haresources转换成cib格式,输出至/var/lib/heartbeat/crm/
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
# rpm -ivh heartbeat-pils-2.1.4-12.el6.x86_64.rpm
...
# yum -y install PyXML libnet perl-TimeDate
...
# rpm -ivh heartbeat-stonith-2.1.4-12.el6.x86_64.rpm heartbeat-2.1.4-12.el6.x86_64.rpm
Preparing... ###########################################
1:heartbeat-stonith ########################################### [ 50%]
2:heartbeat ###########################################
# ls /usr/share/doc/heartbeat-2.1.4/
apphbd.cfChangeLog DirectoryMap.txtGettingStarted.htmlHardwareGuide.htmlhb_report.html heartbeat_api.txtRequirements.htmlrsync.txt
authkeys COPYING faqntips.html GettingStarted.txt HardwareGuide.txt hb_report.txt logd.cf Requirements.txt startstop
AUTHORS COPYING.LGPLfaqntips.txt ha.cf haresources heartbeat_api.htmlREADME rsync.html
# cp /usr/share/doc/heartbeat-2.1.4/{authkeys,ha.cf,haresources} /etc/ha.d/
# cd /etc/ha.d
# ls
authkeysha.cfharcharesourcesrc.dREADME.configresource.dshellfuncs
# ls resource.d/
apache db2 Filesystem ICPIPaddr IPsrcaddrLinuxSCSILVSSyncDaemonSwapOCF Raid1 ServeRAIDWinPopup
AudibleAlarmDelayhto-mapfuncsidsIPaddr2IPv6addr LVM MailTo portblockSendArpWAS Xinetd
# vim ha.cf #编辑配置文件，设置如下几项，其它采用默认设置即可
...
logfile /var/log/ha-log #同时关闭logfacility local0
...
auto_failback on
mcast eth0 225.1.1.1 694 1 0
node node4 node5
ping 192.168.30.2
...
# openssl rand -hex 10
392fa6f47a05ed67a0f7
# vim authkeys
...
auth 1
1 sha1 392fa6f47a05ed67a0f7
# chmod 600 authkeys
# vim haresources
...
node4 192.168.30.100/24/eth0/192.168.30.255 Filesystem::192.168.30.13:/mydata::/mydata::nfs mysqld
node5 192.168.30.101/24/eth0/192.168.30.255 Filesystem::192.168.30.13:/web::/var/www/html::nfs httpd
# scp -p authkeys ha.cf haresources root@node5:/etc/ha.d/
authkeys 100%680 0.7KB/s 00:00
ha.cf 100% 10KB10.3KB/s 00:00
haresources 100% 6105 6.0KB/s 00:00
# service heartbeat start;ssh root@node5 'service heartbeat start'
Starting High-Availability services:
2016/04/14_02:38:06 INFO:Resource is stopped
2016/04/14_02:38:06 INFO:Resource is stopped
Done.
Starting High-Availability services:
2016/04/14_02:38:05 INFO:Resource is stopped
2016/04/14_02:38:06 INFO:Resource is stopped
Done.
# ifconfig
...
eth0:0 Link encap:EthernetHWaddr 00:0C:29:32:52:1C
inet addr:192.168.30.100Bcast:192.168.30.255Mask:255.255.255.0
UP BROADCAST RUNNING MULTICASTMTU:1500Metric:1
...
# service mysqld status
mysqld (pid4094) is running...
# service httpd status
httpd is stopped
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# ifconfig
eth0 Link encap:EthernetHWaddr 00:0C:29:96:45:92
inet addr:192.168.30.15Bcast:192.168.30.255Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe96:4592/64 Scope:Link
UP BROADCAST RUNNING MULTICASTMTU:1500Metric:1
RX packets:21716 errors:0 dropped:0 overruns:0 frame:0
TX packets:11535 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:25422433 (24.2 MiB)TX bytes:922617 (900.9 KiB)
eth0:0 Link encap:EthernetHWaddr 00:0C:29:96:45:92
inet addr:192.168.30.101Bcast:192.168.30.255Mask:255.255.255.0
UP BROADCAST RUNNING MULTICASTMTU:1500Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNINGMTU:16436Metric:1
RX packets:48 errors:0 dropped:0 overruns:0 frame:0
TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3120 (3.0 KiB)TX bytes:3120 (3.0 KiB)
# service httpd status
httpd (pid3820) is running...
# service mysqld status
mysqld is stopped
⑸客户端测试：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# curl 192.168.30.101
hello
# mysql -u root -h 192.168.30.100 -p
...
mysql> create database hellodb;
Query OK, 1 row affected (0.03 sec)
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| hellodb |
| mysql |
| test |
+--------------------+
4 rows in set (0.00 sec)
⑹模拟资源资源转移
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# /usr/lib64/heartbeat/hb_standby
2016/04/14_07:07:52 Going standby .
# ifconfig
eth0 Link encap:EthernetHWaddr 00:0C:29:32:52:1C
inet addr:192.168.30.14Bcast:192.168.30.255Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe32:521c/64 Scope:Link
UP BROADCAST RUNNING MULTICASTMTU:1500Metric:1
RX packets:91672 errors:0 dropped:0 overruns:0 frame:0
TX packets:87427 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:56564639 (53.9 MiB)TX bytes:38563809 (36.7 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNINGMTU:16436Metric:1
RX packets:48 errors:0 dropped:0 overruns:0 frame:0
TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3120 (3.0 KiB)TX bytes:3120 (3.0 KiB)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# ifconfig
eth0 Link encap:EthernetHWaddr 00:0C:29:96:45:92
inet addr:192.168.30.15Bcast:192.168.30.255Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe96:4592/64 Scope:Link
UP BROADCAST RUNNING MULTICASTMTU:1500Metric:1
RX packets:61508 errors:0 dropped:0 overruns:0 frame:0
TX packets:50930 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:33075217 (31.5 MiB)TX bytes:8763944 (8.3 MiB)
eth0:0 Link encap:EthernetHWaddr 00:0C:29:96:45:92
inet addr:192.168.30.101Bcast:192.168.30.255Mask:255.255.255.0
UP BROADCAST RUNNING MULTICASTMTU:1500Metric:1
eth0:1 Link encap:EthernetHWaddr 00:0C:29:96:45:92
inet addr:192.168.30.100Bcast:192.168.30.255Mask:255.255.255.0
UP BROADCAST RUNNING MULTICASTMTU:1500Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNINGMTU:16436Metric:1
RX packets:48 errors:0 dropped:0 overruns:0 frame:0
TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3120 (3.0 KiB)TX bytes:3120 (3.0 KiB)
1
2
3
4
5
6
7
8
9
10
11
12
mysql> select version();
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 2
Current database: *** NONE ***
+------------+
| version()|
+------------+
| 5.1.73-log |
+------------+
1 row in set (0.13 sec)
页:
[1]