pacemaker+corosync实现资源调度

345353 · 发表于 2016-7-8 09:41:43

场景一：定义web资源实现web集群的高可用！
本次环境：
Centos6系统 2台，各安装了http服务。
Pacemaker corosync服务已经安装

1、查看当前环境

1
2
3
4
5
6
7
8
9
10
11
12

crm(live)configure# show
node erick.com \
      attributesstandby=off
node erick1.com \
      attributesstandby=off
property cib-bootstrap-options: \
      dc-version=1.1.14-8.el6-70404b0\
      cluster-infrastructure="classicopenais (with plugin)" \
      expected-quorum-votes=2\
      stonith-enabled=false\
      no-quorum-policy=stop\
      last-lrm-refresh=1467844686

1、定义web的vip资源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

crm(live)configure# primitive webipocf:heartbeat:IPaddr params ip=192.168.223.110
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node erick.com \
attributes standby=off
node erick1.com \
      attributesstandby=off
primitive webip IPaddr \
      paramsip=192.168.223.110
property cib-bootstrap-options: \
      dc-version=1.1.14-8.el6-70404b0\
      cluster-infrastructure="classicopenais (with plugin)" \
      expected-quorum-votes=2\
      stonith-enabled=false\
      no-quorum-policy=stop\
      last-lrm-refresh=1467844686

在节点上查看是否vip已经生成

1
2
3
4
5
6
7
8
9
10
11
12
13

[iyunv@erick ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP>mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000
link/ether 00:0c:29:8c:43:f0 brd ff:ff:ff:ff:ff:ff
inet 192.168.223.131/24 brd 192.168.223.255 scope global eth0
inet 192.168.223.110/24brd 192.168.223.255 scope global secondary eth0
inet6 fe80::20c:29ff:fe8c:43f0/64 scope link
valid_lft forever preferred_lft forever
[iyunv@erick~]#

定义web资源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

crm(live)configure# primitivewebserver lsb:httpd
crm(live)configure# verify
crm(live)configure# comm
crm(live)configure# show
node erick.com \
      attributesstandby=off
node erick1.com \
      attributesstandby=off
primitive webip IPaddr \
      paramsip=192.168.223.110
primitive webserver lsb:httpd
property cib-bootstrap-options: \
      dc-version=1.1.14-8.el6-70404b0\
      cluster-infrastructure="classicopenais (with plugin)" \
      expected-quorum-votes=2\
      stonith-enabled=false\
      no-quorum-policy=stop\
      last-lrm-refresh=1467844686
crm(live)configure#

现在来利用pacemaker启动服务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

crm(live)# resource start webip
crm(live)# resource start webserver
crm(live)#
crm(live)#
crm(live)#
crm(live)# status
Last updated: Thu Jul  7 21:16:03 2016                Last change: Thu Jul  7 21:16:00 2016 by root via cibadmin onerick.com
Stack: classic openais (with plugin)
Current DC: erick.com (version1.1.14-8.el6-70404b0) - partition with quorum
2 nodes and 2 resources configured,2 expected votes

Online: [ erick.com erick1.com ]

Full list of resources:

webip    (ocf::heartbeat:IPaddr):  Started erick.com
webserver    (lsb:httpd):    Started erick1.com

crm(live)#

现在可以看到资源非常均衡的分别在两个节点上起动了。这样有个问题，会导致web无法访问。现在我我们要实现把两个资源定义在一起，使其能够在一个节点启动

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

crm(live)configure# colocationwebserver_before_webip inf: webserver webip
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node erick.com \
      attributesstandby=off
node erick1.com \
      attributesstandby=off
primitive webip IPaddr \
      paramsip=192.168.223.110 \
      metatarget-role=Started
primitive webserver lsb:httpd \
      metatarget-role=Started
colocation webserver_before_webipinf: webserver webip
property cib-bootstrap-options: \
      dc-version=1.1.14-8.el6-70404b0\
      cluster-infrastructure="classicopenais (with plugin)" \
      expected-quorum-votes=2\
      stonith-enabled=false\
      no-quorum-policy=stop\
      last-lrm-refresh=1467844686
crm(live)configure# up
crm(live)# status
Last updated: Thu Jul  7 21:20:50 2016                Last change: Thu Jul  7 21:20:37 2016 by root via cibadmin onerick.com
Stack: classic openais (with plugin)
Current DC: erick.com (version1.1.14-8.el6-70404b0) - partition with quorum
2 nodes and 2 resources configured,2 expected votes

Online: [ erick.com erick1.com ]

Full list of resources:

webip    (ocf::heartbeat:IPaddr):  Started erick.com
webserver    (lsb:httpd):    Started erick.com

可以看到两个资源都在erick节点了，那么我门使用浏览器测试下：

Ok，访问成功！

由于我们是2台机器，现在我们若一台服务器宕机后，默认是的仲裁是服务关闭，所以我们要修改下策略，改成ignore

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

crm(live)configure# propertyno-quorum-policy=ignore
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure#
crm(live)configure# show
node erick.com \
      attributesstandby=off
node erick1.com \
      attributesstandby=off
primitive webip IPaddr \
      paramsip=192.168.223.110 \
      metatarget-role=Started
primitive webserver lsb:httpd \
      metatarget-role=Started
colocation webserver_before_webipinf: webserver webip
property cib-bootstrap-options: \
dc-version=1.1.14-8.el6-70404b0 \
      cluster-infrastructure="classicopenais (with plugin)" \
      expected-quorum-votes=2\
      stonith-enabled=false\
      no-quorum-policy=ignore \
      last-lrm-refresh=1467844686

现在，我们关闭一台虚拟机的服务，（模仿其宕机）查看资源转移情况

1
2
3
4
5

[iyunv@erick ~]# vim/etc/httpd/conf/httpd.conf
[iyunv@erick ~]# /etc/init.d/corosyncstop
Signaling Corosync Cluster Engine(corosync) to terminate: [ OK ]
Waiting for corosync services tounload:. [ OK ]
[iyunv@erick~]#

我们到erick1节点查看情况

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

crm(live)# status
Last updated: Thu Jul  7 07:06:59 2016                Last change: Thu Jul  7 21:24:24 2016 by root via cibadmin onerick.com
Stack: classic openais (with plugin)
Current DC: erick1.com (version1.1.14-8.el6-70404b0) - partition WITHOUT quorum
2 nodes and 2 resources configured,2 expected votes

Online: [ erick1.com ]
OFFLINE: [ erick.com ]

Full list of resources:

webip    (ocf::heartbeat:IPaddr):  Started erick1.com
webserver    (lsb:httpd):    Startederick1.com

crm(live)#

通过web访问下

可以看到资源已经转移到erick1上来了。下面我们把erick节点的服务启动起来。查看资源会不会转移

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

[iyunv@erick ~]# /etc/init.d/corosync start
Starting Corosync Cluster Engine (corosync):             [  OK  ]

crm(live)# status
Last updated: Thu Jul  7 07:09:03 2016                Last change: Thu Jul  7 21:24:24 2016 by root via cibadmin onerick.com
Stack: classic openais (with plugin)
Current DC: erick1.com (version1.1.14-8.el6-70404b0) - partition with quorum
2 nodes and 2 resources configured,2 expected votes

Online: [ erick.com erick1.com ]

Full list of resources:

webip    (ocf::heartbeat:IPaddr):  Started erick1.com
webserver    (lsb:httpd):    Started erick1.com

资源并没有转移，说明不陪权重是不会转移的。

情景二：

我们以上只能做到当机器宕机时资源实现转移，当httpd服务挂掉时，并不会实现转移。下面我们来实现监控资源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

crm(live)configure# monitorwebserver 20s:15s
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node erick.com \
      attributesstandby=off
node erick1.com \
      attributesstandby=off
primitive webip IPaddr \
      paramsip=192.168.223.110 \
      metatarget-role=Started
primitive webserver lsb:httpd \
      meta target-role=Started \
      op monitor interval=20stimeout=15s
colocation webserver_before_webipinf: webserver webip
property cib-bootstrap-options: \
      dc-version=1.1.14-8.el6-70404b0\
      cluster-infrastructure="classicopenais (with plugin)" \
      expected-quorum-votes=2\
      stonith-enabled=false\
      no-quorum-policy=ignore\
      last-lrm-refresh=1467844686

若此时我们httpd服务强行杀掉后，还会自动启动，只有当自己无法启动时才会启动。下面我们把erick1节点httpd的端口改为22这样http服务就无法启动了（过程略），手动kill掉httpd进程（过程略），下面查看资源分布情况

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

Full list of resources:

webip    (ocf::heartbeat:IPaddr):  Started erick.com
webserver       (lsb:httpd):    Startederick.com

Failed Actions:
* webserver_start_0 on erick1.com'unknown error' (1): call=90, status=complete, exitreason='none',
last-rc-change='Thu Jul  707:19:27 2016', queued=0ms, exec=87ms

crm(live)# resource cleanupwebserver ##清除失败的资源
Cleaning up webserver on erick.com,removing fail-count-webserver
Cleaning up webserver on erick1.com,removing fail-count-webserver
Waiting for 2 replies from theCRMd.. OK
crm(live)#
crm(live)# status
Last updated: Thu Jul  7 21:40:15 2016                Last change: Thu Jul  7 07:20:45 2016 by hacluster via crmd onerick1.com
Stack: classic openais (with plugin)
Current DC: erick1.com (version1.1.14-8.el6-70404b0) - partition with quorum
2 nodes and 2 resources configured,2 expected votes

Online: [ erick.com erick1.com ]

Full list of resources:

webip    (ocf::heartbeat:IPaddr):  Started erick.com
webserver    (lsb:httpd):    Startederick.com

通过web验证下