LVS+keepalived高性能集群应用实战
# yum install ipvsadm -y安装ip_vs管理工具
理论部分我们在上篇文章中详细介绍了，这次进入实操阶段
#modprobe ip_vs 加载ip_vs模块
#modprobe ip_vs_rr
# lsmod | grep ip_vs
ip_vs_rr 14200
ip_vs 1265342 ip_vs_rr
libcrc32c 12461 ip_vs
ipv6 335589271 ip_vs
# ip addr add 192.168.189.200/24 dev eth0 label eth0:1 手动加vip地址
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:5a:e1:a3 brd ff:ff:ff:ff:ff:ff
inet 192.168.189.130/24 brd 192.168.189.255 scope global eth0
inet 192.168.189.200/24 scope global secondary eth0:1
inet6 fe80::20c:29ff:fe5a:e1a3/64 scope link
valid_lft forever preferred_lft forever
本次环境规划如下 操作系统 centos6.7
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
# curl 192.168.189.132
web01_192.168.189.132
# curl 192.168.189.133
web02_192.168.189.133
手工执行配置lvs服务器并添加两台rs服务
查看相关的帮助命令
# ipvsadm --help
ipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1)
Usage:
ipvsadm -A|E -t|u|f service-address [-s scheduler] [-p ] [-M netmask] [--pe persistence_engine]
ipvsadm -D -t|u|f service-address
ipvsadm -C
ipvsadm -R
ipvsadm -S [-n]
ipvsadm -a|e -t|u|f service-address -r server-address
ipvsadm -d -t|u|f service-address -r server-address
ipvsadm -L|l
ipvsadm -Z [-t|u|f service-address]
ipvsadm --set tcp tcpfin udp
ipvsadm --start-daemon state [--mcast-interface interface] [--syncid sid]
ipvsadm --stop-daemon state
ipvsadm -h
Commands:
Either long or short options are allowed.
--add-service -A add virtual service with options
--edit-service -E edit virtual service with options
--delete-service-D delete virtual service
--clear -C clear the whole table
--restore -R restore rules from stdin
--save -S save rules to stdout
--add-server -a add real server with options
--edit-server -e edit real server with options
--delete-server -d delete real server
--list -L|-l list the table
--zero -Z zero counters in a service or all services
--set tcp tcpfin udp set connection timeout values
--start-daemon start connection sync daemon
--stop-daemon stop connection sync daemon
--help -h display this help message
# ipvsadm -C 清空全部配置
# ipvsadm -A -t 192.168.189.200:80 -s wrr 指定vip并设定调度算法
# ipvsadm -a -t 192.168.189.200:80 -r192.168.189.132:80 -g -w 1添加rs
# ipvsadm -a -t 192.168.189.200:80 -r192.168.189.133:80 -g -w 3
#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP192.168.189.200:80 wrr
-> 192.168.189.132:80 Route 1 0 0
-> 192.168.189.133:80 Route 3 0 0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
手工在rs服务器上面绑定操作
第一台
# ifconfig lo:132 192.168.189.200 netmask 255.255.255.255 up
# route add -host 192.168.189.200 dev lo
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 192.168.189.200/32 brd 192.168.189.200 scope global lo:132
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:66:66:33 brd ff:ff:ff:ff:ff:ff
inet 192.168.189.132/24 brd 192.168.189.255 scope global eth0
inet6 fe80::20c:29ff:fe66:6633/64 scope link
valid_lft forever preferred_lft forever
第二台
# ip addr add 192.168.189.200/32 dev lo label lo:133
# route add -host 192.168.189.200 dev lo
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 192.168.189.200/32 scope global lo:133
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:6c:a9:25 brd ff:ff:ff:ff:ff:ff
inet 192.168.189.133/24 brd 192.168.189.255 scope global eth0
inet6 fe80::20c:29ff:fe6c:a925/64 scope link
valid_lft forever preferred_lft forever
手工在rs端抑制arp响应
第一台
# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
第二台
# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
测试访问，不停的刷新，同时打开web服务器的log观察
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
web01_132日志
# tail -f /var/log/httpd/access_log
192.168.189.1 - - "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
web01_133日志
# tail -f /var/log/httpd/access_log
192.168.189.1 - - "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET /favicon.ico HTTP/1.1" 404 290 "http://192.168.189.200/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
1
2
3
4
5
6
7
#ipvsadm -Ln --stats可以看到基本符合我们设定的权重1:3
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPktsOutPktsInBytes OutBytes
-> RemoteAddress:Port
TCP192.168.189.200:80 66 335 0 44474 0
-> 192.168.189.132:80 16 82 0 10978 0
-> 192.168.189.133:80 50 253 0 33496 0
上面日志可能不够直观，最好再开一个客户端去执行for循环测试，这里就不在演示
以上手工创建只是为了让我更好的理解lvs的过程，好了下面开始我们的keepalived部分
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
安装keepalived
#yum install keepalived -y
配置keepalived
主节点上面
# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
1335120568@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_01
}
vrrp_instance web {
state MASTER
interface eth0
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.189.200/24
}
}
virtual_server 192.168.189.200 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 192.168.189.132 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.189.133 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
备用节点上面keepalived配置
# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
1335120568@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_02
}
vrrp_instance web {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.189.200/24
}
}
virtual_server 192.168.189.200 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 192.168.189.132 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.189.133 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
启动主备keepalived的观察log
# /etc/init.d/keepalived start
Starting keepalived:
主节点上面
# tail -f /var/log/messages
Aug 13 15:11:10 example Keepalived: Starting Keepalived v1.2.13 (03/19,2015)
Aug 13 15:11:10 example Keepalived: Starting Healthcheck child process, pid=45369
Aug 13 15:11:10 example Keepalived: Starting VRRP child process, pid=45370
Aug 13 15:11:10 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.130 added
Aug 13 15:11:10 example Keepalived_vrrp: Netlink reflector reports IP 192.168.189.130 added
Aug 13 15:11:10 example Keepalived_vrrp: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:11:10 example Keepalived_vrrp: Netlink reflector reports IP fe80::20c:29ff:fe5a:e1a3 added
Aug 13 15:11:10 example Keepalived_vrrp: Registering Kernel netlink reflector
Aug 13 15:11:10 example Keepalived_vrrp: Registering Kernel netlink command channel
Aug 13 15:11:10 example Keepalived_vrrp: Registering gratuitous ARP shared channel
Aug 13 15:11:10 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:11:10 example Keepalived_healthcheckers: Netlink reflector reports IP fe80::20c:29ff:fe5a:e1a3 added
Aug 13 15:11:10 example Keepalived_healthcheckers: Registering Kernel netlink reflector
Aug 13 15:11:10 example Keepalived_healthcheckers: Registering Kernel netlink command channel
Aug 13 15:11:10 example Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:11:10 example Keepalived_vrrp: Configuration is using : 63278 Bytes
Aug 13 15:11:10 example Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
Aug 13 15:11:10 example Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:11:10 example Keepalived_healthcheckers: Configuration is using : 14742 Bytes
Aug 13 15:11:10 example Keepalived_vrrp: VRRP sockpool:
Aug 13 15:11:10 example Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
Aug 13 15:11:10 example Keepalived_healthcheckers: Activating healthchecker for service :80
Aug 13 15:11:10 example Keepalived_healthcheckers: Activating healthchecker for service :80
Aug 13 15:11:11 example Keepalived_vrrp: VRRP_Instance(web) Transition to MASTER STATE
Aug 13 15:11:11 example Keepalived_vrrp: VRRP_Instance(web) Received lower prio advert, forcing new election
Aug 13 15:11:12 example Keepalived_vrrp: VRRP_Instance(web) Entering MASTER STATE
Aug 13 15:11:12 example Keepalived_vrrp: VRRP_Instance(web) setting protocol VIPs.
Aug 13 15:11:12 example Keepalived_vrrp: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:11:17 example Keepalived_vrrp: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:13:36 example dhclient: DHCPREQUEST on eth0 to 192.168.189.254 port 67 (xid=0x3b1957c6)
Aug 13 15:13:36 example dhclient: DHCPACK from 192.168.189.254 (xid=0x3b1957c6)
Aug 13 15:13:38 example dhclient: bound to 192.168.189.130 -- renewal in 678 seconds.
备用节点上面
# tail -f /var/log/messages
Aug 13 15:11:03 example Keepalived: Starting Keepalived v1.2.13 (03/19,2015)
Aug 13 15:11:03 example Keepalived: Starting Healthcheck child process, pid=1940
Aug 13 15:11:03 example Keepalived: Starting VRRP child process, pid=1941
Aug 13 15:11:03 example Keepalived_vrrp: Netlink reflector reports IP 192.168.189.131 added
Aug 13 15:11:03 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.131 added
Aug 13 15:11:03 example Keepalived_healthcheckers: Netlink reflector reports IP fe80::20c:29ff:feea:4c15 added
Aug 13 15:11:03 example Keepalived_healthcheckers: Registering Kernel netlink reflector
Aug 13 15:11:03 example Keepalived_healthcheckers: Registering Kernel netlink command channel
Aug 13 15:11:03 example Keepalived_vrrp: Netlink reflector reports IP fe80::20c:29ff:feea:4c15 added
Aug 13 15:11:03 example Keepalived_vrrp: Registering Kernel netlink reflector
Aug 13 15:11:03 example Keepalived_vrrp: Registering Kernel netlink command channel
Aug 13 15:11:03 example Keepalived_vrrp: Registering gratuitous ARP shared channel
Aug 13 15:11:03 example Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:11:03 example Keepalived_vrrp: Configuration is using : 63278 Bytes
Aug 13 15:11:03 example Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
Aug 13 15:11:03 example Keepalived_vrrp: VRRP_Instance(web) Entering BACKUP STATE
Aug 13 15:11:03 example Keepalived_vrrp: VRRP sockpool:
Aug 13 15:11:03 example Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:11:03 example Keepalived_healthcheckers: Configuration is using : 14742 Bytes
Aug 13 15:11:04 example Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
Aug 13 15:11:04 example Keepalived_healthcheckers: Activating healthchecker for service :80
Aug 13 15:11:04 example Keepalived_healthcheckers: Activating healthchecker for service :80
Aug 13 15:11:04 example kernel: IPVS: scheduler registered.
Aug 13 15:11:07 example Keepalived_vrrp: VRRP_Instance(web) Transition to MASTER STATE
Aug 13 15:11:08 example Keepalived_vrrp: VRRP_Instance(web) Entering MASTER STATE
Aug 13 15:11:08 example Keepalived_vrrp: VRRP_Instance(web) setting protocol VIPs.
Aug 13 15:11:08 example Keepalived_vrrp: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:11:08 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:11:10 example ntpd: Listen normally on 6 eth0 192.168.189.200 UDP 123
Aug 13 15:11:11 example Keepalived_vrrp: VRRP_Instance(web) Received higher prio advert
Aug 13 15:11:11 example Keepalived_vrrp: VRRP_Instance(web) Entering BACKUP STATE
Aug 13 15:11:11 example Keepalived_vrrp: VRRP_Instance(web) removing protocol VIPs.
Aug 13 15:11:11 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.200 removed
Aug 13 15:11:13 example ntpd: Deleting interface #6 eth0, 192.168.189.200#123, interface stats: received=0, sent=0, dropped=0, active_time=3 secs
主节点vip地址情况
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:5a:e1:a3 brd ff:ff:ff:ff:ff:ff
inet 192.168.189.130/24 brd 192.168.189.255 scope global eth0
inet 192.168.189.200/24 scope global secondary eth0:1
inet6 fe80::20c:29ff:fe5a:e1a3/64 scope link
valid_lft forever preferred_lft forever
备用节点vip地址情况
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:ea:4c:15 brd ff:ff:ff:ff:ff:ff
inet 192.168.189.131/24 brd 192.168.189.255 scope global eth0
inet6 fe80::20c:29ff:feea:4c15/64 scope link
valid_lft forever preferred_lft forever
rs服务器配置（两台都需要做）
# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 192.168.189.200/32 scope global lo:1
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:66:66:33 brd ff:ff:ff:ff:ff:ff
inet 192.168.189.132/24 brd 192.168.189.255 scope global eth0
inet6 fe80::20c:29ff:fe66:6633/64 scope link
valid_lft forever preferred_lft forever
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
高可用性功能测试
高可用性是通过LVS的两个Director Server完成的，为了模拟故障，先将主Director Server
上面的Keepalived服务停止，然后观察备用Director Server上Keepalived的运行日志，信息如下：
# /etc/init.d/keepalived stop 主节点
Stopping keepalived:
主节点日志
Aug 13 15:18:53 example Keepalived: Stopping Keepalived v1.2.13 (03/19,2015)
Aug 13 15:18:53 example Keepalived_vrrp: VRRP_Instance(web) sending 0 priority
Aug 13 15:18:53 example Keepalived_vrrp: VRRP_Instance(web) removing protocol VIPs.
Aug 13 15:18:53 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.200 removed
Aug 13 15:18:53 example Keepalived_healthcheckers: Removing service :80 from VS :80
Aug 13 15:18:53 example Keepalived_healthcheckers: Removing service :80 from VS :80
Aug 13 15:18:53 example kernel: IPVS: __ip_vs_del_service: enter
Aug 13 15:18:54 example ntpd: Deleting interface #6 eth0:1, 192.168.189.200#123, interface stats: received=0, sent=0, dropped=0, active_time=5131 secs
备用节点日志
Aug 13 15:18:53 example Keepalived_vrrp: VRRP_Instance(web) Transition to MASTER STATE
Aug 13 15:18:54 example Keepalived_vrrp: VRRP_Instance(web) Entering MASTER STATE
Aug 13 15:18:54 example Keepalived_vrrp: VRRP_Instance(web) setting protocol VIPs.
Aug 13 15:18:54 example Keepalived_vrrp: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:18:54 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:18:56 example ntpd: Listen normally on 7 eth0 192.168.189.200 UDP 123
Aug 13 15:18:59 example Keepalived_vrrp: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
主节点keepalived恢复观察日志情况
# /etc/init.d/keepalived stop
Stopping keepalived:
主节点日志
Aug 13 15:22:04 example Keepalived: Starting Keepalived v1.2.13 (03/19,2015)
Aug 13 15:22:04 example Keepalived: Starting Healthcheck child process, pid=45423
Aug 13 15:22:04 example Keepalived: Starting VRRP child process, pid=45424
Aug 13 15:22:04 example Keepalived_vrrp: Netlink reflector reports IP 192.168.189.130 added
Aug 13 15:22:04 example Keepalived_vrrp: Netlink reflector reports IP fe80::20c:29ff:fe5a:e1a3 added
Aug 13 15:22:04 example Keepalived_vrrp: Registering Kernel netlink reflector
Aug 13 15:22:04 example Keepalived_vrrp: Registering Kernel netlink command channel
Aug 13 15:22:04 example Keepalived_vrrp: Registering gratuitous ARP shared channel
Aug 13 15:22:04 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.130 added
Aug 13 15:22:04 example Keepalived_healthcheckers: Netlink reflector reports IP fe80::20c:29ff:fe5a:e1a3 added
Aug 13 15:22:04 example Keepalived_healthcheckers: Registering Kernel netlink reflector
Aug 13 15:22:04 example Keepalived_healthcheckers: Registering Kernel netlink command channel
Aug 13 15:22:05 example Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:22:05 example Keepalived_vrrp: Configuration is using : 63278 Bytes
Aug 13 15:22:05 example Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
Aug 13 15:22:05 example Keepalived_vrrp: VRRP sockpool:
Aug 13 15:22:05 example Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:22:05 example Keepalived_healthcheckers: Configuration is using : 14742 Bytes
Aug 13 15:22:05 example Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
Aug 13 15:22:05 example Keepalived_healthcheckers: Activating healthchecker for service :80
Aug 13 15:22:05 example Keepalived_healthcheckers: Activating healthchecker for service :80
Aug 13 15:22:05 example Keepalived_vrrp: VRRP_Instance(web) Transition to MASTER STATE
Aug 13 15:22:05 example Keepalived_vrrp: VRRP_Instance(web) Received lower prio advert, forcing new election
Aug 13 15:22:06 example Keepalived_vrrp: VRRP_Instance(web) Entering MASTER STATE
Aug 13 15:22:06 example Keepalived_vrrp: VRRP_Instance(web) setting protocol VIPs.
Aug 13 15:22:06 example Keepalived_vrrp: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:22:06 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:22:07 example ntpd: Listen normally on 7 eth0 192.168.189.200 UDP 123
Aug 13 15:22:11 example Keepalived_vrrp: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
备用节点
Aug 13 15:22:05 example Keepalived_vrrp: VRRP_Instance(web) Received higher prio advert
Aug 13 15:22:05 example Keepalived_vrrp: VRRP_Instance(web) Entering BACKUP STATE
Aug 13 15:22:05 example Keepalived_vrrp: VRRP_Instance(web) removing protocol VIPs.
Aug 13 15:22:05 example Keepalived_healthcheckers: Netlink reflector reports IP 192.168.189.200 removed
Aug 13 15:22:07 example ntpd: Deleting interface #7 eth0, 192.168.189.200#123, interface stats: received=0, sent=0, dropped=0, active_time=191 secs
Aug 13 15:22:54 example dhclient: DHCPREQUEST on eth0 to 192.168.189.254 port 67 (xid=0x6d287b9)
Aug 13 15:22:54 example dhclient: DHCPACK from 192.168.189.254 (xid=0x6d287b9)
Aug 13 15:22:56 example dhclient: bound to 192.168.189.131 -- renewal in 680 seconds.
可以看到备用节点的vip地址被移除了，一切都符合预期
负载均衡测试，打开浏览器测试访问lvs的vip地址
故障切换测试
故障切换是测试在某个节点出现故障后，Keepalived监控模块是否能及时发现，然后屏蔽故障节点，同时将服务转移到正常节点上执行。
这里将real server 1节点服务停掉，假定这个节点出现故障，然后查看主、备机日志信息，相关日志如下：
# /etc/init.d/httpd stop
Stopping httpd:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
观察主节点keepalived日志情况
Aug 13 15:30:05 example Keepalived_healthcheckers: TCP connection to :80 failed !!!
Aug 13 15:30:05 example Keepalived_healthcheckers: Removing service :80 from VS :80
Aug 13 15:30:05 example Keepalived_healthcheckers: Remote SMTP server :25 connected.
Aug 13 15:30:35 example Keepalived_healthcheckers: Timeout reading data to remote SMTP server :25.
观察备用节点keepalived 日志情况
Aug 13 15:30:04 example Keepalived_healthcheckers: TCP connection to :80 failed !!!
Aug 13 15:30:04 example Keepalived_healthcheckers: Removing service :80 from VS :80
Aug 13 15:30:04 example Keepalived_healthcheckers: Remote SMTP server :25 connected.
Aug 13 15:30:34 example Keepalived_healthcheckers: Timeout reading data to remote SMTP server :25.
通过日志可以看出，Keepalived监控模块检测到192.168.189.132这台主机出现故障后，将此节点从集群系统中剔除掉了。
此时访问http://192.168.189.200这个地址，应该只能看到“web02_192.168.189.133”了，
这是因为节点1出现故障，而Keepalived监控模块将节点1从集群系统中剔除了
再次重启web01服务观察主备keepalived 日志情况
主节点keepalived 日志情况
Aug 13 15:33:47 example Keepalived_healthcheckers: TCP connection to :80 success.
Aug 13 15:33:47 example Keepalived_healthcheckers: Adding service :80 to VS :80
Aug 13 15:33:47 example Keepalived_healthcheckers: Remote SMTP server :25 connected.
Aug 13 15:34:17 example Keepalived_healthcheckers: Timeout reading data to remote SMTP server :25.
备用节点keepalived 日志情况
Aug 13 15:33:46 example Keepalived_healthcheckers: TCP connection to :80 success.
Aug 13 15:33:46 example Keepalived_healthcheckers: Adding service :80 to VS :80
Aug 13 15:33:46 example Keepalived_healthcheckers: Remote SMTP server :25 connected.
Aug 13 15:34:16 example dhclient: DHCPREQUEST on eth0 to 192.168.189.254 port 67 (xid=0x6d287b9)
Aug 13 15:34:16 example dhclient: DHCPACK from 192.168.189.254 (xid=0x6d287b9)
Aug 13 15:34:16 example Keepalived_healthcheckers: Timeout reading data to remote SMTP server :25.
Aug 13 15:34:18 example dhclient: bound to 192.168.189.131 -- renewal in 778 seconds.
从日志可知，Keepalived监控模块检测到192.168.189.132这台主机恢复正常后，又将此节点加入到集群系统中。
此时再次访问http://192.168.189.200这个地址，然后不断刷新此页面，应该又能分别看到“web01_192.168.189.132”和
“web01_192.168.189.133”页面了，这说明在web1节点恢复正常后，Keepalived监控模块将此节点加入到集群系统中。
页:
[1]