设为首页 收藏本站

运维网

 找回密码
 立即注册
查看: 1748|回复: 0

[经验分享] LVS+keepalived高性能集群应用实战

[复制链接]
432qw
累计签到：1 天
连续签到：1 天
发表于 2016-8-15 10:55:59 | 显示全部楼层 |阅读模式

[iyunv@example ~]# yum install ipvsadm -y  安装ip_vs管理工具

理论部分我们在上篇文章中详细介绍了，这次进入实操阶段
[iyunv@example ~]#  modprobe ip_vs    加载ip_vs模块
[iyunv@example ~]#  modprobe ip_vs_rr
[iyunv@example ~]# lsmod | grep ip_vs
ip_vs_rr                1420  0
ip_vs                 126534  2 ip_vs_rr
libcrc32c               1246  1 ip_vs
ipv6                  335589  271 ip_vs

[iyunv@example ~]# ip addr add 192.168.189.200/24 dev eth0 label eth0:1 手动加vip地址
[iyunv@example ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:5a:e1:a3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.189.130/24 brd 192.168.189.255 scope global eth0
    inet 192.168.189.200/24 scope global secondary eth0:1
    inet6 fe80::20c:29ff:fe5a:e1a3/64 scope link
       valid_lft forever preferred_lft forever

本次环境规划如下 操作系统 centos6.7
wKiom1euuPSS3PJbAAAS_pvg7FQ640.jpg
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
[iyunv@example ~]# curl 192.168.189.132                                   
web01_192.168.189.132
[iyunv@example ~]# curl 192.168.189.133                                   
web02_192.168.189.133

手工执行配置lvs服务器并添加两台rs服务
查看相关的帮助命令
[iyunv@example ~]# ipvsadm --help
ipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1)
Usage:
  ipvsadm -A|E -t|u|f service-address [-s scheduler] [-p [timeout]] [-M netmask] [--pe persistence_engine]
  ipvsadm -D -t|u|f service-address
  ipvsadm -C
  ipvsadm -R
  ipvsadm -S [-n]
  ipvsadm -a|e -t|u|f service-address -r server-address [options]
  ipvsadm -d -t|u|f service-address -r server-address
  ipvsadm -L|l [options]
  ipvsadm -Z [-t|u|f service-address]
  ipvsadm --set tcp tcpfin udp
  ipvsadm --start-daemon state [--mcast-interface interface] [--syncid sid]
  ipvsadm --stop-daemon state
  ipvsadm -h

Commands:
Either long or short options are allowed.
  --add-service     -A        add virtual service with options
  --edit-service    -E        edit virtual service with options
  --delete-service  -D        delete virtual service
  --clear           -C        clear the whole table
  --restore         -R        restore rules from stdin
  --save            -S        save rules to stdout
  --add-server      -a        add real server with options
  --edit-server     -e        edit real server with options
  --delete-server   -d        delete real server
  --list            -L|-l     list the table
  --zero            -Z        zero counters in a service or all services
  --set tcp tcpfin udp        set connection timeout values
  --start-daemon              start connection sync daemon
  --stop-daemon               stop connection sync daemon
  --help            -h        display this help message

[iyunv@example ~]# ipvsadm -C   清空全部配置
[iyunv@example ~]# ipvsadm -A -t 192.168.189.200:80 -s wrr 指定vip并设定调度算法
[iyunv@example ~]# ipvsadm -a -t 192.168.189.200:80 -r  192.168.189.132:80 -g -w 1  添加rs
[iyunv@example ~]# ipvsadm -a -t 192.168.189.200:80 -r  192.168.189.133:80 -g -w 3

[iyunv@example ~]#  ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.189.200:80 wrr
  -> 192.168.189.132:80           Route   1      0          0         
  -> 192.168.189.133:80           Route   3      0          0



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
手工在rs服务器上面绑定操作
第一台
[iyunv@example ~]# ifconfig lo:132 192.168.189.200 netmask 255.255.255.255 up
[iyunv@example ~]# route add -host 192.168.189.200 dev lo
[iyunv@example ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.189.200/32 brd 192.168.189.200 scope global lo:132
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:66:66:33 brd ff:ff:ff:ff:ff:ff
    inet 192.168.189.132/24 brd 192.168.189.255 scope global eth0
    inet6 fe80::20c:29ff:fe66:6633/64 scope link
       valid_lft forever preferred_lft forever

第二台
[iyunv@example ~]# ip addr add 192.168.189.200/32 dev lo label lo:133
[iyunv@example ~]# route add -host 192.168.189.200 dev lo
[iyunv@example ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.189.200/32 scope global lo:133
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:6c:a9:25 brd ff:ff:ff:ff:ff:ff
    inet 192.168.189.133/24 brd 192.168.189.255 scope global eth0
    inet6 fe80::20c:29ff:fe6c:a925/64 scope link
       valid_lft forever preferred_lft forever


手工在rs端抑制arp响应
第一台
[iyunv@example ~]# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
[iyunv@example ~]# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
[iyunv@example ~]# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
[iyunv@example ~]# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

第二台
[iyunv@example ~]# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
[iyunv@example ~]# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
[iyunv@example ~]# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
[iyunv@example ~]# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce





测试访问，不停的刷新，同时打开web服务器的log观察
wKiom1euv6PyHdGKAABS_lYInSI801.jpg
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
web01_132日志
[iyunv@example ~]# tail -f /var/log/httpd/access_log
192.168.189.1 - - [13/Aug/2016:14:34:21 +0800] "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:24 +0800] "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:25 +0800] "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"

web01_133日志
[iyunv@example ~]# tail -f /var/log/httpd/access_log
192.168.189.1 - - [13/Aug/2016:14:33:54 +0800] "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:33:54 +0800] "GET /favicon.ico HTTP/1.1" 404 290 "http://192.168.189.200/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:05 +0800] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:17 +0800] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:20 +0800] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:22 +0800] "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:23 +0800] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:23 +0800] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:24 +0800] "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:25 +0800] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:25 +0800] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:26 +0800] "GET / HTTP/1.1" 200 22 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
192.168.189.1 - - [13/Aug/2016:14:34:26 +0800] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"



1
2
3
4
5
6
7
[iyunv@example ~]#  ipvsadm -Ln --stats  可以看到基本符合我们设定的权重1:3
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  192.168.189.200:80                 66      335        0    44474        0
  -> 192.168.189.132:80                 16       82        0    10978        0
  -> 192.168.189.133:80                 50      253        0    33496        0



上面日志可能不够直观，最好再开一个客户端去执行for循环测试，这里就不在演示
以上手工创建只是为了让我更好的理解lvs的过程，好了下面开始我们的keepalived部分
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
安装keepalived
[iyunv@example ~]#yum install keepalived -y
配置keepalived
主节点上面

[iyunv@example ~]# cat /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
        1335120568@qq.com
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_01
}

vrrp_instance web {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.189.200/24
    }
}

virtual_server 192.168.189.200 80 {
    delay_loop 6         
    lb_algo wrr               
    lb_kind DR               
    nat_mask 255.255.255.0
    persistence_timeout 50     
    protocol TCP               

    real_server 192.168.189.132 80 {
        weight 1              
        TCP_CHECK {
        connect_timeout 8      
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }

    real_server 192.168.189.133 80 {
        weight 1              
        TCP_CHECK {
        connect_timeout 8      
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
}


备用节点上面keepalived配置
[iyunv@example ~]# cat /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
        1335120568@qq.com
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_02
}

vrrp_instance web {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.189.200/24
    }
}


virtual_server 192.168.189.200 80 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    nat_mask 255.255.255.0
    persistence_timeout 50     
    protocol TCP

    real_server 192.168.189.132 80 {
        weight 1
        TCP_CHECK {
        connect_timeout 8
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }

    real_server 192.168.189.133 80 {
        weight 1
        TCP_CHECK {
        connect_timeout 8
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
}



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
启动主备keepalived的观察log  
[iyunv@example ~]# /etc/init.d/keepalived start
Starting keepalived:                                       [  OK  ]
主节点上面
[iyunv@example keepalived]# tail -f /var/log/messages
Aug 13 15:11:10 example Keepalived[45366]: Starting Keepalived v1.2.13 (03/19,2015)
Aug 13 15:11:10 example Keepalived[45367]: Starting Healthcheck child process, pid=45369
Aug 13 15:11:10 example Keepalived[45367]: Starting VRRP child process, pid=45370
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Netlink reflector reports IP 192.168.189.130 added
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Netlink reflector reports IP 192.168.189.130 added
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Netlink reflector reports IP fe80::20c:29ff:fe5a:e1a3 added
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Registering Kernel netlink reflector
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Registering Kernel netlink command channel
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Registering gratuitous ARP shared channel
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Netlink reflector reports IP fe80::20c:29ff:fe5a:e1a3 added
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Registering Kernel netlink reflector
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Registering Kernel netlink command channel
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Configuration is using : 63278 Bytes
Aug 13 15:11:10 example Keepalived_vrrp[45370]: Using LinkWatch kernel netlink reflector...
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Configuration is using : 14742 Bytes
Aug 13 15:11:10 example Keepalived_vrrp[45370]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Using LinkWatch kernel netlink reflector...
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Activating healthchecker for service [192.168.189.132]:80
Aug 13 15:11:10 example Keepalived_healthcheckers[45369]: Activating healthchecker for service [192.168.189.133]:80
Aug 13 15:11:11 example Keepalived_vrrp[45370]: VRRP_Instance(web) Transition to MASTER STATE
Aug 13 15:11:11 example Keepalived_vrrp[45370]: VRRP_Instance(web) Received lower prio advert, forcing new election
Aug 13 15:11:12 example Keepalived_vrrp[45370]: VRRP_Instance(web) Entering MASTER STATE
Aug 13 15:11:12 example Keepalived_vrrp[45370]: VRRP_Instance(web) setting protocol VIPs.
Aug 13 15:11:12 example Keepalived_vrrp[45370]: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:11:17 example Keepalived_vrrp[45370]: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:13:36 example dhclient[1239]: DHCPREQUEST on eth0 to 192.168.189.254 port 67 (xid=0x3b1957c6)
Aug 13 15:13:36 example dhclient[1239]: DHCPACK from 192.168.189.254 (xid=0x3b1957c6)
Aug 13 15:13:38 example dhclient[1239]: bound to 192.168.189.130 -- renewal in 678 seconds.


备用节点上面
[iyunv@example keepalived]# tail -f /var/log/messages
Aug 13 15:11:03 example Keepalived[1937]: Starting Keepalived v1.2.13 (03/19,2015)
Aug 13 15:11:03 example Keepalived[1938]: Starting Healthcheck child process, pid=1940
Aug 13 15:11:03 example Keepalived[1938]: Starting VRRP child process, pid=1941
Aug 13 15:11:03 example Keepalived_vrrp[1941]: Netlink reflector reports IP 192.168.189.131 added
Aug 13 15:11:03 example Keepalived_healthcheckers[1940]: Netlink reflector reports IP 192.168.189.131 added
Aug 13 15:11:03 example Keepalived_healthcheckers[1940]: Netlink reflector reports IP fe80::20c:29ff:feea:4c15 added
Aug 13 15:11:03 example Keepalived_healthcheckers[1940]: Registering Kernel netlink reflector
Aug 13 15:11:03 example Keepalived_healthcheckers[1940]: Registering Kernel netlink command channel
Aug 13 15:11:03 example Keepalived_vrrp[1941]: Netlink reflector reports IP fe80::20c:29ff:feea:4c15 added
Aug 13 15:11:03 example Keepalived_vrrp[1941]: Registering Kernel netlink reflector
Aug 13 15:11:03 example Keepalived_vrrp[1941]: Registering Kernel netlink command channel
Aug 13 15:11:03 example Keepalived_vrrp[1941]: Registering gratuitous ARP shared channel
Aug 13 15:11:03 example Keepalived_vrrp[1941]: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:11:03 example Keepalived_vrrp[1941]: Configuration is using : 63278 Bytes
Aug 13 15:11:03 example Keepalived_vrrp[1941]: Using LinkWatch kernel netlink reflector...
Aug 13 15:11:03 example Keepalived_vrrp[1941]: VRRP_Instance(web) Entering BACKUP STATE
Aug 13 15:11:03 example Keepalived_vrrp[1941]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Aug 13 15:11:03 example Keepalived_healthcheckers[1940]: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:11:03 example Keepalived_healthcheckers[1940]: Configuration is using : 14742 Bytes
Aug 13 15:11:04 example Keepalived_healthcheckers[1940]: Using LinkWatch kernel netlink reflector...
Aug 13 15:11:04 example Keepalived_healthcheckers[1940]: Activating healthchecker for service [192.168.189.132]:80
Aug 13 15:11:04 example Keepalived_healthcheckers[1940]: Activating healthchecker for service [192.168.189.133]:80
Aug 13 15:11:04 example kernel: IPVS: [wrr] scheduler registered.
Aug 13 15:11:07 example Keepalived_vrrp[1941]: VRRP_Instance(web) Transition to MASTER STATE
Aug 13 15:11:08 example Keepalived_vrrp[1941]: VRRP_Instance(web) Entering MASTER STATE
Aug 13 15:11:08 example Keepalived_vrrp[1941]: VRRP_Instance(web) setting protocol VIPs.
Aug 13 15:11:08 example Keepalived_vrrp[1941]: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:11:08 example Keepalived_healthcheckers[1940]: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:11:10 example ntpd[1525]: Listen normally on 6 eth0 192.168.189.200 UDP 123
Aug 13 15:11:11 example Keepalived_vrrp[1941]: VRRP_Instance(web) Received higher prio advert
Aug 13 15:11:11 example Keepalived_vrrp[1941]: VRRP_Instance(web) Entering BACKUP STATE
Aug 13 15:11:11 example Keepalived_vrrp[1941]: VRRP_Instance(web) removing protocol VIPs.
Aug 13 15:11:11 example Keepalived_healthcheckers[1940]: Netlink reflector reports IP 192.168.189.200 removed
Aug 13 15:11:13 example ntpd[1525]: Deleting interface #6 eth0, 192.168.189.200#123, interface stats: received=0, sent=0, dropped=0, active_time=3 secs


主节点vip地址情况
[iyunv@example ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:5a:e1:a3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.189.130/24 brd 192.168.189.255 scope global eth0
    inet 192.168.189.200/24 scope global secondary eth0:1
    inet6 fe80::20c:29ff:fe5a:e1a3/64 scope link
       valid_lft forever preferred_lft forever

备用节点vip地址情况
[iyunv@example ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:ea:4c:15 brd ff:ff:ff:ff:ff:ff
    inet 192.168.189.131/24 brd 192.168.189.255 scope global eth0
    inet6 fe80::20c:29ff:feea:4c15/64 scope link
       valid_lft forever preferred_lft forever





rs服务器配置（两台都需要做）
[iyunv@example ~]# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
[iyunv@example ~]# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
[iyunv@example ~]# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
[iyunv@example ~]# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
[iyunv@example ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.189.200/32 scope global lo:1
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:66:66:33 brd ff:ff:ff:ff:ff:ff
    inet 192.168.189.132/24 brd 192.168.189.255 scope global eth0
    inet6 fe80::20c:29ff:fe66:6633/64 scope link
       valid_lft forever preferred_lft forever



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
高可用性功能测试
高可用性是通过LVS的两个Director Server完成的，为了模拟故障，先将主Director Server
上面的Keepalived服务停止，然后观察备用Director Server上Keepalived的运行日志，信息如下：

[iyunv@example ~]# /etc/init.d/keepalived stop   主节点
Stopping keepalived:                                       [  OK  ]


主节点日志


Aug 13 15:18:53 example Keepalived[45367]: Stopping Keepalived v1.2.13 (03/19,2015)
Aug 13 15:18:53 example Keepalived_vrrp[45370]: VRRP_Instance(web) sending 0 priority
Aug 13 15:18:53 example Keepalived_vrrp[45370]: VRRP_Instance(web) removing protocol VIPs.
Aug 13 15:18:53 example Keepalived_healthcheckers[45369]: Netlink reflector reports IP 192.168.189.200 removed
Aug 13 15:18:53 example Keepalived_healthcheckers[45369]: Removing service [192.168.189.132]:80 from VS [192.168.189.200]:80
Aug 13 15:18:53 example Keepalived_healthcheckers[45369]: Removing service [192.168.189.133]:80 from VS [192.168.189.200]:80
Aug 13 15:18:53 example kernel: IPVS: __ip_vs_del_service: enter
Aug 13 15:18:54 example ntpd[1525]: Deleting interface #6 eth0:1, 192.168.189.200#123, interface stats: received=0, sent=0, dropped=0, active_time=5131 secs


备用节点日志

Aug 13 15:18:53 example Keepalived_vrrp[1941]: VRRP_Instance(web) Transition to MASTER STATE
Aug 13 15:18:54 example Keepalived_vrrp[1941]: VRRP_Instance(web) Entering MASTER STATE
Aug 13 15:18:54 example Keepalived_vrrp[1941]: VRRP_Instance(web) setting protocol VIPs.
Aug 13 15:18:54 example Keepalived_vrrp[1941]: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:18:54 example Keepalived_healthcheckers[1940]: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:18:56 example ntpd[1525]: Listen normally on 7 eth0 192.168.189.200 UDP 123
Aug 13 15:18:59 example Keepalived_vrrp[1941]: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200



wKioL1euykbQRjLkAABYq6eHy_M150.jpg
wKiom1euyoHggofLAABLeM3vbfQ378.jpg
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
主节点keepalived恢复观察日志情况
[iyunv@example ~]# /etc/init.d/keepalived stop
Stopping keepalived:                                       [  OK  ]

主节点日志
Aug 13 15:22:04 example Keepalived[45420]: Starting Keepalived v1.2.13 (03/19,2015)
Aug 13 15:22:04 example Keepalived[45421]: Starting Healthcheck child process, pid=45423
Aug 13 15:22:04 example Keepalived[45421]: Starting VRRP child process, pid=45424
Aug 13 15:22:04 example Keepalived_vrrp[45424]: Netlink reflector reports IP 192.168.189.130 added
Aug 13 15:22:04 example Keepalived_vrrp[45424]: Netlink reflector reports IP fe80::20c:29ff:fe5a:e1a3 added
Aug 13 15:22:04 example Keepalived_vrrp[45424]: Registering Kernel netlink reflector
Aug 13 15:22:04 example Keepalived_vrrp[45424]: Registering Kernel netlink command channel
Aug 13 15:22:04 example Keepalived_vrrp[45424]: Registering gratuitous ARP shared channel
Aug 13 15:22:04 example Keepalived_healthcheckers[45423]: Netlink reflector reports IP 192.168.189.130 added
Aug 13 15:22:04 example Keepalived_healthcheckers[45423]: Netlink reflector reports IP fe80::20c:29ff:fe5a:e1a3 added
Aug 13 15:22:04 example Keepalived_healthcheckers[45423]: Registering Kernel netlink reflector
Aug 13 15:22:04 example Keepalived_healthcheckers[45423]: Registering Kernel netlink command channel
Aug 13 15:22:05 example Keepalived_vrrp[45424]: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:22:05 example Keepalived_vrrp[45424]: Configuration is using : 63278 Bytes
Aug 13 15:22:05 example Keepalived_vrrp[45424]: Using LinkWatch kernel netlink reflector...
Aug 13 15:22:05 example Keepalived_vrrp[45424]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Aug 13 15:22:05 example Keepalived_healthcheckers[45423]: Opening file '/etc/keepalived/keepalived.conf'.
Aug 13 15:22:05 example Keepalived_healthcheckers[45423]: Configuration is using : 14742 Bytes
Aug 13 15:22:05 example Keepalived_healthcheckers[45423]: Using LinkWatch kernel netlink reflector...
Aug 13 15:22:05 example Keepalived_healthcheckers[45423]: Activating healthchecker for service [192.168.189.132]:80
Aug 13 15:22:05 example Keepalived_healthcheckers[45423]: Activating healthchecker for service [192.168.189.133]:80
Aug 13 15:22:05 example Keepalived_vrrp[45424]: VRRP_Instance(web) Transition to MASTER STATE
Aug 13 15:22:05 example Keepalived_vrrp[45424]: VRRP_Instance(web) Received lower prio advert, forcing new election
Aug 13 15:22:06 example Keepalived_vrrp[45424]: VRRP_Instance(web) Entering MASTER STATE
Aug 13 15:22:06 example Keepalived_vrrp[45424]: VRRP_Instance(web) setting protocol VIPs.
Aug 13 15:22:06 example Keepalived_vrrp[45424]: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200
Aug 13 15:22:06 example Keepalived_healthcheckers[45423]: Netlink reflector reports IP 192.168.189.200 added
Aug 13 15:22:07 example ntpd[1525]: Listen normally on 7 eth0 192.168.189.200 UDP 123
Aug 13 15:22:11 example Keepalived_vrrp[45424]: VRRP_Instance(web) Sending gratuitous ARPs on eth0 for 192.168.189.200


备用节点
Aug 13 15:22:05 example Keepalived_vrrp[1941]: VRRP_Instance(web) Received higher prio advert
Aug 13 15:22:05 example Keepalived_vrrp[1941]: VRRP_Instance(web) Entering BACKUP STATE
Aug 13 15:22:05 example Keepalived_vrrp[1941]: VRRP_Instance(web) removing protocol VIPs.
Aug 13 15:22:05 example Keepalived_healthcheckers[1940]: Netlink reflector reports IP 192.168.189.200 removed
Aug 13 15:22:07 example ntpd[1525]: Deleting interface #7 eth0, 192.168.189.200#123, interface stats: received=0, sent=0, dropped=0, active_time=191 secs
Aug 13 15:22:54 example dhclient[1239]: DHCPREQUEST on eth0 to 192.168.189.254 port 67 (xid=0x6d287b9)
Aug 13 15:22:54 example dhclient[1239]: DHCPACK from 192.168.189.254 (xid=0x6d287b9)
Aug 13 15:22:56 example dhclient[1239]: bound to 192.168.189.131 -- renewal in 680 seconds.




wKioL1euyyjjYwFtAABGQK4YJq4369.jpg
可以看到备用节点的vip地址被移除了，一切都符合预期

负载均衡测试，打开浏览器测试访问lvs的vip地址
wKiom1euy8yDQSaYAAA8ByYKa9c791.jpg

wKiom1euy-qhNLPQAABPk1_Q9XA317.jpg

故障切换测试
故障切换是测试在某个节点出现故障后，Keepalived监控模块是否能及时发现，然后屏蔽故障节点，同时将服务转移到正常节点上执行。
这里将real server 1节点服务停掉，假定这个节点出现故障，然后查看主、备机日志信息，相关日志如下：

[iyunv@example ~]# /etc/init.d/httpd stop
Stopping httpd:                                            [  OK  ]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
观察主节点keepalived  日志情况

Aug 13 15:30:05 example Keepalived_healthcheckers[45423]: TCP connection to [192.168.189.132]:80 failed !!!
Aug 13 15:30:05 example Keepalived_healthcheckers[45423]: Removing service [192.168.189.132]:80 from VS [192.168.189.200]:80
Aug 13 15:30:05 example Keepalived_healthcheckers[45423]: Remote SMTP server [192.168.200.1]:25 connected.
Aug 13 15:30:35 example Keepalived_healthcheckers[45423]: Timeout reading data to remote SMTP server [192.168.200.1]:25.

观察备用节点keepalived 日志情况

Aug 13 15:30:04 example Keepalived_healthcheckers[1940]: TCP connection to [192.168.189.132]:80 failed !!!
Aug 13 15:30:04 example Keepalived_healthcheckers[1940]: Removing service [192.168.189.132]:80 from VS [192.168.189.200]:80
Aug 13 15:30:04 example Keepalived_healthcheckers[1940]: Remote SMTP server [192.168.200.1]:25 connected.
Aug 13 15:30:34 example Keepalived_healthcheckers[1940]: Timeout reading data to remote SMTP server [192.168.200.1]:25.

通过日志可以看出，Keepalived监控模块检测到192.168.189.132这台主机出现故障后，将此节点从集群系统中剔除掉了。

此时访问http://192.168.189.200这个地址，应该只能看到“web02_192.168.189.133”了，
这是因为节点1出现故障，而Keepalived监控模块将节点1从集群系统中剔除了

再次重启web01服务观察主备keepalived 日志情况
主节点keepalived 日志情况

Aug 13 15:33:47 example Keepalived_healthcheckers[45423]: TCP connection to [192.168.189.132]:80 success.
Aug 13 15:33:47 example Keepalived_healthcheckers[45423]: Adding service [192.168.189.132]:80 to VS [192.168.189.200]:80
Aug 13 15:33:47 example Keepalived_healthcheckers[45423]: Remote SMTP server [192.168.200.1]:25 connected.
Aug 13 15:34:17 example Keepalived_healthcheckers[45423]: Timeout reading data to remote SMTP server [192.168.200.1]:25.

备用节点keepalived 日志情况
Aug 13 15:33:46 example Keepalived_healthcheckers[1940]: TCP connection to [192.168.189.132]:80 success.
Aug 13 15:33:46 example Keepalived_healthcheckers[1940]: Adding service [192.168.189.132]:80 to VS [192.168.189.200]:80
Aug 13 15:33:46 example Keepalived_healthcheckers[1940]: Remote SMTP server [192.168.200.1]:25 connected.
Aug 13 15:34:16 example dhclient[1239]: DHCPREQUEST on eth0 to 192.168.189.254 port 67 (xid=0x6d287b9)
Aug 13 15:34:16 example dhclient[1239]: DHCPACK from 192.168.189.254 (xid=0x6d287b9)
Aug 13 15:34:16 example Keepalived_healthcheckers[1940]: Timeout reading data to remote SMTP server [192.168.200.1]:25.
Aug 13 15:34:18 example dhclient[1239]: bound to 192.168.189.131 -- renewal in 778 seconds.




wKiom1euzcKiQ05mAABRfl7frQI251.jpg


从日志可知，Keepalived监控模块检测到192.168.189.132这台主机恢复正常后，又将此节点加入到集群系统中。
此时再次访问http://192.168.189.200这个地址，然后不断刷新此页面，应该又能分别看到“web01_192.168.189.132”和
“web01_192.168.189.133”页面了，这说明在web1节点恢复正常后，Keepalived监控模块将此节点加入到集群系统中。


运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表，该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有，请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意，请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息，一经发现立即删除。若您因此触犯法律，一切后果自负，我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集，我们仅提供一个展示、介绍、观摩学习的平台，我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责，亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏，不得用于商业或者其他用途，否则，一切后果均由您自己承担，我们对此不承担任何法律责任
7、如涉及侵犯版权等问题，请您及时通知我们，我们将立即采取措施予以解决
8、联系人Email：admin@iyunv.com 网址：www.yunweiku.com

所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其承担任何法律责任,如涉及侵犯版权等问题,请您及时通知我们,我们将立即处理,联系人Email:kefu@iyunv.com,QQ:1061981298 本贴地址：https://www.yunweiku.com/thread-258085-1-1.html 上篇帖子： CentOS下LVS DR模式详细搭建过程 下篇帖子： LVS原理总结
回复

使用道具 举报

返回列表
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

扫码加入运维网微信交流群X

扫码加入运维网微信交流群

扫描二维码加入运维网微信交流群，最新一手资源尽在官方微信交流群！快快加入我们吧...

扫描微信二维码查看详情

客服 E-mail:iyunvcom@gmail.com

本站由青云提供云计算服务

运维网--中国最专业的运维工程师交流社区

豫ICP备20007574号-1 Copyright © 2012-2025

关于我们

发展历程

联系我们

关注我们

广告服务

广告案例

隐私条款

免责声明

用户登录

用户注册

版主守则

申请版主

手机版

金币获取

帮助中心

每日签到

Good Good

Study

Day Day

UP


客服E-mail:kefu@iyunv.com 客服QQ：1061981298


QQ群⑦：运维网交流群⑦ QQ群⑧：运维网交流群⑧ k8s群：运维网kubernetes交流群


提醒：禁止发布任何违反国家法律、法规的言论与图片等内容；本站内容均来自个人观点与网络等信息，非本站认同之观点.


本站大部分资源是网友从网上搜集分享而来，其版权均归原作者及其网站所有，我们尊重他人的合法权益，如有内容侵犯您的合法权益，请及时与我们联系进行核实删除！



合作伙伴: 青云cloud

快速回复 返回顶部 返回列表