LVS+Keepalied高可用架构
LVS+Keepalied高可用架构LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。LVS的负载能力强，因为其工作方式逻辑非常简单，仅进行请求分发，而且工作在网络的第4层，没有流量，所以其效率不需要有过多的忧虑。所以在很多企业是应用很广泛的。LVS基本能支持所有应用，因为工作在第4层，所以LVS可以对几乎所有应用进行负载均衡，包括Web、数据库等。LVS可以提供负载均衡，keepalived可以提供健康检查，故障转移，两者结合很好地提高系统的可用性。
LVS并不能完全判别节点故障，比如在WLC规则下，如果集群里有一个节点没有配置VIP，将会导致整个集群不能使用。
接下来我们依据lvs的优点结合keepalived部署高可用架构，实现更高的可用性。
环境主机centos7,环境需求如下：
node2测试主机 ens33：172.25.0.32/24
node3Master ens33： 172.25.0.33/24
node4backup ens33：172.25.0.34/24
vip : 172.25.0.200/32
为了简便我这里直接用172.25.0.33/24 172.25.0.34/24 直接做revel server
搭建拓扑图如下：
http://s1.运维网.com/images/20180106/1515235973810739.jpg
一、环境安装
1、开启路由转发并 、安装lvs。
# echo "1" >/proc/sys/net/ipv4/ip_forward
# echo "1" >/proc/sys/net/ipv4/ip_forward 在node3和node4使用yum进行lvs安装
# yum install -yipvsadm
# yum install -yipvsadm
2、keepalived安装。
我们可以采取源码在node3和node4上安装keepalived就可以了,我这里在node3演示做，另一台也要同样的操作。
先安装keepalived依赖组件
#yum -y install libnl libnl-devel libnfnetlink-devel popt-develgcc make
# cd /usr/local/src/ #下载keepalived压缩包
# wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz
#tar zxvf keepalived-1.2.7.tar.gz-C/usr/local
#cd ../keepalived-1.2.7
#./configure #编译成功的结果如下:
Keepalived configuration
------------------------
Keepalived version : 1.2.7
Compiler : gcc
Compiler flags : -g -O2
Extra Lib : -lpopt -lssl -lcrypto-lnl
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
IPVS use libnl : Yes
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
SNMP support : No
Use Debug flags : No #继续以下步骤：
# make && make install
#cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
#mkdir /etc/keepalived
#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
#cp /usr/local/sbin/keepalived /usr/sbin/ 3、web服务安装与测试。
node3和node4上安装web服务,用作测试。
# yum install -y httpd
# yum install -y httpd
#配置完httpd后我们要可以访问以下本地的web服务.
# curl 172.25.0.33
node3
# curl 172.25.0.34
node4 二、lvs、keepalived配置
1、环境需求。
我们可以先理解好整个架构的配置先
Master: 172.25.0.33
backup: 172.25.0.34
VIP ：172.25.0.200
测试直接用本地web服务测试
2、Real server 的配置
在lvs的DR和TUN模式下，用户的访问请求到达真实服务器后，是直接返回给用户的，而不再经过前端的Director Server，因此，就需要在每个Real server节点上增加虚拟的VIP地址，这样数据才能直接返回给用户，增加VIP地址的操作可以通过下面的命令来实现：
在各自主机绑定vip地址(两台上都执行)。
#ifconfig lo:0 172.25.0.200 broadcast 172.25.0.200 netmask 255.255.255.255 up
#ifconfig lo:0 172.25.0.200 broadcast 172.25.0.200 netmask 255.255.255.255 up #子网掩码255.255.255.255表示这个整个网段只有这一个地址
# route add -host 172.25.0.200 dev lo:0
# route add -host 172.25.0.200 dev lo:0 #上面的命令表示网络请求地址是172.25.0.200的请求，返回的时候的源地址是lo:0网卡上配置的地址，这样出口src地址就是绑定这个假VIP地址，就不会引起丢弃这个包。
3、在Real server上抑制ARP请求
node3上：
# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
# sysctl -p node4上：
# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
# sysctl -p #当然上面的过程可以直接写个脚本执行，就可以省了很多时间
4、查看到vip状态
node3上查看vip状态
# ip addr show lo
1: lo:mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 172.25.0.200/32 brd 172.25.0.200 scope global lo:0
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
node4上查看vip状态
# ip addr show lo
1: lo:mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 172.25.0.200/32 brd 172.25.0.200 scope global lo:0
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
#可以发现两个vip都已经起来了
5、因为keepalived是为了lvs而生的，所以我们可以直接用keepalived直接配置lvs的DR模型。
注意：在企业模式下使用keepalived时，建议是两端都为backup，因为为了防止业务切换繁忙，而使得整个业务系统出问题。
master配置：
# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from root@localhost
smtp_server localhost
smtp_connect_timeout 30
router_idlvs
vrrp_mcast_group4 224.0.100.19
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.0.200
}
}
virtual_server 172.25.0.200 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.25.0.33 80 { ##后台的真实服务器IP
weight 1
HTTP_GET {
url {
path /index.html
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.25.0.34 80 { ##后台的真实服务器IP
weight 1
HTTP_GET {
url {
path /index.html
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
} backup配置：
# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from root@localhost
smtp_server localhost
smtp_connect_timeout 30
router_idlvs
vrrp_mcast_group4 224.0.100.19
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.0.200
}
}
virtual_server 172.25.0.200 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.25.0.33 80 { ##后台的真实服务器IP
weight 1
HTTP_GET {
url {
path /index.html
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.25.0.34 80 { ##后台的真实服务器IP
weight 1
HTTP_GET {
url {
path /index.html
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
} 5、启动keepalived服务
# service keepalived start
Unit keepalived.service could not be found.
Reloading systemd:
Starting keepalived (via systemctl):
# service keepalived start
Unit keepalived.service could not be found.
Reloading systemd:
Starting keepalived (via systemctl): 三、高可用测试
1、我们可以查看一下两端的keepalived状态
master上查看keepalived状态：
# service keepalived status -l
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since Sat 2018-01-06 17:30:53 CST; 3min 25s ago
Docs: man:systemd-sysv-generator(8)
Process: 73275 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 73278 (keepalived)
CGroup: /system.slice/keepalived.service
├─73278 keepalived -D
├─73280 keepalived -D
└─73281 keepalived -D
Jan 06 17:30:53 node3 Keepalived_vrrp: VRRP sockpool:
Jan 06 17:30:53 node3 Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
Jan 06 17:30:53 node3 Keepalived_healthcheckers: Activating healthchecker for service :80
Jan 06 17:30:53 node3 Keepalived_healthcheckers: Activating healthchecker for service :80
Jan 06 17:30:54 node3 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 06 17:30:55 node3 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 06 17:30:55 node3 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 06 17:30:55 node3 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 172.25.0.200
Jan 06 17:30:55 node3 Keepalived_healthcheckers: Netlink reflector reports IP 172.25.0.200 added
Jan 06 17:31:00 node3 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 172.25.0.200
Hint: Some lines were ellipsized, use -l to show in full.
backup上查看keepalived状态：
#service keepalived status -l
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since Sat 2018-01-06 17:31:01 CST; 7min ago
Docs: man:systemd-sysv-generator(8)
Process: 18947 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 18950 (keepalived)
CGroup: /system.slice/keepalived.service
├─18950 keepalived -D
├─18952 keepalived -D
└─18953 keepalived -D
Jan 06 17:31:01 node4 Keepalived_healthcheckers: Configuration is using : 16719 Bytes
Jan 06 17:31:01 node4 Keepalived_vrrp: Configuration is using : 63020 Bytes
Jan 06 17:31:01 node4 Keepalived_vrrp: Using LinkWatch kernel netlink reflector...
Jan 06 17:31:01 node4 Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan 06 17:31:01 node4 Keepalived_vrrp: VRRP sockpool:
Jan 06 17:31:01 node4 Keepalived_healthcheckers: Using LinkWatch kernel netlink reflector...
Jan 06 17:31:01 node4 Keepalived_healthcheckers: Activating healthchecker for service :80
Jan 06 17:31:01 node4 Keepalived_healthcheckers: Activating healthchecker for service :80
Jan 06 17:35:38 node4 Keepalived_vrrp: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:35:38 node4 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.65.135 added
#我们可以发现在我标注的地方可以看到vip已经在master即node3起来了
2、通过vip对web服务测试
# curl 172.25.0.200
node3
# curl 172.25.0.200
node4 发现访问vip可以访问成功
3、把master的keepalived停掉
# service keepalived stop
Stopping keepalived (via systemctl): 查看node4的keepalived状态
#service keepalived status -l
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since Sat 2018-01-06 17:31:01 CST; 25min ago
Docs: man:systemd-sysv-generator(8)
Process: 18947 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 18950 (keepalived)
CGroup: /system.slice/keepalived.service
├─18950 keepalived -D
├─18952 keepalived -D
└─18953 keepalived -D
Jan 06 17:35:38 node4 Keepalived_vrrp: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:35:38 node4 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:47:09 node4 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:47:09 node4 Keepalived_vrrp: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:55:37 node4 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 06 17:55:38 node4 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 06 17:55:38 node4 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 06 17:55:38 node4 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 172.25.0.200
Jan 06 17:55:38 node4 Keepalived_healthcheckers: Netlink reflector reports IP 172.25.0.200 added
Jan 06 17:55:43 node4 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 172.25.0.200
Hint: Some lines were ellipsized, use -l to show in full.
可以发现已经切换过来了，继续访问vip
# curl 172.25.0.200
node4
# curl 172.25.0.200
node3
# curl 172.25.0.200
node4
# curl 172.25.0.200
node3 也是可以访问的
到这里为止，我们就已经完成了整个lvs+keepalived的高可用实现了。
总结：我们可以发现，整个过程主要是让vip跑在lvsDR模式上，然后通过keepalived来调度后台的真实服务器。
！！以上就是我的实现过程，希望能帮到大家！！！
页:
[1]