部署Keepalived+LVS
部署keepalived高可用
部署环境：
KEEP1 IP :10.0.0.5
KEEP2 IP : 10.0.0.6
两台服务器的环境都是如下
# cat /etc/redhat-release
CentOS release 6.6 (Final)
# uname -v
#1 SMP Wed Jul 15 10:13:09 UTC 2015
# uname -a
Linux KEEP2 2.6.32-504.30.3.el6.x86_64 #1 SMP Wed Jul 15 10:13:09 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
# uname -r
2.6.32-504.30.3.el6.x86_64
下面是正式开始部署Keepalived，在KEEP1和KEEP2做相同的操作，知道第四部，配置配置文件才有所差别
#######################################################################
#######################################################################
#######################################################################
[*] 做相关内核软连接
安装keepalived之前要先做软连接
ls /usr/src/kernels/`uname -r` -ld
ln -s /usr/src/kernels/`uname -r` /usr/src/linux
ls /usr/src/ -l
=========================================================================
# ls /usr/src/kernels/`uname -r` -ld
drwxr-xr-x. 22 root root 4096 Jul 20 10:47 /usr/src/kernels/2.6.32-431.el6.x86_64
# ln -s /usr/src/kernels/`uname -r` /usr/src/linux
# ls /usr/src/ -l
total 8
drwxr-xr-x. 2 root root 4096 Sep 232011 debug
drwxr-xr-x. 4 root root 4096 Jul 21 09:16 kernels
lrwxrwxrwx1 root root 38 Aug 18 08:29 linux -> /usr/src/kernels/2.6.32-431.el6.x86_64
#
#######################################################################
#######################################################################
#######################################################################
操作步骤2之前可以需要安装下面依赖库
yum install openssl* -y
yum install libnl* -y
2.下载并安装keepalived软件
[ ! -d /home/lvnian/tools ] && mkdir /home/lvnian/tools -p
cd /home/lvnian/tools
wget http://www.keepalived.org/software/keepalived-1.2.10.tar.gz
tar xf keepalived-1.2.10.tar.gz
cd keepalived-1.2.10
./configure
make && make install
======================================================
安装简略过程
# ./configure
config.status: creating keepalived/libipvs-2.6/Makefile
Keepalived configuration
------------------------
Keepalived version : 1.2.10
Keepalived configuration
------------------------
Keepalived version : 1.2.10
Compiler : gcc
Compiler flags : -g -O2 -DFALLBACK_LIBNL1
Extra Lib : -lssl -lcrypto -lcrypt-lnl
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
IPVS use libnl : Yes(如果这个为no 可以选择yum install libnl* -y 在编译安装)
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
SNMP support : No
SHA1 support : No
Use Debug flags : No
# make && make install
make: Leaving directory `/home/deng/tools/keepalived-1.2.10/keepalived'
make -C genhash install
make: Entering directory `/home/deng/tools/keepalived-1.2.10/genhash'
install -d /usr/local/bin
install -m 755 ../bin/genhash /usr/local/bin/
install -d /usr/local/share/man/man1
install -m 644 ../doc/man/man1/genhash.1 /usr/local/share/man/man1
make: Leaving directory `/home/deng/tools/keepalived-1.2.10/genhash'
#
编译参数说明：
--with-kernel-dir=DIR 这个参数很重要，但是这个参数并不是要把keepalived编译到内核中，而是指定使用的内核源码里面的头文件，就是include目录，如果使用LVS，才需要这个参数，但是就算使用了LVS也可以不需要这个参数,但需要做软连接：
ln -s /usr/src/kernels/`uname -r` /usr/src/linux
#######################################################################
#######################################################################
#######################################################################
3.配置启动规范
/bin/cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
/bin/cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
/bin/cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
/bin/cp /usr/local/sbin/keepalived /usr/sbin/
/etc/init.d/keepalived start
ps -ef |grep keepalived
=================================
# /bin/cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
# /bin/cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
/etc/init.d/keepalived start
# mkdir /etc/keepalived
ps -ef |grep keepalived
# /bin/cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
# /bin/cp /usr/local/sbin/keepalived /usr/sbin/
# /etc/init.d/keepalived start
Starting keepalived:
#ps -ef |grep keepalived
root 2765 10 06:04 ? 00:00:00 keepalived -D
root 276813480 06:04 pts/0 00:00:00 grep keepalived
root 276927650 06:04 ? 00:00:00 keepalived -D
root 277027650 06:04 ? 00:00:00 keepalived -D
#
安装配置成功
上面的配置需要在KEEP1和KEEP2中都执行
#######################################################################
#######################################################################
#######################################################################
4.配置keepalived单实例
# pwd
/etc/keepalived
主配置文件如下MASTER：
# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
759685538@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_1 ##路由ID，主备必须要不相同
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 55 ##虚拟路由ID，主备必须要相同
priority 150 ##主的优先级必须比备的大
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.100/24
10.0.0.200/25
}
}
#
备机配置文件BACKUP:
# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
759685538@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 55
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.100/24
10.0.0.200/25
}
}
#
查看VIP情况
KEEP1：
http://s3.运维网.com/wyfs02/M02/74/61/wKioL1YbvzmAYeKZAAEI3yukNJA995.jpg
KEEP2
http://s3.运维网.com/wyfs02/M00/74/64/wKiom1YbvxvCkNu1AADeF7_Q4RU479.jpg
VIP会跟着主备服务器的切换而漂移
把KEEP1的keepalived服务停止
# /etc/init.d/keepalived stop
Stopping keepalived:
KEEP1
http://s3.运维网.com/wyfs02/M00/74/61/wKioL1Ybv9bQDPgmAAEEyEi_iLE608.jpg
KEEP2
http://s3.运维网.com/wyfs02/M01/74/64/wKiom1Ybv7mgsXuBAAD3dtJBigY185.jpg
#######################################################################
#######################################################################
#######################################################################
5．keepalived的日志信息
默认情况下keepalived的日志文件在/var/log/message
# tail -f /var/log/messages
Aug 18 09:06:49 LVS1 Keepalived: Starting Keepalived v1.2.10 (08/18,2015)
Aug 18 09:06:49 LVS1 Keepalived: Starting Healthcheck child process, pid=6049
Aug 18 09:06:49 LVS1 Keepalived: Starting VRRP child process, pid=6050
Aug 18 09:06:49 LVS1 Keepalived_vrrp: Netlink reflector reports IP 10.0.0.3 added
Aug 18 09:06:49 LVS1 Keepalived_healthcheckers: Netlink reflector reports IP 10.0.0.3 added
Aug 18 09:06:49 LVS1 Keepalived_vrrp: Netlink reflector reports IP fe80::20c:29ff:fea7:2f91 added
Aug 18 09:06:49 LVS1 Keepalived_vrrp: Registering Kernel netlink reflector
Aug 18 09:06:49 LVS1 Keepalived_vrrp: Registering Kernel netlink command channel
Aug 18 09:06:49 LVS1 Keepalived_vrrp: Registering gratuitous ARP shared channel
Aug 18 09:06:49 LVS1 Keepalived_healthcheckers: Netlink reflector reports IP fe80::20c:29ff:fea7:2f91 added
Aug 18 09:06:49 LVS1 Keepalived_healthcheckers: Registering Kernel netlink reflector
Aug 18 09:06:49 LVS1 Keepalived_healthcheckers: Registering Kernel netlink command channel
#######################################################################
配置keepalived日志信息到单独文件/var/log/keepalived.log
默认情况如下
# cat /etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp -P Only run with VRRP subsystem.
# --check -C Only run with Health-checker subsystem.
# --dont-release-vrrp-V Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs-I Dont remove IPVS topology on daemon stop.
# --dump-conf -d Dump the configuration data.
# --log-detail -D Detailed log messages.
# --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON)
#
KEEPALIVED_OPTIONS="-D"（把这个注销换成下面内容）
#
#######################################################################
修改后的内容
# cat /etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp -P Only run with VRRP subsystem.
# --check -C Only run with Health-checker subsystem.
# --dont-release-vrrp-V Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs-I Dont remove IPVS topology on daemon stop.
# --dump-conf -d Dump the configuration data.
# --log-detail -D Detailed log messages.
# --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON)
#
#KEEPALIVED_OPTIONS="-D"
KEEPALIVED_OPTIONS="-D -d -S 0"
#
#######################################################################
配置/etc/rsyslog.conf
cp /etc/rsyslog.conf /etc/rsyslog.conf.ori
echo "local0.* /var/log/keepalived.log" >>/etc/rsyslog.conf
tail -4 /etc/rsyslog.conf
=================================
# tail -4 /etc/rsyslog.conf
#*.* @@remote-host:514
# ### end of the forwarding rule ###
#keepalived
local0.* /var/log/keepalived.log
#
#######################################################################
#######################################################################
#######################################################################
重启日志服务，检查配置结果
/etc/init.d/rsyslog restart
tail /var/log/keepalived.log-f
==================
# /etc/init.d/rsyslog restart
Shutting down system logger:
Starting system logger:
# tail /var/log/keepalived.log-f
Aug 18 09:36:07 KEEP1 Keepalived_healthcheckers: Netlink reflector reports IP 10.0.0.3 added
Aug 18 09:36:07 KEEP1 Keepalived_healthcheckers: Netlink reflector reports IP fe80::20c:29ff:fea7:2f91 added
Aug 18 09:36:07 KEEP1 Keepalived_healthcheckers: Registering Kernel netlink reflector
Aug 18 09:36:07 KEEP1 Keepalived_healthcheckers: Registering Kernel netlink command channel
Aug 18 09:36:07 KEEP1 Keepalived_vrrp: Netlink reflector reports IP 10.0.0.3 added
Aug 18 09:36:07 KEEP1 Keepalived_vrrp: Netlink reflector reports IP fe80::20c:29ff:fea7:2f91 added
Aug 18 09:36:07 KEEP1 Keepalived_vrrp: Registering Kernel netlink reflector
Aug 18 09:36:07 KEEP1 Keepalived_vrrp: Registering Kernel netlink command channel
Aug 18 09:36:07 KEEP1 Keepalived_vrrp: Registering gratuitous ARP shared channel
Aug 18 09:36:07 KEEP1 Keepalived_vrrp: Initializing ipvs 2.6
如果没看到日志内容，但也不提示错误，那你重启一下你的keepalived服务就能看到日志内容了
###################################################################
###################################################################
如果要keepalived配合LVS使用，使用下面的配置文件即可。master即KEEP1，BACKCP即KEEP2.
MASTER:
! Configuration File for keepalived
global_defs {
notification_email {
759685538@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_1
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 55
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.100/24
# 10.0.0.200/25
}
}
####################################
virtual_server 10.0.0.100 80{
delay_loop 3
lb_algo rr
lb_kind DR
persistence_timeout 1
protocol TCP
real_server 10.0.0.3 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.4 80{
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
###################################################################
BACKUP:
! Configuration File for keepalived
global_defs {
notification_email {
759685538@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 55
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.100/24
# 10.0.0.200/25
}
}
################################################
virtual_server 10.0.0.100 80{
delay_loop 3
lb_algo rr
lb_kind DR
persistence_timeout 1
protocol TCP
real_server 10.0.0.3 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.4 80{
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
###################################################################
后端的RS服务配置好web服务之后再执行下面命令即可：
每台RS配置VIP
ifconfig lo:100 10.0.0.100 netmask 255.255.255.255 up
ifconfig lo:100
ifconfig
route add -host 10.0.0.100 dev lo
抑制ARP响应
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
或者用下面脚本执行：
cat /etc/init.d/realserver.sh
#!/bin/bash
# description: Config realserver lo and apply noarp
VIP=10.0.0.100
/etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
/sbin/route add -host $VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
###################################################################
###################################################################
问题解决
故障1：
# ./configure
checking for sys/time.h... yes
checking openssl/ssl.h usability... no
checking openssl/ssl.h presence... no
checking for openssl/ssl.h... no
configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files.
yum install openssl* -y
故障2
# ./configure
libipvs.c:73: error: ‘NLE_PERM’ undeclared (first use in this function)
libipvs.c:74: error: ‘NLE_BUSY’ undeclared (first use in this function)
libipvs.c:75: error: ‘NLE_RANGE’ undeclared (first use in this function)
libipvs.c:76: error: ‘NLE_NODEV’ undeclared (first use in this function)
make: *** Error 1
make: Leaving directory `/home/lvnian/tools/keepalived-1.2.10/keepalived/libipvs-2.6'
make: *** Error 1
make: Leaving directory `/home/lvnian/tools/keepalived-1.2.10/keepalived'
make: *** Error 2
解决办法
# yum install libnl* -y
页:
[1]