keepalived服务配置

86754tr

keepalived
keepalived.service - LVS and VRRP High Availability Monitor

server1(10.71)--server2(10.72)
注意：建议实验环境使用NAT模式来连接，否则客户端可能无法正常访问测试。需要在两台keepalived上配置默认路由，如ip route add default dev eth0.
需要先同步服务器的时间，centos7.3建议使用chrony来同步时间
systemctl restart chronyd
建议指定/etc/hosts文件，对应两台主机的hostname，可以通过hostname通信

安装服务
yum install keepalived
查看配置文件帮助
man keepalived.conf

vrrp单主配置(即一主一备)--server1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

cp keepalived.conf{,.bak} vi /etc/keepalived.conf global_defs {     notification_email {         root@localhost  #接收邮件地址     }     notification_email_from keepalived@localhost  #发送邮件地址     smtp_server 127.0.0.1  #本地邮件服务     smtp_connect_timeout 30     router_id node1  #主机名     vrrp_mcast_group4 224.10.10.18  #IPv4的多播地址，默认为224.0.0.18 } vrrp_instance VI_1 {     state MASTER  #定义该主机的状态，大写MASTER或BACKUP     interface eth0  #定义发送vrrp的接口     virtual_router_id 14  #唯一性，BACKUP需要配置一致     priority 100  #指定优先级     advert_int 1     authentication {         auth_type PASS  #简单密码认证         auth_pass 571f97b2  #只支持8位字符长度，可以通过命令获取openssl rand -base64 6     }     virtual_ipaddress {         192.168.10.100/24 dev eth0  #配置虚拟IP地址及对应的接口     } }

vrrp单主配置--server2
配置同上，需要更改项如下：
router_id node2  #主机名
state BACKUP  #定义该主机的状态
priority 95  #优先级要低于master
注意：密码认证和虚拟IP地址配置需要和master一致

启动服务
systemctl start keepalived
systemctl status keepalived  //会显示状态的信息，master主机会显示添加的虚拟IP地址

ip a  //可以看到新添加的接口IP

查看发送的组播信息--只有master才会发送组播：
yum install tcpdump
tcpdump -nn -i eth0 host 224.10.10.18

测试停止server1的keepalived服务，虚拟IP地址会自动转移至server2上，如果重新开启server1的服务，则其会重新获取虚拟IP地址，即keepalived默认为抢占模式。
server1: systemctl stop keepalived
server2: ip a

nopreempt：定义工作模式为非抢占模式；
preempt_delay 300：抢占式模式下，节点上线后触发新选举操作的延迟时长；

######################################
vrrp双主配置(即一台服务器同时配置两台虚拟路由)--server1
vi keepalived.conf
在单主配置的基础上，复制上述vrrp_instance VI_1配置段内容，更改如下，（即增加一个虚拟路由器）：

1 2 3 4 5 6 7 8 9 10 11 12 13 14

vrrp_instance VI_2 {  #更改名称为VI_2     state BACKUP  #更改为BACKUP     interface eth0  #定义发送vrrp的接口     virtual_router_id 24  #唯一性，需要区别于vrrp1，这里更改为24     priority 95  #指定优先级，需低于master     advert_int 1     authentication {         auth_type PASS  #简单密码认证         auth_pass 5a1fe7b2  #更改为新密码     }     virtual_ipaddress {         192.168.10.200/24 dev eth0  #更改虚拟IP地址及对应的接口，这里更改为.200     } }

vrrp双主配置--server2
配置同上，需要更改项如下：
state MASTER  #更改为MASTER
priority 100  #优先级
注意：密码认证和虚拟IP地址配置需要和master一致
配置如下：

1 2 3 4 5 6 7 8 9 10 11 12 13 14

vrrp_instance VI_2 {     state MASTER     interface eth0     virtual_router_id 24     priority 100     advert_int 1     authentication {         auth_type PASS         auth_pass 5a1fe7b2     }     virtual_ipaddress {         192.168.10.200/24 dev eth0     } }

测试，停止server1和server2的服务，先开启server1的服务，可以看到两个虚拟IP都在server1上：
server1: systemctl stop keepalived
server1：systemctl start keepalived
server1: ip a
然后开启server2的keepalived服务，可以看到第二个虚拟IP转移到了server2上
server2: systemctl start keepalived
server2: ip a

DNS解析为两条A记录，分别对应两个虚拟IP地址，则可以实现高可用。

##################################
通知脚本的使用方式

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

vi notify.sh #!/bin/bash contact='root@localhost' notify() {     local mailsubject="$(hostname) to be $1, vip floating"     local mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"     echo "$mailbody" | mail -s "$mailsubject" $contact } case $1 in master)     notify master     ;; backup)     notify backup     ;; fault)     notify fault     ;; *)     echo "Usage: $(basename $0) {master|backup|fault}"     exit 1     ;; esac

脚本的调用方法，在vrrp_instance配置段中添加以下通知参数，建议放在虚拟IP的配置项之后：

1 2 3

notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault"

测试，启动或重启keepalived服务，然后查看mail信息
systemctl restart keepalived
systemctl status keepalived
ip a
yum install mailx  //minimal安装的centos默认没有安装mail程序
mail
或者直接查看邮件文件
more /var/mail/root

#################################
增加两台服务器server3和server4，配置为real server (nginx + lvs)
server1(10.71)----server2(10.72)
          |   
server3(10.73)----server4(10.74)

注意同步时间
systemctl restart chronyd

两台服务器均安装nginx
yum install -y nginx
创建各自的首页文件
server3: echo "RS1-73" > /usr/share/nginx/html/index.html
server4: echo "RS2-74" > /usr/share/nginx/html/index.html
启动两台nginx服务器
systemctl start nginx

rs配置脚本：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

vi setrs.sh #!/bin/bash vip=192.168.10.100 mask=255.255.255.255 iface="lo:0" case $1 in start)     echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore     echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore     echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce     echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce     ifconfig $iface $vip netmask $mask broadcast $vip up     route add -host $vip dev $iface     echo "The RS Server is Ready!"     ;; stop)     ifconfig $iface down     echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore     echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore     echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce     echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce     echo "The RS Server is Canceled!"     ;; *)     echo "Usage: $(basename $0) start|stop"     exit 1     ;; esac

分别在server3和server4上执行此脚本
bash -x setrs.sh start
ifconfig
验证文件参数是否更改
more /proc/sys/net/ipv4/conf/all/arp_ignore
more /proc/sys/net/ipv4/conf/all/arp_announce

#################################
在两台keepalived主机上安装lvs和nginx(用于配置sorry server)
yum install ipvsadm nginx
server1: echo "sorry server 71" > /usr/share/nginx/html/index.html
server2: echo "sorry server 72" > /usr/share/nginx/html/index.html
systemctl start nginx

配置单主keepalived，在server1和server2上配置，需要和之前定义的虚拟IP地址相同为192.168.10.100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

virtual_ipaddress {     192.168.10.100/24 dev eth0  //设置虚拟IP .... #注释掉之前的双主配置段VI_2 #vrrp_instance VI_2 {...} virtual_server 192.168.10.100 80 {  //地址同上10.100     delay_loop 6     lb_algo rr     lb_kind DR     protocol TCP     sorry_server 127.0.0.1 80     real_server 192.168.10.73 80 {         weight 1         HTTP_GET {             url {                 path /                 status_code 200             }         connect_timeout 2         nb_get_retry 3         delay_before_retry 1         }     }     real_server 192.168.10.74 80 {         weight 1         HTTP_GET {             url {                 path /                 status_code 200             }         connect_timeout 2         nb_get_retry 3         delay_before_retry 1         }     } }

重启keepalived服务，并查看虚拟地址，如果无法获取地址，则尝试先stop在start
systemctl restart keepalived
systemctl status keepalived
ip a
ipvsadm -Ln  //查看lvs规则，由keepalived自动生成
如果无法查看到后端主机的IP地址，则查看日志信息或messages文件，是否是由于path路径错误导致的，如HTTP status code error to [192.168.10.72]:80 url(/), status_code [301].
然后确认后端主机上是否存在index.html的首页文件。或者直接配置keepalived.conf指定详细的path路径,如path /index.html.

客户端测试,默认为轮询显示，且停止任意一台keepalived，客户端访问无影响；或者停止一台RS来测试：
for i in {1..10};do curl http://192.168.10.100;done
server2: systemctl stop keepalived
server3: systemctl stop nginx

请求的url验证方式也支持md5验证：
HTTP_GET {
    url {
        path /
        digest ff20ad2481f97b1754ef3e12ecd3a9cc
    }
通过如下命令可以生成该md5码
genhash -s 192.168.10.73 -p 8080 -u /
MD5SUM = 9f9b481ce80b3be16685ce4a39ced5cf

#####################
keepalived调用外部的辅助脚本进行资源监控，并根据监控的结果状态能实现优先动态调整；
分两步：(1) 先定义一个脚本；(2) 调用此脚本；
vrrp_script <SCRIPT_NAME> {
    script ""
    interval INT
    weight -INT
}

track_script {
    SCRIPT_NAME_1
    SCRIPT_NAME_2
    ...
}

例如：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

vrrp_script chk_down {     script "[ -f /etc/keepalived/down ] && exit 1 || exit 0"     interval 1     weight -6 } vrrp_script chk_nginx {     script "killall -0 nginx && exit 0 || exit 1"     interval 1     weight -6     fall 2     rise 1 } vrrp_instance VI_1 {     state MASTER     interface eth0     virtual_router_id 14     priority 100     advert_int 1     authentication {         auth_type PASS         auth_pass 571f97b2     }     virtual_ipaddress {         192.168.10.100/16 dev eth0     }     track_script {         chk_down         chk_nginx     } }

测试并查看信息提示IP地址已移除
touch /etc/keepalived/down
tail -f /var/log/messages
ip a

#########################
开机启动服务
systemctl enable ipvsadm
systemctl enable httpd
systemctl enable keepalived

单主配置实例：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

more keepalived.conf ! Configuration File for keepalived global_defs {   notification_email {     root@localhost   }   notification_email_from keepalived@localhost   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id node1 } vrrp_script chk_down {     script "[ -f /etc/keepalived/down ] && exit 1 || exit 0"     interval 1     weight -6 } vrrp_instance VI_1 {     state MASTER     interface eth1     virtual_router_id 51     priority 100     advert_int 1     authentication {         auth_type PASS         auth_pass 1111     }     virtual_ipaddress {         192.168.80.100     }     track_script {         chk_down     } } virtual_server 192.168.80.100 80 {     delay_loop 6     lb_algo rr     lb_kind DR     nat_mask 255.255.255.0     protocol TCP     sorry_server 127.0.0.1 80     real_server 192.168.80.165 80 {         weight 1         HTTP_GET {             url {               path /                 status_code 200             }             connect_timeout 3             nb_get_retry 3             delay_before_retry 3         }     }     real_server 192.168.80.166 80 {         weight 1         HTTP_GET {             url {               path /                 status_code 200             }             connect_timeout 3             nb_get_retry 3             delay_before_retry 3         }     } }

#########################
配置nginx+keepalived来提供高可用前端代理
环境：
websrv1:192.168.80.165
websrv2:192.168.80.166
webshareIP:192.168.80.100
ng-keep-1:192.168.80.151
ng-keep-2:192.168.80.162

代理服务器ng-keep-1的配置

1 2 3 4 5 6 7 8 9 10 11 12 13

vi /etc/nginx/conf.d/nginx.conf upstream proxy {     server 192.168.80.165;     server 192.168.80.166;     server 127.0.0.1 backup;   //backup表示sorry server } server {     listen 80;     server_name 192.168.80.100;     location / {         proxy_pass http://proxy;     } }

配置sorry server测试页
echo "Sorry from nginx-1-151" > /usr/share/nginx/html/index.html
echo "Sorry from nginx-2-162" > /usr/share/nginx/html/index.html

keepalived的配置只需要vrrp_instance配置段，virtual_server,real_server是lvs的配置，nginx用不到，可以删除或注释掉。

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs {   notification_email {     root@localhost   }   notification_email_from keepalived@localhost   smtp_server 127.0.0.1   smtp_connect_timeout 30   router_id node1 } vrrp_script chk_down {     script "[ -f /etc/keepalived/down ] && exit 1 || exit 0"     interval 1     weight -6 } vrrp_instance VI_1 {     state MASTER     interface eth1     virtual_router_id 51     priority 100     advert_int 1     authentication {         auth_type PASS         auth_pass 1111     }     virtual_ipaddress {         192.168.80.100     }     track_script {         chk_down     } }

重启服务ng-keep-1
systemctl restart nginx
systemctl restart keepalived
测试关闭webserver
service httpd stop

客户端测试：
for i in {1..100};do curl 192.168.80.100;sleep 0.5;done