keepalived+nginx 双主模型实现高可用服务

uth5321 · 发表于 2015-10-19 09:13:04

一、keepalived的工作原理

keepalived是以VRRP协议为实现基础的，VRRP全称Virtual Router Redundancy Protocol，即虚拟路由冗协议。

虚拟路由冗余协议，可以认为是实现路由器高可用的协议，即将N台提供相同功能的路由器组成一个虚拟路由器组，这个组里面有一个master和多个backup，master上面有一个对外提供服务的vip（该路由器所在局域网内其他机器的默认路由为该vip），master会发组播，当backup收不到vrrp包时就认为master出问题了，这时就需要根据VRRP的优先级从backup中选举出一个master。这样的话就可以保证路由器的高可用了。

keepalived主要有三个模块，分别是core、checkers和vrrp。core模块为keepalived的核心，负责主进程的启动、维护以及全局配置文件的加载和解析。checkers负责健康检查，包括常见的各种检查方式(tcp、http、ssl)。vrrp模块是来实现VRRP协议的。

二、keepalived的配置文件 keepalived只有一个配置文件keepalived.conf，里面主要包括以下几个配置区域，分别是global_defs、vrrp_script、vrrp_instance和virtual_server。
三、keepalived双主模型高可用nginx服务

QQ截图20151019091123.png

3.1、环境设置

keepalived-nginx1 ： 172.16.16.11
keepalived-nginx2 ： 172.16.16.12
vip1 ： 172.16.16.9
vip2: 172.16.16.10

upstream_server1 : 172.16.16.3
upstream_server2: 172.16.16.4

3.2、准备工作
（1）设置nginx1 ，nginx2 关闭selinux和iptables

1
2
3
4
5

sed -i 's@^SELINUX=.*@SELINUX=permissive@' /etc/selinux/config
setenforce 0
iptables -F
service iptables stop &> /dev/null
chkconfig iptables off

（2）配置epel源，查看相关包所在yum源

1
2
3
4
5
6
7

[iyunv@localhost ~]# vim /etc/resolv.conf       //配置域名服务器
nameserver 172.16.0.1

[iyunv@localhost ~]# yum list all keepalived
keepalived.x86_64       1.2.13-4.el6    centos6.6
[iyunv@localhost ~]# yum list all nginx
nginx.x86_64          1.0.15-12.el6    epel

(3) 修改keepalived主备节点的主机名

1
2
3
4

sed -i 's@HOSTNAME=.*@HOSTNAME=nginx1@' /etc/sysconfig/network //设置主机名
sed -i 's@HOSTNAME=.*@HOSTNAME=nginx2@' /etc/sysconfig/network
hostname nginx1
hostname nginx2

（4）主机互信

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

vim /etc/hosts                   //添加域名解析
172.16.16.11 nginx1
172.16.16.12 nginx2
172.16.16.3 web1.bengbengtu.com web1
172.16.16.4 web2.bengbengtu.com web2

[iyunv@nginx1 ~]# yum install openssh-clients
[iyunv@nginx1 ~]# ssh-keygen -t rsa -P ''             //生成一对密钥
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
fa:14:8d:95:f3:48:bf:d1:b8:13:b4:dd:89:0e:51:97 root@nginx1
The key's randomart image is:
+--[ RSA 2048]----+
|          . ..|
|          .. .E |
|       =.. |
|       = *.=...|
|       S o.B.o..|
|    . .  o= |
|    . . +. |
|    o    . |
|       .       |
+-----------------+
[iyunv@nginx1 ~]# ssh-copy-id -i .ssh/id_rsa.pub nginx2 //公钥发给nginx2

[iyunv@nginx2 ~]# ssh-keygen -t rsa -P ''
[iyunv@nginx2 ~]# ssh-copy-id -i .ssh/id_rsa.pub nginx1 //公钥发给nginx1

（5）同步时间

1
2
3
4
5
6
7

  # yum install ntpdate
[iyunv@nginx1 ~]# ntpdate ntp.sjtu.edu.cn       //同步时间
[iyunv@nginx1 ~]# hwclock -w

[iyunv@nginx1 ~]# date ; ssh nginx2 'date'    //查看nginx1和nginx2时间是否同步
   Thu Oct 15 22:44:40 CST 2015
   Thu Oct 15 22:44:41 CST 2015

3.3、安装并配置nginx
这里的nginx用做反向代理，并检查后端upstream的

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

[iyunv@nginx1 ~]# yum -y install nginx ; ssh nginx2 'yum -y install nginx'

#vim /etc/nginx/nginx.conf
worker_processes  2;       //定义2个worker进程
upstream web {                //定义上游服务器
   server 172.16.16.3:80  weight=1 max_fails=2 fail_timeout=30s;
   server 172.16.16.4:80  weight=2 max_fails=3 fail_timeout=40s;
}

#find /etc/nginx/conf.d/ -name '*.conf' -exec mv {} {}.bak \;
# vim /etc/nginx/conf.d/webserver.conf          // 定义server
      server {
      listen 80;
      server_name nginx1 nginx2;

      location / {
      proxy_pass    //反向代理至后端的上游服务器
      }
}

3.4、后端的upstream_server安装httpd
web1.bengbengtu.com : 172.16.16.3
web2.bengbengtu.com : 172.16.16.4
只需安装httpd

（1)、设置主机名

1 2	# hostname web1.bengbengtu.com # hostname web2.bengbengtu.com

（2)、安装httpd服务

1
2
3
4
5
6
7
8
9
10

# yum install -y httpd
[iyunv@web1 ~]# echo "<h1> web1.bengbengtu.com - 172.16.16.3 </h1>" > /var/www/html/index.html
[iyunv@web1 ~]# echo "<h1> web2.bengbengtu.com - 172.16.16.4 </h1>" > /var/www/html/index.html
#启动服务出现如下错误
Starting httpd: httpd: apr_sockaddr_info_get() failed for web2.bengbengtu.com
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
解决方法：
# vim /etc/httpd/conf/httpd.conf
ServerName web1.bengbengtu.com
ServerName web2.bengbengtu.com

先测试一下~~~

3.5、安装配置keepalived高可用nginx服务
说明：如果要监控nginx服务是否是在线状态，需要用到监控系统来实现nginx服务的重启操作！！
nginx1服务器上的keepalived配置如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

[iyunv@nginx1 ~]# vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {                               //全局定义
notification_email {
      root@localhost          //给root发邮件
}
notification_email_from keepalive@localhost
smtp_server 127.0.0.1       //定义邮件服务器
smtp_connect_timeout 30       //连接邮件服务器的超时时间
router_id LVS_DEVEL             //路由id
}

vrrp_instance VI_1 {          //定义虚拟路由实例
state MASTER          //主路由
interface eth0
virtual_router_id 235       //虚拟路由id
priority 100             //优先级
advert_int 1
authentication {             //明文加密认证
      auth_type PASS
      auth_pass 2b316a978532
}

virtual_ipaddress {       //定义vip1
      172.16.16.9/16
}
}

vrrp_instance VI_2 {
state BACKUP       //备路由
interface eth0
virtual_router_id 236
priority 99       //优先级
advert_int 1
authentication {
      auth_type PASS
      auth_pass 24985cea886c
}

virtual_ipaddress {
      172.16.16.10/16       //vip2
}
}

nginx2服务器上的keepalived配置如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

[iyunv@nginx2 ~]# vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
notification_email {
      root@localhost
}
notification_email_from keepalive@localhost
smtp_server 127.0.0.1             //定义邮件服务器
smtp_connect_timeout 30          //连接邮件服务器的超时时间
router_id LVS_DEVEL          //路由id
}

vrrp_instance VI_1 {
state BACKUP          //备路由
interface eth0
virtual_router_id 235
priority 99             //优先级
advert_int 1
authentication {
      auth_type PASS
      auth_pass 2b316a978532
}

virtual_ipaddress {             //定义vip1
      172.16.16.9/16
}
}

vrrp_instance VI_2 {
state MASTER          //主路由
interface eth0
virtual_router_id 236    //虚拟路由id
priority 100             //优先级
advert_int 1
authentication {          //明文认证
      auth_type PASS
      auth_pass 24985cea886c
}

virtual_ipaddress {
      172.16.16.10/16          //定义vip2
}
}

测试：
（1）停掉nginx1的keepalived，查nginx2，vip1，vip2都在；
[iyunv@nginx1 ~]# service keepalived stop
QQ截图20151019091218.png