Nginx + Keepalived 实例

0755mx

Nginx + Keepalived 实例
(2013-08-06 11:37:20)转载▼标签：nginx
keepalived
高可用
负载均衡
主备
分类：LinuxNginx_Master: 192.168.1.103 提供负载均衡Nginx_BackUp: 192.168.1.104 负载均衡备机
Nginx_VIP_TP: 192.168.1.108 网站的 VIP 地址(虚拟 IP)
原理：VIP 是外网访问的IP地址，通过 keepalived 设置，以及 VRRP 将 VIP绑定到主机和备机上，通过权重实现控制。当主机挂掉后，keepalived 释放对主机的控制，备机接管VIP。
扩展：主机和备机可进一步延伸，它们只通过 Nginx 提供负载均衡。再让其它的机器提供真实的 web 服务。
**********************************************************************************************安装 Nginx (省略)-----------------------http://blog.sina.com.cn/s/blog_5f54f0be0100yqm7.html
Keepalived 安装-----------------------地址: http://www.keepalived.org/download.htmlwgethttp://www.keepalived.org/software/keepalived-1.2.7.tar.gztar -zxf keepalived-1.2.7.tar.gzcd keepalived-1.2.7
64 位系统：./configure --sysconf=/etc --prefix=/usr/local/keepalived--with-kernel-dir=/usr/src/kernels/2.6.32-358.2.1.el6.x86_64/或 32 位系统:./configure --sysconf=/etc --prefix=/usr/local/keepalived--with-kernel-dir=/usr/src/kernels/2.6.32-358.6.2.el6.i686/
可通过 getconf LONG_BIT 得到系统位数。
参数解释:--sysconf 指定了配置文件的地址.即：/etc/keepalived/keepalived.conf--prefix 指定了安装目录--with-kernel-dir 指定使用内核源码中的头文件,即 include 目录.只有使用 LVS时才需要这个参数，其它的时候不需要。
如果报错:configure: error: Popt libraries is required
解决:yum install popt-devel
再 configue .成功后提示：
Keepalived configuration------------------------Keepalived version      : 1.2.7Compiler               :gccCompiler flags          : -g-O2Extra Lib              : -lpopt-lssl -lcrypto Use IPVS Framework      : YesIPVS sync daemon support : YesIPVS use libnl          : NoUse VRRP Framework      : YesUse VRRP VMAC          : YesSNMP support           : NoUse Debug flags         : No
安装：makemake install
设置成为服务并开机启动:cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
/etc/rc.d/init.d/keepalived statuschkconfig --add keepalivedchkconfig keepalived on
设置主机上的配置文件内容：
vi/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {   notification_email{    sunyu@easymobi.cn    wuxuegang.123@163.com   }   notification_email_frompub@easymobi.cn   smtp_server127.0.0.1   smtp_connect_timeout30   router_id LVS_DEVEL}
vrrp_instance VI_1 {    state MASTER    interface eth0    virtual_router_id51    #此处是主 Nginx 的 IP 地址.    mcast_src_ip192.168.1.103    #该机的 priority(优先) 为 100    priority 100    advert_int 1    authentication {       auth_type PASS       auth_pass 1111222    }    virtual_ipaddress{       192.168.1.108    }}
前面的结构那里已经规定好了 VIP 和 主备机的 IP, 所以这里按上面的填。
备机的配置文件:
! Configuration File for keepalived
global_defs {   notification_email{    sunyu@easymobi.cn    wuxuegang.123@163.com   }   notification_email_frompub@easymobi.cn   smtp_server127.0.0.1   smtp_connect_timeout30   router_id LVS_DEVEL}
vrrp_instance VI_1 {    state SLAVER    interface eth0    virtual_router_id51    #此处是备 Nginx 的 IP 地址.    mcast_src_ip192.168.1.104    #该机的 priority(优先) 为 99    priority 99    advert_int 1    authentication {       auth_type PASS       auth_pass 1111222    }    virtual_ipaddress{       192.168.1.108    }}
这时候 ping 192.168.1.108 是不通的.
然后在两台机器上分别启动 keepalived 服务
这时候再 ping 192.168.1.108 .通了.实际上这时候 108 是被绑到主机上的。在主机上：查看系统日志:tailf/var/log/messagesMay 29 18:32:16 localhost Keepalived_vrrp[27731]: Opening file'/etc/keepalived/keepalived.conf'.May 29 18:32:16 localhost Keepalived_vrrp[27731]:Configuration is using : 62906 BytesMay 29 18:32:16 localhost Keepalived_vrrp[27731]: UsingLinkWatch kernel netlink reflector...May 29 18:32:16 localhost Keepalived_healthcheckers[27729]:Using LinkWatch kernel netlink reflector...May 29 18:32:16 localhost Keepalived_vrrp[27731]: VRRPsockpool: [ifindex(2), proto(112), fd(11,12)]May 29 18:32:17 localhost Keepalived_vrrp[27731]:VRRP_Instance(VI_1) Transition to MASTER STATEMay 29 18:32:18 localhost Keepalived_vrrp[27731]:VRRP_Instance(VI_1) Entering MASTER STATEMay 29 18:32:18 localhost Keepalived_vrrp[27731]:VRRP_Instance(VI_1) setting protocol VIPs.May 29 18:32:18 localhost Keepalived_vrrp[27731]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for192.168.1.108May 29 18:32:18 localhost Keepalived_healthcheckers[27729]:Netlink reflector reports IP 192.168.1.108 added
可以看到.VRRP(虚拟路由冗余协议)已经启动.我们可以通过命令 ip addr 来检查主 Nginx上的 IP 分配情况.
[iyunv@localhost ~]# ipaddr1: lo: mtu 16436 qdisc noqueue stateUNKNOWN     link/loopback00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scopehost lo    inet6 ::1/128 scopehost       valid_lft forever preferred_lft forever2: eth0: mtu 1500 qdisc mq state UP qlen 1000    link/ether00:15:c5:ef:53:8c brd ff:ff:ff:ff:ff:ff    inet 192.168.1.103/25brd 192.168.1.255 scope global eth0    inet 192.168.1.108/32 scope global eth0    inet6fe80::215:c5ff:feef:538c/64 scope link       valid_lft forever preferred_lft forever3: eth1: mtu 1500 qdisc noop state DOWN qlen 1000    link/ether00:15:c5:ef:53:8e brd ff:ff:ff:ff:ff:ff
可以看到 VIP 地址已经绑定到主 Nginx 机器上: inet 192.168.1.108/32 scopeglobal eth0
我们通过 tcpdump 抓包：
[iyunv@localhost ~]# tcpdumpvrrptcpdump: verbose output suppressed, use -v or -vv for fullprotocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size65535 bytes13:38:27.797982 IP htuidc.bgp.ip >vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtypesimple, intvl 1s, length 2013:38:28.794693 IP htuidc.bgp.ip >vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtypesimple, intvl 1s, length 2013:38:29.794518 IP htuidc.bgp.ip >vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtypesimple, intvl 1s, length 2013:38:30.798581 IP htuidc.bgp.ip >vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtypesimple, intvl 1s, length 2013:38:31.795902 IP htuidc.bgp.ip >vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtypesimple, intvl 1s, length 2013:38:32.804050 IP htuidc.bgp.ip >vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtypesimple, intvl 1s, length 2013:38:33.801191 IP htuidc.bgp.ip >vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtypesimple, intvl 1s, length 2013:38:34.798793 IP htuidc.bgp.ip >vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtypesimple, intvl 1s, length 20
这样，一个 Nginx + Keepalived 的架构就完成了。
监控和主备切换接下来可以完善一下，加上实时监控，如果发现负载均衡的 Nginx 出现问题，就将该机器上的 Keepalived服务停掉。nginx_check.sh:
#!/bin/bashwhile :donginxpid = 'ps -C nginx --no-header | wc -l'if[ $nginxpid -eq 0 ];thenservice nginx startsleep 3nginxpid = 'ps -C nginx --no-header | wc -l'echo $nginxpidif[ $nginxpid -eq 0 ];thenservice keepalived stopfifisleep 3done
然后让该脚本一直在后台运行:nohup /etc/nginx_check.sh或者将它添加成服务，让它开机自启动:http://blog.sina.com.cn/s/blog_5f54f0be0101b3bs.html
测试:--------------------在两台机器的 web 服务器上分别放一个 index.html, 里面内容分别是自己机器的IP.通过VIP访问：http://192.168.1.108/index.html 发现显示的是主机的IP.此时，关掉主机的 nginx, 这时候由于上面的监控脚本。主机的 keepalived也会关闭。这时候再访问上面地址，发现显示的是备机的IP。可见，切换成功。
负载均衡--------------------http://blog.sina.com.cn/s/blog_5f54f0be0100zvf5.html通过 ip_hash 实现会话保持