负载均衡软件HAProxy案例二 带cookie前缀和高可用的HTTP负载均衡

760176104

　　案例二  带cookie前缀和高可用的HTTP负载均衡
　　如果不想增加多余的cookie，而是用现成的cookie。后端应用已经生成了一个名为JSESSIONID的cookie，下面将在这个cookie前面加上服务器名称的前缀。由于负载均衡器是整个架构中最为重要的部件，所以这里通过使用Keepalived为LB1作备份。
　　Keepalived 软件可以通过yum install keepalived 安装

　　

　　LB1和LB2共享一个VIP 192.168.1.1，它们通过Keepalived进行VIP的接管。

　　在LB1和LB2配置HAProxy如下内容:

listen webfarm *:80
       mode http
       balance roundrobin
       cookie JSESSIONID prefix
       option httpclose
       option forwardfor
       option httpchk HEAD /index.html HTTP/1.0
       server webA 192.168.1.11:80 cookie A check
       server webB 192.168.1.12:80 cookie B check
       server webC 192.168.1.13:80 cookie C check
       server webD 192.168.1.14:80 cookie D check　　这里需要注意一下， listen 绑定的IP地址必须是真实存在的IP地址，如果不存在HAProxy将启动失败，如，LB2还没有接管192.168.1.1这个IP地址，那么在LB2上HAProxy将启动失败。为了避免因为VIP没有被接管而启动不了HAProxy的情况发生，可以将绑定IP和端口设置成 *.80 或 0.0.0.0:80 .
　　

　　在LB1和LB2配置Keepalived如下内容:
! Configuration File for keepalived
global_defs {
   notification_email {
     admin@example.com                  设置通知接收邮件地址
   }
   notification_email_from lb1@example.com    设置通知发送邮件地址
   smtp_server 127.0.0.1                     
   smtp_connect_timeout 30
   router_id LB1_MASTER
}
vrrp_script chk_haproxy {                  检查HAProxy是否存在
        script "killall -0 haproxy"   
        interval 2                     每2秒检查一下HAProxy进程
weight 2                        
    }


vrrp_instance VI_1 {
    state MASTER                    LB1上设置为MASTER,LB2上设置为BACKUP
    interface eth0
    virtual_router_id 51            
    priority 100             MASTER设置为100，BACKUP设置为99，MASTER的值必须大于BACKUP
    advert_int 1             设置MASTER和BACKUP之间同步检查时间间隔
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
    192.168.1.1/24  brd 192.168.1.255  dev eth0 label eth0:ha
             设置虚拟IP地址，为了便于查看虚拟IP，可以设置一个
    }
    track_script {
        chk_haproxy
                 }
}　　

　　在LB1上启动HAProxy和Keepalived，查看/var/log/messages日志
Jun  2 15:26:21 LB1 Keepalived[14391]: Starting Keepalived v1.2.7 (02/21,2013)
Jun  2 15:26:21 LB1 Keepalived[14392]: Starting Healthcheck child process, pid=14394
Jun  2 15:26:21 LB1 Keepalived[14392]: Starting VRRP child process, pid=14395
Jun  2 15:26:21 LB1 Keepalived_vrrp[14395]: Interface queue is empty
Jun  2 15:26:21 LB1 Keepalived_vrrp[14395]: Netlink reflector reports IP 192.168.1.3 added
Jun  2 15:26:21 LB1 Keepalived_vrrp[14395]: Netlink reflector reports IP fe80::a00:27ff:fec8:f51b added
Jun  2 15:26:21 LB1 Keepalived_vrrp[14395]: Registering Kernel netlink reflector
Jun  2 15:26:21 LB1 Keepalived_vrrp[14395]: Registering Kernel netlink command channel
Jun  2 15:26:21 LB1 Keepalived_healthcheckers[14394]: Interface queue is empty
Jun  2 15:26:21 LB1 Keepalived_healthcheckers[14394]: Netlink reflector reports IP 192.168.1.3 added
Jun  2 15:26:21 LB1 Keepalived_healthcheckers[14394]: Netlink reflector reports IP fe80::a00:27ff:fec8:f51b added
Jun  2 15:26:21 LB1 Keepalived_healthcheckers[14394]: Registering Kernel netlink reflector
Jun  2 15:26:21 LB1 Keepalived_healthcheckers[14394]: Registering Kernel netlink command channel
Jun  2 15:26:21 LB1 Keepalived_vrrp[14395]: Registering gratuitous ARP shared channel
Jun  2 15:26:41 LB1 Keepalived_vrrp[14395]: Opening file '/etc/keepalived/keepalived.conf'.
Jun  2 15:26:41 LB1 Keepalived_healthcheckers[14394]: Opening file '/etc/keepalived/keepalived.conf'.
Jun  2 15:26:41 LB1 Keepalived_vrrp[14395]: Configuration is using : 38559 Bytes
Jun  2 15:26:41 LB1 Keepalived_vrrp[14395]: Using LinkWatch kernel netlink reflector...
Jun  2 15:26:41 LB1 Keepalived_healthcheckers[14394]: Configuration is using : 6366 Bytes
Jun  2 15:26:41 LB1 Keepalived_vrrp[14395]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Jun  2 15:26:41 LB1 Keepalived_healthcheckers[14394]: Using LinkWatch kernel netlink reflector...
Jun  2 15:26:41 LB1 Keepalived_vrrp[14395]: VRRP_Script(chk_haproxy) succeeded
Jun  2 15:26:42 LB1 Keepalived_vrrp[14395]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun  2 15:26:43 LB1 Keepalived_vrrp[14395]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun  2 15:26:43 LB1 Keepalived_vrrp[14395]: VRRP_Instance(VI_1) setting protocol VIPs.
Jun  2 15:26:43 LB1 Keepalived_vrrp[14395]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.1
Jun  2 15:26:43 LB1 Keepalived_healthcheckers[14394]: Netlink reflector reports IP 192.168.1.1 added
Jun  2 15:26:48 LB1 Keepalived_vrrp[14395]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.1
Jun  2 15:27:19 localhost haproxy[14376]: 192.168.2.101:64188 [02/Jun/2014:15:27:19.465] webfarm webfarm/webA 3/0/2/1/13 200 850 - - CDNN 1/1/0/0/0 0/0 "GET / HTTP/1.1"　　

　　从日志中可以看到Keepalived的启动过程，Keepalived主进程启动后启动Healthcheck 和VRRP两个子进程，然后Keepalived的Netlink reflector检查本地IP地址设置，然后读取/etc/keepalived.conf配置文件，最后根据是否为MASTER或BACKUP设置VIP。
　　

　　在LB2上启动HAProxy和Keepalived，查看/var/log/messages日志
Jun  2 15:48:00 localhost Keepalived[13673]: Starting Keepalived v1.2.7 (02/21,2013)
Jun  2 15:48:00 localhost Keepalived[13674]: Starting Healthcheck child process, pid=13675
Jun  2 15:48:00 localhost Keepalived[13674]: Starting VRRP child process, pid=13677
Jun  2 15:48:00 localhost Keepalived_vrrp[13677]: Interface queue is empty
Jun  2 15:48:00 localhost Keepalived_vrrp[13677]: Netlink reflector reports IP 192.168.1.104 added
Jun  2 15:48:00 localhost Keepalived_vrrp[13677]: Netlink reflector reports IP fe80::a00:27ff:fec2:457 added
Jun  2 15:48:00 localhost Keepalived_vrrp[13677]: Registering Kernel netlink reflector
Jun  2 15:48:00 localhost Keepalived_vrrp[13677]: Registering Kernel netlink command channel
Jun  2 15:48:00 localhost Keepalived_vrrp[13677]: Registering gratuitous ARP shared channel
Jun  2 15:48:00 localhost Keepalived_healthcheckers[13675]: Interface queue is empty
Jun  2 15:48:00 localhost Keepalived_healthcheckers[13675]: Netlink reflector reports IP 192.168.1.104 added
Jun  2 15:48:00 localhost Keepalived_healthcheckers[13675]: Netlink reflector reports IP fe80::a00:27ff:fec2:457 added
Jun  2 15:48:00 localhost Keepalived_healthcheckers[13675]: Registering Kernel netlink reflector
Jun  2 15:48:00 localhost Keepalived_healthcheckers[13675]: Registering Kernel netlink command channel
Jun  2 15:48:20 localhost Keepalived_healthcheckers[13675]: Opening file '/etc/keepalived/keepalived.conf'.
Jun  2 15:48:20 localhost Keepalived_healthcheckers[13675]: Configuration is using : 6364 Bytes
Jun  2 15:48:20 localhost Keepalived_vrrp[13677]: Opening file '/etc/keepalived/keepalived.conf'.
Jun  2 15:48:20 localhost Keepalived_vrrp[13677]: Configuration is using : 38557 Bytes
Jun  2 15:48:20 localhost Keepalived_vrrp[13677]: Using LinkWatch kernel netlink reflector...
Jun  2 15:48:20 localhost Keepalived_vrrp[13677]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jun  2 15:48:20 localhost Keepalived_vrrp[13677]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Jun  2 15:48:20 localhost Keepalived_healthcheckers[13675]: Using LinkWatch kernel netlink reflector...
Jun  2 15:48:20 localhost Keepalived_vrrp[13677]: VRRP_Script(chk_haproxy) succeeded　　

　　查看LB1的IP地址
eth0      Link encap:Ethernet  HWaddr 08:00:27:C8:F5:1B  
          inet addr:192.168.1.3  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fec8:f51b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:111413 errors:0 dropped:0 overruns:0 frame:0
          TX packets:121677 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:19049418 (18.1 MiB)  TX bytes:11302763 (10.7 MiB)
eth0:ha   Link encap:Ethernet  HWaddr 08:00:27:C8:F5:1B  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:713 errors:0 dropped:0 overruns:0 frame:0
          TX packets:713 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:134641 (131.4 KiB)  TX bytes:134641 (131.4 KiB　　可以看到由于LB1上的Keepalived设置为MASTER状态并且HAProxy进程存在，所以LB1上的Keepalived将接管VIP 192.168.1.1。
　　

　　在这个架构中，HAProxy代理将修改客户端和服务器发送的每一个cookie，确保HAProxy能够访问每个session会话中的所有请求的所有cookie很重要。所以，如果不确定客户端一定不使用keep-alive(HTTP 1.1),则不要轻易去掉 option  httpcolse 这个选项。
　　如果一个请求不包含任何cookie，它将被转发到后端一个可用的服务器。反过来，如果一个名为JSESIONID的cookie出现，HAProxy将会把这个cookie的值修改为JSESIONID=A~xxx这样的形式。

　　当一个客户端再次请求服务器响应，并在请求中带有JSESSIONID=A~xxx 这样的cookie，LB1会直接将它转发给webA。在这个请求到达webA之前，cookie中的A~ 将被移除。
　　如果webA不可用，请求将被转发到另外的可用服务器上，响应的cookie将被重置。

　　

　　

　　在有些情况下，在同一个集群下的服务器，有些处理能力较强，有些处理能力较弱，这种情况，可以通过HAProxy为不同性能的服务器设置不同的权重，处理能力越强，权重越高。
       server webA 192.168.1.11:80 cookie A weight 12 check
       server webB 192.168.1.12:80 cookie B weight 12 check
       server webC 192.168.1.13:80 cookie C weight 26 check
       server webD 192.168.1.14:80 cookie D weight 26 check　　

　　下面测试一下故障切换
　　停掉LB1上的HAProxy，查看LB1和LB2的Keepalived相关日志和IP接管情况
　　LB1上日志如下:
Jun  2 17:45:46 LB1 Keepalived_vrrp[21299]: VRRP_Script(chk_haproxy) failed
Jun  2 17:45:47 LB1 Keepalived_vrrp[21299]: VRRP_Instance(VI_1) Received higher prio advert
Jun  2 17:45:47 LB1 Keepalived_vrrp[21299]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jun  2 17:45:47 LB1 Keepalived_vrrp[21299]: VRRP_Instance(VI_1) removing protocol VIPs.
Jun  2 17:45:47 LB1 Keepalived_healthcheckers[21298]: Netlink reflector reports IP 192.168.1.1 removed　　可以看到在LB1上停掉HAProxy后，Keepalived检查HAProxy进程不存在，VRRP进入BACKUP状态，Keepalived将移除VIP 192.168.1.1
　　

　　LB2上日志如下:
Jun  2 17:45:47 localhost Keepalived_vrrp[13677]: VRRP_Instance(VI_1) forcing a new MASTER election
Jun  2 17:45:48 localhost Keepalived_vrrp[13677]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun  2 17:45:49 localhost Keepalived_vrrp[13677]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun  2 17:45:49 localhost Keepalived_vrrp[13677]: VRRP_Instance(VI_1) setting protocol VIPs.
Jun  2 17:45:49 localhost Keepalived_vrrp[13677]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.1
Jun  2 17:45:49 localhost Keepalived_healthcheckers[13675]: Netlink reflector reports IP 192.168.2.21 added
Jun  2 17:45:54 localhost Keepalived_vrrp[13677]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.1　　LB2上的VRRP进入MASTER状态，接管VIP 192.168.1.1
　　查看LB2的IP

eth0      Link encap:Ethernet  HWaddr 08:00:27:C2:04:57  
          inet addr:192.168.1.4  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fec2:457/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:130629 errors:0 dropped:0 overruns:0 frame:0
          TX packets:109630 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:19059352 (18.1 MiB)  TX bytes:8767499 (8.3 MiB)
eth0:ha   Link encap:Ethernet  HWaddr 08:00:27:C2:04:57  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:88 errors:0 dropped:0 overruns:0 frame:0
          TX packets:88 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:17488 (17.0 KiB)  TX bytes:17488 (17.0 KiB)　　

　　然后将LB1上的HAProxy重新启动，查看LB1和LB2的Keepalived日志和IP接管情况
Jun  2 18:13:16 LB1 Keepalived_vrrp[24199]: VRRP_Script(chk_haproxy) succeeded
Jun  2 18:13:16 LB1 Keepalived_vrrp[24199]: VRRP_Instance(VI_1) forcing a new MASTER election
Jun  2 18:13:17 LB1 Keepalived_vrrp[24199]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun  2 18:13:18 LB1 Keepalived_vrrp[24199]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun  2 18:13:18 LB1 Keepalived_vrrp[24199]: VRRP_Instance(VI_1) setting protocol VIPs.
Jun  2 18:13:18 LB1 Keepalived_vrrp[24199]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.1
Jun  2 18:13:18 LB1 Keepalived_healthcheckers[24198]: Netlink reflector reports IP 192.168.1.1 added
Jun  2 18:13:23 LB1 Keepalived_vrrp[24199]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.1　　从日志中可以看到当LB1上的HAProxy重新启动后，Keepalived检查到HAProxy进程存在，然后VRRP根据优先级重新选举MASTER，从LB2接管VIP192.168.1.1