|
一、LVS+Keepalived 介绍 <!--[if !supportLists]-->l <!--[endif]-->LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的自由软件项目之一。 <!--[if !supportLists]-->l <!--[endif]-->目前有三种IP负载均衡技术（VS/NAT、VS/TUN和VS/DR）； 十种调度算法（rrr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）。 <!--[if !supportLists]-->l <!--[endif]-->Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现 <!--[if !supportLists]-->l <!--[endif]-->在DR 模式的集群中,LVS 负载调度器作为群集的访问入口，但不作为网关使用，服务池中所有节点都各自接入Internet， 发送给客户机的web响应数据包不需要经过lvs负载调度器。 二、实验环境与目标网站负载均衡集群拓扑图
系统环境（配置好yum源）： [iyunv@host1 ~]# cat /etc/redhat-release CentOS release 6.2 (Final) [iyunv@host1 ~]# uname -a Linux localhost.localdomain 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux 通过部署Apache应用构建Web服务器测试效果，通过部属iSCSI做为存储服务器。 实验目标： 实现Web服务器的备份和冗余，其中任意借点出现故障都会自动切 换，保证应用正常运行。 三、LVS调度器的配置[iyunv@host1 ~]# modprobe ip_vs [iyunv@host1 ~]# cat /proc/net/ip_vs IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn [iyunv@host1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0:0 DEVICE=eth0:0 ONBOOT=yes BOOTPROTO=static IPADDR=192.168.10.100 NETMASK=255.255.255.0 [iyunv@host1 ~]# /etc/init.d/network restart [iyunv@host1 ~]# ifconfig eth0:0 eth0:0 Link encap:Ethernet HWaddr 00:0C:29:46:D9:DE inet addr:192.168.10.100 Bcast:192.168.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 [iyunv@host1 ~]# vim /etc/sysctl.conf #避免网络内的ARP解析出现异常，应该关闭Linux内核的重定向参数响应 #在最后添加 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0 net.ipv4.conf.eth0.send_redirects = 0 |
[iyunv@host1 ~]# sysctl -p [iyunv@host1 ~]# yum -y install ipvsadm #ipvsadm是负载调度器上使用LVS群集管理工具，通过调用ip_vs模块来添加、删除服务器节点，以及查看群集的运行状态 [iyunv@host1 ~]# ipvsadm -v ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1) [iyunv@host1 ~]# mkdir /etc/sysconfig/ipvsadm [iyunv@host1 ~]# /etc/init.d/ipvsadm restart [iyunv@host1 ~]# chkconfig ipvsadm on [iyunv@host1 ~]# yum -y install gcc kernel-devel openssl-devel popt-devel #安装支持库 [iyunv@host1 ~]# tar -zxf keepalived-1.2.2.tar.gz [iyunv@host1 ~]# cd keepalived-1.2.2 [iyunv@host1 ~]# vi keepalived/libipvs-2.6/ip_vs.h 将#include <sys/types.h>放到#include <linux/types.h>的上面 [iyunv@host1 keepalived-1.2.2]# ./configure --prefix=/ --with-kernel-dir=/usr/src/kernels/2.6.18-194.el5-i686/ [iyunv@host1 keepalived-1.2.2]# make [iyunv@host1 keepalived-1.2.2]# make install [iyunv@host1 keepalived-1.2.2]# chkconfig --add keepalived [iyunv@host1 keepalived-1.2.2]# chkconfig keepalived on [iyunv@host1 keepalived-1.2.2]# cd /etc/keepalived/ [iyunv@host1 keepalived]# cp keepalived.conf keepalived.conf.bak [iyunv@host1 keepalived]# vim keepalived.conf #文件里面所有内容 ! Configuration File for keepalived global_defs { router_id LVS_MASTER #从调度器上改为router_id LVS_SLAVE } vrrp_instance VI_1 { state MASTER #从调度器上改为state SLAVE interface eth0 virtual_router_id 51 priority 100 #从调度器上改为priority 80 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.10.100 } } virtual_server 192.168.10.100 80 { delay_loop 6 lb_algo rr lb_kind DR nat_make 255.255.255.0 persistence_timeout 50 protocol TCP real_server 192.168.10.3 80 { weight 1 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 3 } } real_server 192.168.10.4 80 { weight 1 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 3 } } } |
配置负载分配策略 # /etc/init.d/ipvsadm stop # ipvsadm –A –t 192.168.10.100:80 –s rr # ipvsadm –a –t 192.168.10.100:80 –r 192.168.10.3 –g –w 1 # ipvsadm –a –t 192.168.10.100:80 –r 192.168.10.4 –g –w 1 # /etc/init.d/ipvsadm save # chkconfig ipvsadm on ipvsadm 从调度器上同上所有配置，先安装相关软件包，在安装keepalived，最后配置/etc/keepalived.conf,只需将注释的地方修改即可 四、配置iSCSI存储服务器（1）安装服务端所需软件包 # yum -y install scsi-target-utils （2）准备存储设备 创建一个LVM逻辑卷 # pvcreate /dev/sdb5 # vgcreate vol0 /dev/sdb5 # lvcreate -L 500M -n lv-host4 vol0 # lvscan ACTIVE '/dev/vol0/lv-host4' [500.00 MiB] inherit 创建一个软RAID # mdadm -Cv /dev/md0 -a yes -n3 -l5 /dev/sdb[678] # mdadm -Q /dev/md0 # mdadm -D -s >> /etc/mdadm.conf # cat /etc/mdadm.conf ARRAY /dev/md0 metadata=1.2 spares=1 name=host1.zc.com:0 UUID=a0093bb0:d52fd73f:fb9469f3:0fbee1f2 （3）启动iscsi服务，设置开启启动 # /etc/init.d/tgtd start # netstat -anpt | grep tgt tcp 0 0 0.0.0.0:3260 0.0.0.0:* LISTEN 1850/tgtd tcp 0 0 :::3260 :::* LISTEN 1850/tgtd [iyunv@host1 ~]# chkconfig tgtd on （4）创建ISCSI对象（Target） # tgtadm -L iscsi -o new -m target -t 1 -T iqn.2012-06.com.dong.www:lvm # tgtadm -L iscsi -o new -m target -t 2 -T iqn.2012-06.com.dong.www:raid # tgtadm -L iscsi -o show -m target （5）为客户机分配存储空间（LUN） # tgtadm --lld iscsi -o new -m logicalunit -t 1 -l 1 -b /dev/vol0/lv-host4 # tgtadm --lld iscsi -o new -m logicalunit -t 2 -l 1 -b /dev/md0 # tgtadm --lld iscsi -o bind -m target -t 1 -I 192.168.10.3 # tgtadm --lld iscsi -o bind -m target -t 2 -I 192.168.10.4 # tgtadm --lld iscsi -o bind -m target -t 1 -I 192.168.10.3 # tgtadm --lld iscsi -o bind -m target -t 2 -I 192.168.10.4 # tgtadm --lld iscsi -o show -m target （6）保存target和LUN配置 # vim /etc/tgt/targets.conf #配置永久生效 <target iqn.2012-06.com.dong.www:lvm> #最后添加以下内容 backing-store /dev/vol0/lv-host21 initiator-address 192.168.10.5 </target> <target iqn.2012-06.com.dong.www:raid> backing-store /dev/md0 initiator-address 192.168.10.5 </target> 五、节点配置（1）分别在2台RealServer(节点)上面编写脚本 [iyunv@host3 ~]# vim /etc/init.d/realserver.sh #!/bin/bash VIP=192.168.10.100 startrs() { echo "start LVS of REALServer" /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up /sbin/route add -host $VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce } stoprs() { /sbin/ifconfig lo:0 down echo "close LVS Directorserver" echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce } # ============ Main =========== case $1 in "start") startrs;; "stop") stoprs;; "*") echo "Usage $0 {start|stop}" exit 1 esac |
[iyunv@host3 ~]# chmod +x /etc/init.d/realserver.sh [iyunv@host3 ~]#/etc/init.d/realserver.sh start （2）挂载存储客户端 安装iSCSI客户端相关软件 # yum -y install iscsi-initiator-utils lsscsi # /etc/init.d/iscsi restart # chkconfig iscsi on 发现ISCSI设备 # iscsiadm -m discovery -t sendtargets -p 192.168.10.5 192.168.10.5:3260,1 iqn.2012-06.com.dong.www:lvm 192.168.10.5:3260,1 iqn.2012-06.com.dong.www:raid 查看现有分区 # fdisk -cul 连接ISCSI设备 # iscsiadm -m node -T iqn.2012-06.com.dong.www:lvm --login # iscsiadm -m node -T iqn.2012-06.com.dong.www:raid --login 查看分区 # lsscsi [1:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0 [2:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda [3:0:0:0] storage IET Controller 0001 - [3:0:0:1] disk IET VIRTUAL-DISK 0001 /dev/sdb [4:0:0:0] storage IET Controller 0001 - [4:0:0:1] disk IET VIRTUAL-DISK 0001 /dev/sdc # fdisk -cul …… Disk /dev/sdb: 524 MB, 524288000 bytes 17 heads, 59 sectors/track, 1020 cylinders, total 1024000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 2146 MB, 2146435072 bytes 67 heads, 62 sectors/track, 1009 cylinders, total 4192256 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdc doesn't contain a valid partition table 将/dev/sdc划分并使用 # partprobe /dev/sdc # fdisk -cu /dev/sdc # fdisk -cul /dev/sdc Disk /dev/sdc: 2146 MB, 2146435072 bytes 15 heads, 2 sectors/track, 139741 cylinders, total 4192256 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x8338546c Device Boot Start End Blocks Id System /dev/sdc1 2048 4192255 2095104 83 Linux # mkfs.ext4 /dev/sdc1 # mkdir /opt/data # mount /dev/sdc1 /opt/data/ #临时挂载，重启失效 # cat /etc/mtab /dev/sda2 / ext4 rw 0 0 proc /proc proc rw 0 0 sysfs /sys sysfs rw 0 0 devpts /dev/pts devpts rw,gid=5,mode=620 0 0 tmpfs /dev/shm tmpfs rw 0 0 /dev/sda1 /boot ext4 rw 0 0 /dev/sda3 /data ext4 rw 0 0 none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0 /dev/sdc1 /opt/data ext4 rw 0 0 # blkid /dev/sdb1 >> /etc/fstab #挂载永久生效 # vim /etc/fstab ...... //省略部分信息 UUID="78e6d91e-10bf-4180-b308-4259145853d6" /var/www/html/ ext4 defaults,_netdev 0 0 (3)在2台RealServer(节点)中分别配置HTTP并启动realserver.sh [iyunv@host3 ~]# yum -y install httpd [iyunv@host3 ~]# cd /var/www/html/ [iyunv@host3 html]# echo "192.168.10.3" > index.html [iyunv@host3 html]# cat index.html 192.168.10.3 [iyunv@host4 html]# cat index.html #两台节点的配置除了网页不一样（为了更明显的看到测试效果），其他都一样 192.168.10.4 [iyunv@host3 ~]# /etc/init.d/realserver.sh start [iyunv@host3 ~]# echo "/etc/init.d/realserver.sh start" >> /etc/rc.local [iyunv@host3 html]# /etc/init.d/httpd start [iyunv@host3 html]# chkconfig httpd on (4)给2台节点添加一条路由记录，将访问VIP的数据限制在本地以避免通信紊乱 [iyunv@host4 ~]# route add -host 192.168.10.100 dev lo:0 [iyunv@host4 ~]# echo "/sbin/route add -host 192.168.10.100 dev lo:0" >> /etc/rc.local 另一台的配置过程一样，过程略 六、测试在两台调度器上分别启动Keepalived [iyunv@host1 ~]# /etc/init.d/keepalived restart [iyunv@host1 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.10.100:80 rr persistent 50 -> 192.168.10.3:80 Route 1 0 0 -> 192.168.10.4:80 Route 1 0 0 使用客户机访问<http://192.168.10.100>,使主调度器挂掉看是否能够访问 使用客户机访问<http://192.168.10.100>,使一个节点挂掉看是否能够访问 | 
QQ群⑧：
窥视卡
雷达卡
发表于 2012-8-30 08:39:51
提升卡
置顶卡
沉默卡
喧嚣卡
变色卡
千斤顶
显身卡