构建LVS+keepalived+iSCSI集群（DR）
一、LVS+Keepalived 介绍<!---->l<!---->LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的自由软件项目之一。<!---->l<!---->目前有三种IP负载均衡技术（VS/NAT、VS/TUN和VS/DR）；十种调度算法（rrr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）。<!---->l<!---->Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现<!---->l<!---->在DR 模式的集群中,LVS 负载调度器作为群集的访问入口，但不作为网关使用，服务池中所有节点都各自接入Internet， 发送给客户机的web响应数据包不需要经过lvs负载调度器。 二、实验环境与目标网站负载均衡集群拓扑图系统环境（配置好yum源）： # cat /etc/redhat-release CentOS release 6.2 (Final) # uname -aLinux localhost.localdomain 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux 通过部署Apache应用构建Web服务器测试效果，通过部属iSCSI做为存储服务器。实验目标：实现Web服务器的备份和冗余，其中任意借点出现故障都会自动切换，保证应用正常运行。 三、LVS调度器的配置# modprobe ip_vs # cat /proc/net/ip_vsIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn # vim /etc/sysconfig/network-scripts/ifcfg-eth0:0DEVICE=eth0:0ONBOOT=yesBOOTPROTO=staticIPADDR=192.168.10.100NETMASK=255.255.255.0 # /etc/init.d/network restart# ifconfig eth0:0eth0:0 Link encap:EthernetHWaddr 00:0C:29:46:D9:DE inet addr:192.168.10.100Bcast:192.168.10.255Mask:255.255.255.0 UP BROADCAST RUNNING MULTICASTMTU:1500Metric:1 # vim /etc/sysctl.conf #避免网络内的ARP解析出现异常，应该关闭Linux内核的重定向参数响应#在最后添加
net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0net.ipv4.conf.eth0.send_redirects = 0
# sysctl -p # yum -y install ipvsadm #ipvsadm是负载调度器上使用LVS群集管理工具，通过调用ip_vs模块来添加、删除服务器节点，以及查看群集的运行状态 # ipvsadm -v ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1) # mkdir /etc/sysconfig/ipvsadm # /etc/init.d/ipvsadm restart # chkconfig ipvsadm on # yum -y install gcc kernel-devel openssl-devel popt-devel#安装支持库 # tar -zxf keepalived-1.2.2.tar.gz # cd keepalived-1.2.2 # vi keepalived/libipvs-2.6/ip_vs.h 将#include <sys/types.h>放到#include <linux/types.h>的上面 # ./configure --prefix=/ --with-kernel-dir=/usr/src/kernels/2.6.18-194.el5-i686/ # make # make install # chkconfig --add keepalived # chkconfig keepalived on # cd /etc/keepalived/ # cp keepalived.conf keepalived.conf.bak # vim keepalived.conf #文件里面所有内容
! Configuration File for keepalived global_defs { router_id LVS_MASTER #从调度器上改为router_id LVS_SLAVE} vrrp_instance VI_1 { state MASTER #从调度器上改为state SLAVE interface eth0 virtual_router_id 51 priority 100 #从调度器上改为priority 80 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.10.100 }} virtual_server 192.168.10.100 80 { delay_loop 6 lb_algo rr lb_kind DR nat_make 255.255.255.0 persistence_timeout 50 protocol TCP real_server 192.168.10.3 80 { weight 1 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 3 } } real_server 192.168.10.4 80 { weight 1 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 3 } }}
配置负载分配策略# /etc/init.d/ipvsadm stop# ipvsadm –A –t 192.168.10.100:80 –s rr# ipvsadm –a –t 192.168.10.100:80 –r 192.168.10.3 –g –w 1# ipvsadm –a –t 192.168.10.100:80 –r 192.168.10.4 –g –w 1# /etc/init.d/ipvsadm save# chkconfig ipvsadm onipvsadm 从调度器上同上所有配置，先安装相关软件包，在安装keepalived，最后配置/etc/keepalived.conf,只需将注释的地方修改即可 四、配置iSCSI存储服务器（1）安装服务端所需软件包 # yum -y install scsi-target-utils （2）准备存储设备创建一个LVM逻辑卷# pvcreate /dev/sdb5# vgcreate vol0 /dev/sdb5# lvcreate -L 500M -n lv-host4 vol0# lvscanACTIVE '/dev/vol0/lv-host4' inherit创建一个软RAID# mdadm -Cv /dev/md0 -a yes -n3 -l5 /dev/sdb# mdadm -Q /dev/md0# mdadm -D -s >> /etc/mdadm.conf# cat /etc/mdadm.confARRAY /dev/md0 metadata=1.2 spares=1 name=host1.zc.com:0UUID=a0093bb0:d52fd73f:fb9469f3:0fbee1f2（3）启动iscsi服务，设置开启启动# /etc/init.d/tgtd start# netstat -anpt | grep tgttcp 0 0 0.0.0.0:3260 0.0.0.0:*LISTEN 1850/tgtdtcp 0 0 :::3260 :::*LISTEN 1850/tgtd# chkconfig tgtd on（4）创建ISCSI对象（Target）# tgtadm -L iscsi -o new -m target -t 1 -T iqn.2012-06.com.dong.www:lvm# tgtadm -L iscsi -o new -m target -t 2 -T iqn.2012-06.com.dong.www:raid# tgtadm -L iscsi -o show -m target（5）为客户机分配存储空间（LUN）# tgtadm --lld iscsi -o new -m logicalunit -t 1 -l 1 -b/dev/vol0/lv-host4# tgtadm --lld iscsi -o new -m logicalunit -t 2 -l 1 -b /dev/md0# tgtadm --lld iscsi -o bind -m target -t 1 -I 192.168.10.3# tgtadm --lld iscsi -o bind -m target -t 2 -I 192.168.10.4# tgtadm --lld iscsi -o bind -m target -t 1 -I 192.168.10.3# tgtadm --lld iscsi -o bind -m target -t 2 -I 192.168.10.4# tgtadm --lld iscsi -o show -m target （6）保存target和LUN配置# vim /etc/tgt/targets.conf #配置永久生效<target iqn.2012-06.com.dong.www:lvm> #最后添加以下内容backing-store /dev/vol0/lv-host21initiator-address 192.168.10.5</target><target iqn.2012-06.com.dong.www:raid>backing-store /dev/md0initiator-address 192.168.10.5</target> 五、节点配置（1）分别在2台RealServer(节点)上面编写脚本# vim /etc/init.d/realserver.sh
#!/bin/bashVIP=192.168.10.100startrs(){echo "start LVS of REALServer"/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up/sbin/route add -host $VIP dev lo:0echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/lo/arp_announceecho "1" >/proc/sys/net/ipv4/conf/all/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/all/arp_announce}stoprs(){/sbin/ifconfig lo:0 downecho "close LVS Directorserver"echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignoreecho "0" >/proc/sys/net/ipv4/conf/lo/arp_announceecho "0" >/proc/sys/net/ipv4/conf/all/arp_ignoreecho "0" >/proc/sys/net/ipv4/conf/all/arp_announce}# ============ Main ===========case $1 in"start")startrs;;"stop")stoprs;;"*")echo "Usage $0 {start|stop}"exit 1esac
# chmod +x /etc/init.d/realserver.sh #/etc/init.d/realserver.sh start （2）挂载存储客户端安装iSCSI客户端相关软件# yum -y install iscsi-initiator-utils lsscsi# /etc/init.d/iscsi restart# chkconfig iscsi on发现ISCSI设备# iscsiadm -m discovery -t sendtargets -p 192.168.10.5192.168.10.5:3260,1 iqn.2012-06.com.dong.www:lvm192.168.10.5:3260,1 iqn.2012-06.com.dong.www:raid查看现有分区# fdisk -cul连接ISCSI设备# iscsiadm -m node -T iqn.2012-06.com.dong.www:lvm --login# iscsiadm -m node -T iqn.2012-06.com.dong.www:raid --login查看分区# lsscsi cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0 disk VMware, VMware Virtual S 1.0 /dev/sda storage IET Controller 0001 - disk IET VIRTUAL-DISK 0001 /dev/sdb storage IET Controller 0001 - disk IET VIRTUAL-DISK 0001 /dev/sdc# fdisk -cul……Disk /dev/sdb: 524 MB, 524288000 bytes17 heads, 59 sectors/track, 1020 cylinders, total 1024000 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sdb doesn't contain a valid partition tableDisk /dev/sdc: 2146 MB, 2146435072 bytes67 heads, 62 sectors/track, 1009 cylinders, total 4192256 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sdc doesn't contain a valid partition table将/dev/sdc划分并使用# partprobe /dev/sdc# fdisk -cu /dev/sdc# fdisk -cul /dev/sdcDisk /dev/sdc: 2146 MB, 2146435072 bytes15 heads, 2 sectors/track, 139741 cylinders, total 4192256 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x8338546cDevice Boot Start End Blocks Id System/dev/sdc1 2048 4192255 2095104 83 Linux# mkfs.ext4 /dev/sdc1# mkdir /opt/data# mount /dev/sdc1 /opt/data/ #临时挂载，重启失效# cat /etc/mtab/dev/sda2 / ext4 rw 0 0proc /proc proc rw 0 0sysfs /sys sysfs rw 0 0devpts /dev/pts devpts rw,gid=5,mode=620 0 0tmpfs /dev/shm tmpfs rw 0 0/dev/sda1 /boot ext4 rw 0 0/dev/sda3 /data ext4 rw 0 0none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0/dev/sdc1 /opt/data ext4 rw 0 0# blkid /dev/sdb1 >> /etc/fstab #挂载永久生效# vim /etc/fstab...... //省略部分信息UUID="78e6d91e-10bf-4180-b308-4259145853d6" /var/www/html/ ext4defaults,_netdev 0 0 (3)在2台RealServer(节点)中分别配置HTTP并启动realserver.sh# yum -y install httpd # cd /var/www/html/ # echo "192.168.10.3" > index.html # cat index.html 192.168.10.3 # cat index.html #两台节点的配置除了网页不一样（为了更明显的看到测试效果），其他都一样192.168.10.4 # /etc/init.d/realserver.sh start # echo "/etc/init.d/realserver.sh start" >> /etc/rc.local # /etc/init.d/httpd start # chkconfig httpd on (4)给2台节点添加一条路由记录，将访问VIP的数据限制在本地以避免通信紊乱# route add -host 192.168.10.100 dev lo:0 # echo "/sbin/route add -host 192.168.10.100 dev lo:0" >> /etc/rc.local 另一台的配置过程一样，过程略 六、测试在两台调度器上分别启动Keepalived# /etc/init.d/keepalived restart # ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP192.168.10.100:80 rr persistent 50-> 192.168.10.3:80 Route 1 0 0 -> 192.168.10.4:80 Route 1 0 0 使用客户机访问<http://192.168.10.100>,使主调度器挂掉看是否能够访问使用客户机访问<http://192.168.10.100>,使一个节点挂掉看是否能够访问 楼主很专业，写得很好！ 只要不下流，我们就是主流! 勿以坑小而不灌，勿以坑大而灌之。 我本非随便的人，但如果你想随便，那我就随你的便好啦！ 怀揣两块，胸怀500万！ 男人靠的住，母猪能上树! 过来学习一下
页:
[1]