DRBD+nfs+heartbeat
Heartbeat+nfs+DRBD实验一、DRBD试验简介
本实验部署DRBD + HEARDBEAT + NFS 环境，建立一个高可用(HA)的文件服务器集群。在方案中，通
过DRBD保证了服务器数据的完整性和一致性。DRBD类似于一个网络RAID-1功能。当你将数据写入本
地文件系统时，数据还将会被发送到网络中另一台主机上，以相同的形式记录在一个另文件系统中。主节
点与备节点的数据可以保证实时相互同步。当本地主服务器出现故障时，备份服务器上还会保留有一份相
同的数据，可以继续使用。在高可用(HA)中使用DRBD功能，可以代替使用一个共享盘阵。因为数据同时
存在于本地主服务器和备份服务器上。切换时，远程主机只要使用它上面的那份备份数据，就可以继续提
供主服务器上相同的服务，并且client用户对主服务器的故障无感知。
实验环境
虚拟机操作系统：Centos5.4 32bit
两台服务器：node1.a.com ip 192.168.101.250
node2.1.com ip 192.168.101.249
VIP:192.168.101.200
两台服务器将/dev/sda4互为镜像
两台服务器/etc/export配置相同
Node1配置：
给接口配置ip
http://blog.运维网.com/attachment/201205/175523919.png
修改主机名
# vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=node1.a.com
# init 6重启使主机名生效
# hwclock –s 同步时间，必须有
修改hosts文件：
# vim /etc/hosts 修改dns缓存
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.101.250 node1.a.com
192.168.101.249 node2.a.com
配置yum：
# vim /etc/yum.repos.d/server.repo
name=Red Hat Enterprise Linux server
baseurl=file:///mnt/cdrom/Server
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
同时要挂载光盘
Drbd配置
# yum -y localinstall *.rpm –nogpgcheck安装drbd包
# modprobe drbd 加载drbd模块
# lsmod |grep drbd 查看加载
drbd 2285280
创建新分区：
# fdisk /dev/sda
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Selected partition 4
First cylinder (1416-5221, default 1416): 1416
Last cylinder or +size or +sizeM or +sizeK (1416-5221, default 5221): +1G
Command (m for help): w
# partprobe /dev/sda
# cat /proc/partitions
major minor#blocksname
8 0 41943040 sda
8 1 104391 sda1
8 2 10241437 sda2
8 3 1020127 sda3
8 4 9775552 sda4
复制相关文件到node2上：
# scp *.rpm node2.a.com:/root
修改配置文件：
# cd /etc/
# cp /usr/share/doc/drbd83-8.3.8/drbd.conf ./
# cd /etc/drbd.d/
# ll
total 4
-rwxr-xr-x 1 root root 1418 Jun42010 global_common.conf
# cp global_common.conf global_common.conf.bak 备份文件
# vim global_common.conf
1 global {
2 usage-count no;
3 # minor-count dialog-refresh disable-ip-verification
4 }
5
6 common {
7 protocol C;
8
9 startup {
10 wfc-timeout 120;
11 degr-wfc-timeout 120;
12 }
13
14 disk {
15 on-io-error detach;
16 # fencing resource-only;
17 }
18 net {
19 cram-hmac-alg "sha1";
20 shared-secret "mydrbdlab";
21
22 }
23
24 syncer {
25 rate 100m;
26 }
27 }
4、检测配置文件
# drbdadm adjust r0
no resources defined
# vim web.res创建资源文件
resourceweb {
on node1.a.com {
device /dev/drbd0;
disk/dev/sda4;
address 192.168.101.250:7789;
meta-disk internal;
}
on node2.a.com {
device /dev/drbd0;
disk/dev/sda4;
address 192.168.101.249:7789;
meta-disk internal;
# scp /etc/drbd.conf node2.a.com:/etc/
# scp /etc/drbd.d/* node2.a.com:/etc/drbd.d/
# drbdadm create-md web创建ro资源
# service drbd start两边需要都启动，才可以启动成功
# drbdadm -- --overwrite-data-of-peer primary web将其设为主设备
# watch -n 1 'cat /proc/drbd'
# cat /proc/drbd 查看drbd状态
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by mockbuild@builder10.centos.org, 2010-06-04 08:04:16
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
# mkfs -t ext3 -L drbdweb /dev/drbd0 格式化分区
# mkdir /data
# mount /dev/drbd0 /data挂载分区
# service drbd status
drbd driver loaded OK; device status:
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by mockbuild@builder10.centos.org, 2010-06-04 08:04:16
m:rescs ro ds pmountedfstype
0:webConnectedPrimary/SecondaryUpToDate/UpToDateC/mnt/1 ext3
Nfs配置：
# vim /etc/exports两台服务器一致
/data *(rw,sync,insecure,no_root_squash,no_wdelay)
# service portmap start
Starting portmap:
# service nfs start
Starting NFS services:
Starting NFS quotas:
Starting NFS daemon:
Starting NFS mountd:
# chkconfig portmap on
# chkconfig nfs on
# vim /etc/init.d/nfs
116 stop)
117 # Stop daemons.
118 echo -n $"Shutting down NFS mountd: "
119 killproc rpc.mountd
120 echo
121 echo -n $"Shutting down NFS daemon: "
122 killproc nfsd -9两台服务器都修改nfs 启动脚本。将/etc/init.d/nfs 脚本中的stop 部分中的killproc ，nfsd -2 修改为 -9
# scp /etc/exports node2.a.com:/etc/
Heartbeat配置：
安装相关包：
# yum localinstall heartbeat-2.1.4-9.el5.i386.rpm heartbeat-pils-2.1.4-10.el5.i386.rpm libnet-1.1.4-3.el5.i386.rpm heartbeat-stonith-2.1.4-10.el5.i386.rpm perl-MailTools-1.77-1.el5.noarch.rpm
拷贝配置文档：
# cd /usr/share/doc/heartbeat-2.1.4/
# cp authkeys ha.cf haresources /etc/ha.d/
安装配置文档：
# cd /etc/ha.d/
# vim ha.cf
24 debugfile /var/log/ha-debug
29 logfile /var/log/ha-log
48 keepalive 2
49 #
56 deadtime 30
76 udpport 692
121 ucast eth0 192.168.101.249
157 auto_failback off
211 node node1.a.com
212 node node2.a.com
220 ping 192.168.101.1
指明主服务器
# echo "node1.a.com IPaddr::192.168.101.200/24/eth0 drbddisk::web Filesystem::/dev/drbd0::/data::ext3 killnfsd" >> /etc/ha.d/haresources
# vim authkeys
23 auth 1
24 1 crc
# echo "killall -9 nfsd; /etc/init.d/nfs restart; exit 0" >> ./ resource.d/killnfsd
# chmod 600 ./authkeys
# chmod 755 ./resource.d/killnfsd
# service heartbeat start
# service heartbeat restart
# chkconfig heartbeat on
查看node1是否出现vip：
http://blog.运维网.com/attachment/201205/175549564.png
Node1上面出现vip 192.168.101.200，node2上面没有：
http://blog.运维网.com/attachment/201205/175614564.png
Node2配置：
http://blog.运维网.com/attachment/201205/175643569.png
# vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=node2.a.com
# init 6
# hwclock -s
# vim /etc/hosts
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.101.250 node1.a.com
192.168.101.249 node2.a.com
# vim /etc/yum.repos.d/server.repo
name=Red Hat Enterprise Linux server
baseurl=file:///mnt/cdrom/Server
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
同时要挂载光盘
Drbd配置：
# yum -y localinstall *.rpm –nogpgcheck
# fdisk /dev/sda
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Selected partition 4
First cylinder (1416-5221, default 1416): 1416
Last cylinder or +size or +sizeM or +sizeK (1416-5221, default 5221): +10000m
Command (m for help): w
# partprobe /dev/sda
# cat /proc/partitions
major minor#blocksname
8 0 41943040 sda
8 1 104391 sda1
8 2 10241437 sda2
8 3 1020127 sda3
8 4 9775552 sda4
# drbdadm create-md web
# service drbd start
# cat /proc/drbd
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by mockbuild@builder10.centos.org, 2010-06-04 08:04:16
0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:9775216
# service drbd status
drbd driver loaded OK; device status:
version: 8.3.8 (api:88/proto:86-94)
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by mockbuild@builder10.centos.org, 2010-06-04 08:04:16
m:rescs ro ds pmountedfstype
0:webConnectedSecondary/PrimaryUpToDate/UpToDateC
Nfs配置：
# service portmap start
Starting portmap:
# service nfs start
Starting NFS services:
Starting NFS quotas:
Starting NFS daemon:
Starting NFS mountd:
# chkconfig portmap on
# chkconfig nfs on
配置启动脚本
# vim /etc/init.d/nfs
116 stop)
117 # Stop daemons.
118 echo -n $"Shutting down NFS mountd: "
119 killproc rpc.mountd
120 echo
121 echo -n $"Shutting down NFS daemon: "
122 killproc nfsd -9两台服务器都修改nfs 启动脚本。将/etc/init.d/nfs 脚本中的stop 部分中的killproc ，nfsd -2 修改为 -9
# service heartbeat start
Starting High-Availability services:
2012/05/08_00:52:46 INFO:Resource is stopped
Heartbeat配置：
安装heartbeat相关包：
# yum localinstall heartbeat-2.1.4-9.el5.i386.rpm heartbeat-pils-2.1.4-10.el5.i386.rpm heartbeat-stonith-2.1.4-10.el5.i386.rpm libnet-1.1.4-3.el5.i386.rpm perl-MailTools-1.77-1.el5.noarch.rpm –nogpgcheck
拷贝配置文档并配置配置文档
# cd /usr/share/doc/heartbeat-2.1.4/
# cp authkeys ha.cf haresources /etc/ha.d/
# cd /etc/ha.d/
# ls
README.configauthkeysha.cfharcharesourcesrc.dresource.dshellfuncs
# vim ha.cf
24 debugfile /var/log/ha-debug
29 logfile /var/log/ha-log
48 keepalive 2
49 #
56 deadtime 30
76 udpport 692
121 ucast eth0 192.168.101.249
157 auto_failback off
211 node node1.a.com
212 node node2.a.com
220 ping 192.168.101.1
指明主服务器：
# echo "node1.a.com IPaddr::192.168.101.200/24/eth0 drbddisk::web Filesystem::/dev/drbd0::/data::ext3 killnfsd" >> /etc/ha.d/haresources
# vim authkeys
23 auth 1
24 1 crc
# echo "killall -9 nfsd; /etc/init.d/nfs restart; exit 0" >> ./ resource.d/killnfsd
修改文件权限
# chmod 600 ./authkeys
# chmod 755 ./resource.d/killnfsd
# service heartbeat start
Starting High-Availability services:
2012/05/08_00:52:46 INFO:Resource is stopped
# service heartbeat restart
# chkconfig heartbeat on
测试机配置：
1、在测试机上将192.168.10.188:/data挂到本地/mnt/nfs
# mkdir /mnt/nfs
# mount 192.168.101.200:/data /mnt/nfs
2、在测试机上创建测试shell，二秒一个
# vim /mnt/test.sh
while true
do
echo ---\> trying touch x : `date`
touch x
echo \ trying touch x : Sun May 6 20:03:12 CST 2012
trying touch x : Sun May 6 20:03:14 CST 2012
trying touch x : Sun May 6 20:03:16 CST 2012
trying touch x : Sun May 6 20:03:18 CST 2012
trying touch x : Sun May 6 20:03:20 CST 2012
trying touch x : Sun May 6 20:03:22 CST 2012
trying touch x : Sun May 6 20:03:24 CST 2012
trying touch x : Sun May 6 20:03:26 CST 2012
trying touch x : Sun May 6 20:03:28 CST 2012
touch: cannot touch `x': Stale NFS file handle
trying touch x : Sun May 6 20:03:33 CST 2012
trying touch x : Sun May 6 20:03:35 CST 2012
页:
[1]