Installing and Configuring Openfiler with DRBD and Heartbeat(HA)--1

天空下的承诺

　　
　　Introduction
　　Openfiler isa high performance operating system tailored for use as a SAN/NAS appliance. This configuration will enable two Openfiler appliances to work in an Active/Passive high availability scenario.
　　
　　

Requirements

Hardware

• 2 x boxes that meet the minimum spec of Openfiler's hardware specifications.
  
• 2 x ethernet interfaces in each box
  
• Openfiler 2.3 installation media.
  
• Both boxes should have the same size drives in each to avoid any replication inconsistencies.
  

　　

Software
　　Install Openfiler 2.3 on both boxes utilizing a disk setup such as the following:

• 3 GB root (“/”) partition
  
• 2 GB “swap” partition
  
• 512 MB “/meta” partition (used for DRBD0)
  
• Data partition configured as an unmounted LVM (used for DRBD1)
  

　　

Configuration

Network
　　Each Openfiler appliance will have two NICs: one for communicating with the LAN, the other for communicating with the

other SAN (via direct cable). The first will be used for administration, to communicate directly with each node.
　　A third “virtual” interface is used by the heartbeat service and is what will be used by computers on the LAN.
　　Below is what is used:
　　filer01

• LAN Interface (eth0) 192.168.1.18
  
• Replication Interface (eth1) 10.188.188.1
  

　　filer02

• LAN Interface (eth0) 192.168.1.19
  
• Replication Interface (eth1) 10.188.188.2
  

　　HA NAS/SAN Address (eth0) 192.168.1.17

• This is configured in the cluster.xml file (do not attempt to configure anywhere else)
  

　　

Hostname Setup
　　For both nodes to be able to recognize each other by name, configure the hosts file on each computer.

Modify our /etc/hosts (on filer01):

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 filer01 localhost.localdomain localhost
10.188.188.2 filer02
　　Modify our /etc/hosts (on filer02):

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 filer02 localhost.localdomain localhost
10.188.188.1 filer01
　　

SSH Shared keys
　　To allow the two Openfiler appliances to talk to each other without having to use a password, use SSH shared keys.
　　On filer01:
root@filer01 ~# ssh-keygen -t dsa
　　Hit enter at the prompts (don't set a password on the key).
　　On filer02:
root@filer02 ~# ssh-keygen -t dsa
　　Hit enter at the prompts (don't set a password on the key).
　　The above command will generate a file called "id_dsa.pub" in
~/.ssh/, which is the public key that will need to be copied to

the other node:
root@filer01 ~# scp .ssh/id_dsa.pub root@filer02:~/.ssh/authorized_keys2

root@filer02 ~# scp .ssh/id_dsa.pub root@filer01:~/.ssh/authorized_keys2
　　

Configure DRBD
　　DRBD is what will keep the data between the two nodes consistent.
　　On filer01:
root@filer01 ~# mv /etc/drbd.conf /etc/drbd.conf.org
　　Then modify drbd.conf (version 8) according to following:

global {
# minor-count 64;
# dialog-refresh 5; # 5 seconds
# disable-ip-verification;
usage-count ask;
}
common {
syncer { rate 100M; }
}
resource cluster_metadata {
protocol C;
handlers {
pri-on-incon-degr "echo O > /proc/sysrq-trigger ; halt -f";
pri-lost-after-sb "echo O > /proc/sysrq-trigger ; halt -f";
local-io-error "echo O > /proc/sysrq-trigger ; halt -f";
# outdate-peer "/usr/sbin/drbd-peer-outdater";
}
startup {
# wfc-timeout 0;
degr-wfc-timeout 120; # 2 minutes.
}
disk {
on-io-error detach;
}
net {
after-sb-0pri disconnect;
after-sb-1pri disconnect;
after-sb-2pri disconnect;
rr-conflict disconnect;
}
syncer {
# rate 10M;
# after "r2";
al-extents 257;
}
on filer01 {
device /dev/drbd0;
disk /dev/sda3;
address 10.188.188.1:7788;
meta-disk internal;
}
on filer02 {
device /dev/drbd0;
disk /dev/sda3;
address 10.188.188.2:7788;
meta-disk internal;
}
}
resource vg0drbd {
protocol C;
startup {
wfc-timeout 0; ## Infinite!
degr-wfc-timeout 120; ## 2 minutes.
}
disk {
on-io-error detach;
}
net {
# timeout 60;
# connect-int 10;
# ping-int 10;
# max-buffers 2048;
# max-epoch-size 2048;
}
syncer {
after "cluster_metadata";
}
on filer01 {
device /dev/drbd1;
disk /dev/sda5;
address 10.188.188.1:7789;
meta-disk internal;
}
on filer02 {
device /dev/drbd1;
disk /dev/sda5;
address 10.188.188.2:7789;
meta-disk internal;
}
}
　　Both hosts need the same drbd.conf, so the drbd.conf file from filer01 will be copied to filer02:
root@filer01 ~# scp /etc/drbd.conf root@filer02:/etc/drbd.conf
　　Initialise metadata on /dev/drbd0 (cluster_metadata) and/dev/drbd1 (vg0drbd) on both nodes:
root@filer01 ~# drbdadm create-md cluster_metadata

root@filer01 ~# drbdadm create-md vg0drbd

root@filer02 ~# drbdadm create-md cluster_metadata

root@filer02 ~# drbdadm create-md vg0drbd
　　Note: if the commands above generate errors about needing to zero out the file system, use the following command:
root@filer01 ~# dd if=/dev/zero of=/dev/sda3
　　Be careful with this command and make sure its on the correct drive.
　　Before starting the DRBD service, make sure that the partition used for drbd0 (in the cluster_metadata resource in thedrbd.conf file) is not already mounted (which it will be by default if it was created during the installation).
root@filer01 ~# umount /dev/sda3
　　Now, start DRBD on both hosts:
root@filer01 ~# service drbd start
root@filer02 ~# service drbd start
　　If all goes well, they should connect and running "service drbd status" should present output similar to the following:
root@filer1 /# service drbd status
drbd driver loaded OK; device status:

version: 8.0.12 (api:86/proto:86)

GIT-hash: 5c9f89594553e32adb87d9638dce591782f947e3 build by phil@mescal, 2008-04-24 13:29:44

m:res cs st ds p mounted fstype

0:cluster_metadata Connected Secondary/Secondary Inconsistent/Inconsistent C

1:vg0drbd Connected Secondary/Secondary Inconsistent/Inconsistent C
　　Once both drbd resources are connected and both nodes are in Secondary state (as above), set a Primary node:
root@filer01 ~# drbdsetup /dev/drbd0 primary -o

root@filer01 ~# drbdsetup /dev/drbd1 primary -o
　　This should give you a status result of something like the following:
root@filer1 /# service drbd status
drbd driver loaded OK; device status:

version: 8.0.12 (api:86/proto:86)

GIT-hash: 5c9f89594553e32adb87d9638dce591782f947e3 build by phil@mescal, 2008-04-24 13:29:44

m:res cs st ds p mounted fstype

... sync'ed: 17.9% (247232/297152)K

0:cluster_metadata SyncSource? Primary/Secondary UpToDate/Inconsistent C

1:vg0drbd PausedSyncS? Primary/Secondary UpToDate/Inconsistent C
　　Note: if the vg0drbd LVM is large, it will take a long time to sync (perhaps overnight).
　　Enable DRBD to startup at boot:
root@filer01 ~# chkconfig --level 2345 drbd on

root@filer02 ~# chkconfig --level 2345 drbd on
　　Now create the cluster_metadata filesystem. Use this 512 MB partition to keep all of the Openfiler configuration data and the data for the services that should be available in HA (eg. NFS, iSCSI, SMB).
root@filer01 ~# mkfs.ext3 /dev/drbd0
　　Don't add this partition to an /etc/fstab, as this is managed by Heartbeat (and will be configured shortly).
　　


　　(http://www.howtoforge.com/installing-and-configuring-openfiler-with-drbd-and-heartbeat)