centos6.2+heartbeat+mysql5.5+drbd84高可用安装

yuandan

CentOS6.2 64位
Mysql版本mysql-5.5.35.tar.gz,编译安装
drbd yum安装 drbd kmod-drbd84
heartbeat yum安装 heartbeat-3.0.4-2.el6.x86_64
主：192.168.2.230
从：192.168.2.231
VIP：192.168.2.232
配置安装编译环境
yum groupinstall "Development Tools" -y
yum -y install vim wget gcc-c++ ncurses ncurses-devel cmake make bison openssl openssl-devel gcc* libxml2 libxml2-devel curl-devel libjpeg* libpng* freetype* libxslt
reboot
rpm -ivh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm
yum install drbd kmod-drbd84
chkconfig --add drbd
chkconfig drbd on
检查安装是否成功：
drbdadm
看是否以执行成功
cat /proc/drbd
是否可以看到版本信息
重启后再次检查cat /proc/drbd
查看drbd模块信息
modinfo /lib/modules/2.6.32-358.el6.x86_64/updates/drbd.ko
在两台服务器中分出一个空分区或者添加一个空的硬盘，未格式化文件系统的
fdisk /dev/sda
配置服务器的hostname，在两台服务器中编辑hosts文件
vi /etc/hosts 添加：
192.168.2.230  www.a.com
192.168.2.231  www.b.com
在两台服务器上配置两个drbd的源文件
vi /etc/drbd.d/mysqldata.res
resource mysqldata {
protocol C;
on www.a.com   {
device    /dev/drbd0;
disk      /dev/sda5;
address   192.168.2.230:7789;
meta-disk internal;
}
on www.b.com   {
device    /dev/drbd0;
disk      /dev/sda5;
address   192.168.2.231:7789;
meta-disk internal;
}
}
往两台服务器的/dev/sda5设备写入一些数据,否则drbd在create-md时会报错
dd if=/dev/zero of=/dev/sda5 bs=1M count=128
drbdadm create-md all
service drbd restart
cat /proc/drbd
在主节点上执行
drbdadm -- --overwrite-data-of-peer primary all
=========================
上面执行的语句的所有横线不能丢。如果第一次设置主节点时使用“/sbin/drbdadm primary mysqldata”命令，会提示如下错误：
0: State change failed: (-2) Need access to UpToDate data
Command '/sbin/drbdsetup 0 primary' terminated with exit code 17
只要第一次用上面命令成功后，以后就可以用“/sbin/drbdadm primary mysqldata”命令了。
========================
用下面命令查看同步的进度
cat /proc/drbd
同步成功后，主从节点都会出现UpToDate/UpToDate的关键字
version: 8.4.4 (api:1/proto:86-101)
GIT-hash: 599f286440bd633d15d5ff985204aff4bccffadd build by phil@Build64R6, 2013-10-14 15:33:06
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:8393532 nr:0 dw:0 dr:8394196 al:0 bm:513 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
由于mount操作只能在主节点进行，所以只有设置了主节点后才能格式化磁盘分区，然后挂载，
备机的DRBD设备无法被挂载，因为它是用来接收主机数据的，由DRBD负责操作。
在主节点格式化/dev/drbd0
mkfs.ext4 /dev/drbd0
建立挂载点目录,由于主从切换时都要有挂载点，所以都要建立该目录
mkdir /mysqldata
挂载
mount /dev/drbd0 /mysqldata
测试drbd同步
先在主节点上写入数据，随便写入什么：
touch /mysqldata/111
ll /mysqldata/
然后，我们先卸载目录/mysqldata，再把主节点的primary降级成secondary，然后把从节点的secondary提升成primary：
umount /mysqldata
drbdadm secondary mysqldata
把primary降级前，一定要先umount设备才行。
然后提升secondary
在从节点上执行：
drbdadm primary mysqldata
mount /dev/drbd0 /mysqldata
ll /mysqldata
可以看到，数据已经完全同步过去了。
还原
在从节点上
umount /mysqldata
drbdadm secondary mysqldata
在主节点上
drbdadm primary mysqldata
mount /dev/drbd0 /mysqldata
安装mysql
建立mysql所需账号、目录及权限
useradd -d /usr/local/mysql/ mysql   #主从上创建一个Mysql用户，指定家目录到/use/local目录下。
chown -R mysql:mysql /mysqldata/     #将主上面建立的mysqldata文件夹作为mysql的data存放位置，并赋予mysql权限
chmod 750 /mysqldata
3、解压并安装mysql
tar xf mysql-5.5.35.tar.gz  
cd mysql-5.5.35
cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql \
-DDEFAULT_CHARSET=utf8 \
-DDEFAULT_COLLATION=utf8_general_ci \
-DEXTRA_CHARSETS=all \
-DWITH_MYISAM_STORAGE_ENGINE=1 \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DWITH_ARCHIVE_STORAGE_ENGINE=1 \
-DWITH_BLACKHOLE_STORAGE_ENGINE=1 \
-DWITH_MEMORY_STORAGE_ENGINE=1 \
-DWITH_FEDERATED_STORAGE_ENGINE=1 \
-DWITH_READLINE=1 \
-DENABLED_LOCAL_INFILE=1 \
-DMYSQL_DATADIR=/mysqldata \
-DMYSQL_USER=mysql \
-DMYSQL_TCP_PORT=3306 \
-DSYSCONFDIR=/etc \
-DWITH_SSL=yes
make & make install
编译注解：
-DCMAKE_INSTALL_PREFIX=/usr/local/mysql \         #安装目录
-DDEFAULT_CHARSET=utf8 \                           #默认字符
-DDEFAULT_COLLATION=utf8_general_ci \              #校验字符
-DEXTRA_CHARSETS=all \                            #安装所有扩展字符集
-DWITH_MYISAM_STORAGE_ENGINE=1 \                  #安装myisam存储引擎
-DWITH_INNOBASE_STORAGE_ENGINE=1 \                #安装innodb存储引擎
-DWITH_ARCHIVE_STORAGE_ENGINE=1 \                 #安装archive存储引擎
-DWITH_BLACKHOLE_STORAGE_ENGINE=1 \               #安装blackhole存储引擎
-DWITH_MEMORY_STORAGE_ENGINE=1 \                  #安装memory存储引擎
-DWITH_FEDERATED_STORAGE_ENGINE=1 \               #安装frderated存储引擎
-DWITH_READLINE=1 \                               #快捷键功能
-DENABLED_LOCAL_INFILE=1 \                        #允许从本地导入数据
-DMYSQL_DATADIR=/mysqldata \                      #数据库存放目录
-DMYSQL_USER=mysql \                              #数据库属主
-DMYSQL_TCP_PORT=3306 \                           #数据库端口
-DSYSCONFDIR=/etc \                               #MySQL配辑文件
-DWITH_SSL=yes                                    #数据库SSL
vim /etc/my.cnf
[mysqld]
datadir=/mysqldata
socket=/mysqldata/mysql.sock
port=3306
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
default_storage_engine         = InnoDB
skip-external-locking
# CACHES AND LIMITS #
key_buffer_size                = 256M
max_allowed_packet             = 32M
sort_buffer_size               = 16M
read_buffer_size               = 4M
read_rnd_buffer_size           = 16M
thread_stack                   = 8M
tmp_table_size                 = 32M
max_heap_table_size            = 32M
query_cache_type               = 1
query_cache_size               = 128M
query_cache_limit              = 2M
max_connections                = 2048
thread_cache_size              = 512
open_files_limit               = 65535
table_definition_cache         = 1024
table_open_cache               = 2048
character_set_server = utf8
default-storage-engine = innodb
init_connect="SET NAMES 'utf8'"
skip-name-resolve
wait_timeout=3600
interactive-timeout=3600
# INNODB #
innodb_log_files_in_group      = 2
innodb_log_file_size           = 64M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 0
innodb_support_xa=0
innodb_file_per_table          = 1
innodb_buffer_pool_size        = 128M
innodb_additional_mem_pool_size =64M
innodb_data_file_path =ibdata1:1024M:autoextend
innodb_autoextend_increment =50
# MyISAM #
key_buffer_size                = 32M
myisam_recover                 = FORCE,BACKUP
# LOGGING #
log-error=/mysqldata/error.log
general_log=1
general_log_file=/mysqldata/mysql.log
slow_query_log=1
slow_query_log_file=/mysqldata/slowquery.log
log-output=FILE
# BINARY LOGGING #
log-bin                        =/mysqldata/bin.log
expire_logs_days               = 30
sync_binlog =0
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
[client]
socket=/mysqldata/mysql.sock
将mysql的库文件路径加入系统的库文件搜索路径中
利用ldconfig导入系统库
echo "/usr/local/mysql/lib" >> /etc/ld.so.conf.d/mysql.conf
ldconfig
输出mysql的头文件到系统头文件
ln -s /usr/local/mysql/include/mysql /usr/include/mysql
进入安装路径,初始化配置脚本,只在主节点上执行，从节点不执行
cd /usr/local/mysql
scripts/mysql_install_db --user=mysql --datadir=/mysqldata
复制mysql启动脚本到系统服务目录
cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysqld
系统启动项相关配置，主节点配置，从节点不配置
chkconfig --add mysqld  #添加开机启动服务
chkconfig mysqld on  #设置mysql启动
启动mysql
service mysqld start
vim /etc/profile
配置mysql环境变量
export PATH=$PATH:/usr/local/mysql/bin
加载环境变量
. /etc/profile
设置初始账户,并登陆后台:
/usr/local/mysql/bin/mysqladmin -u root password 111111
进入mysql
mysql -u root -p111111
安装heartbeat
rpm -ivh http://mirrors.sohu.com/fedora-epel/6/x86_64/epel-release-6-8.noarch.rpm
yum install heartbeat
cd /etc/ha.d/
该目录下需要三个配置文件
vim authkeys
#
#   Authentication file.  Must be mode 600
#
#
#   Must have exactly one auth directive at the front.
#   auth    send authentication using this method-id
#
#   Then, list the method and key that go with that method-id
#
#   Available methods: crc sha1, md5.  Crc doesn't need/want a key.
#
#   You normally only have one authentication method-id listed in this file
#
#   Put more than one to make a smooth transition when changing auth
#   methods and/or keys.
#
#
#   sha1 is believed to be the "best", md5 next best.
#
#   crc adds no security, except from packet corruption.
#       Use only on physically secure networks.
#
auth 1
1 crc
#2 sha1 HI!
#3 md5 Hello!
vim ha.cf
debugfile /var/log/ha-debug
logfacility local0
keepalive 2
deadtime 30
warntime 10
initdead 120
udpport 694       //upd监听端口，iptables需要对各自开放
ucast eth0 192.168.2.230  //主从各自指向相对的ip,
auto_failback on
node www.a.com
node www.b.com
respawn root /usr/lib64/heartbeat/ipfail
apiauth ipfail gid=root uid=root
vim haresources 该文件主从需要保持一致，heartbeat根据该文件切换主从、启动和挂载drbd以及mysqld程序，并使VIP地址生效。
www.a.com  IPaddr::192.168.2.232/22/eth0 drbddisk::mysqldata Filesystem::/dev/drbd0::/mysqldata::ext4 mysqld
将主从的该文件权限进行修改
chmod 600 authkeys
默认heartbeat安装后没有切换drbd的脚本，需要在主从的/etc/ha.d/resource.d下建立drbddisk脚本，并赋予755执行权限
vim /etc/ha.d/resource.d/drbddisk
#!/bin/bash
#
# This script is inteded to be used as resource script by heartbeat
#
# Copright 2003-2008 LINBIT Information Technologies
# Philipp Reisner, Lars Ellenberg
#
###
DEFAULTFILE="/etc/default/drbd"
DRBDADM="/sbin/drbdadm"
if [ -f $DEFAULTFILE ]; then
. $DEFAULTFILE
fi
if [ "$#" -eq 2 ]; then
RES="$1"
CMD="$2"
else
RES="all"
CMD="$1"
fi
## EXIT CODES
# since this is a "legacy heartbeat R1 resource agent" script,
# exit codes actually do not matter that much as long as we conform to
#  http://wiki.linux-ha.org/HeartbeatResourceAgent
# but it does not hurt to conform to lsb init-script exit codes,
# where we can.
#  http://refspecs.linux-foundation.org/LSB_3.1.0/
#   LSB-Core-generic/LSB-Core-generic/iniscrptact.html
####
drbd_set_role_from_proc_drbd()
{
local out
if ! test -e /proc/drbd; then
ROLE="Unconfigured"
return
fi
dev=$( $DRBDADM sh-dev $RES )
minor=${dev#/dev/drbd}
if [[ $minor = *[!0-9]* ]] ; then
# sh-minor is only supported since drbd 8.3.1
minor=$( $DRBDADM sh-minor $RES )
fi
if [[ -z $minor ]] || [[ $minor = *[!0-9]* ]] ; then
ROLE=Unknown
return
fi
if out=$(sed -ne "/^ *$minor: cs:/ { s/:/ /g; p; q; }" /proc/drbd); then
set -- $out
ROLE=${5%/**}
: ${ROLE:=Unconfigured} # if it does not show up
else
ROLE=Unknown
fi
}
case "$CMD" in
start)
# try several times, in case heartbeat deadtime
# was smaller than drbd ping time
try=6
while true; do
$DRBDADM primary $RES && break
let "--try" || exit 1 # LSB generic error
sleep 1
done
;;
stop)
# heartbeat (haresources mode) will retry failed stop
# for a number of times in addition to this internal retry.
try=3
while true; do
$DRBDADM secondary $RES && break
# We used to lie here, and pretend success for anything != 11,
# to avoid the reboot on failed stop recovery for "simple
# config errors" and such. But that is incorrect.
# Don't lie to your cluster manager.
# And don't do config errors...
let --try || exit 1 # LSB generic error
sleep 1
done
;;
status)
if [ "$RES" = "all" ]; then
echo "A resource name is required for status inquiries."
exit 10
fi
ST=$( $DRBDADM role $RES )
ROLE=${ST%/**}
case $ROLE in
Primary|Secondary|Unconfigured)
# expected
;;
*)
# unexpected. whatever...
# If we are unsure about the state of a resource, we need to
# report it as possibly running, so heartbeat can, after failed
# stop, do a recovery by reboot.
# drbdsetup may fail for obscure reasons, e.g. if /var/lock/ is
# suddenly readonly.  So we retry by parsing /proc/drbd.
drbd_set_role_from_proc_drbd
esac
case $ROLE in
Primary)
echo "running (Primary)"
exit 0 # LSB status "service is OK"
;;
Secondary|Unconfigured)
echo "stopped ($ROLE)"
exit 3 # LSB status "service is not running"
;;
*)
# NOTE the "running" in below message.
# this is a "heartbeat" resource script,
# the exit code is _ignored_.
echo "cannot determine status, may be running ($ROLE)"
exit 4 #  LSB status "service status is unknown"
;;
esac
;;
*)
echo "Usage: drbddisk [resource] {start|stop|status}"
exit 1
;;
esac
exit 0
chmod 755 /etc/ha.d/resource.d/drbddisk
启动heartbeat。先启动主，再启动从。heartbeat启动时会自动启动drbd和mysql,并且会根据/etc/ha.d/haresources中的配置加载和挂载drbd的源
service heartbeat start
在主上查看VIP
ip addr
查看主从的heartbeat日志
tail -f /var/log/ha-debug
测试
首先现在主上查看/mysqldata是否挂载成功
ll /mysqldata
查看drbd状态
cat /proc/drbd
查看mysql启动状态和进程
service mysqld status
ps -ef|grep mysqld
本地登陆mysql查看数据库和表是否正常
利用mysql客户端工具连接VIP地址，看是否能通过VIP连接到mysql并且查看数据库和表状态
以上都成功后，测试切换
停止主上的heartbeat进程
在主上查看/mysqldata目录内容，以及mysql进程是否关闭
在从上查看VIP地址是否切换成功
ip addr
在从上查看/mysqldata目录内容，以及mysql进程是否启动
本地登陆mysql查看数据库和表是否正常
利用mysql客户端工具连接VIP地址，看是否能通过VIP连接到mysql并且查看数据库和表状态