CentOS5.8 HA集群之基于crm配置 heartbeat + nfs + mysql
规划：让mysql的数据目录位于一个NFS Server上，并且这个数据目录是一个LVM，方便备份和扩容
注意：
要做高可用集群，事先的准备工作有三个：①时间同步②主机名与uname -n保持一致，且能够通过/etc/hosts文件解析③ssh互信通信，此处不再重复此操作，我的这篇博客中很详细：Linux HA集群之基于crm配置 heartbeat + nfs + httpd
1、NFS Server准备一个逻辑卷
此处以一个分区做演示
# fdisk /dev/sda
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610): +15G
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)
Command (m for help): p
Disk /dev/sda: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x7dd89809
Device Boot Start End Blocks IdSystem
/dev/sda1 1 1959 15735636 8eLinux LVM
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
# cat /proc/partitions
major minor#blocksname
8 0 20971520 sda
8 1 15735636 sda1
8 16 20971520 sdb
8 17 512000 sdb1
8 18 20458496 sdb2
253 0 18423808 dm-0
253 1 2031616 dm-1
创建逻辑卷
# pvremove /dev/sda
Physical Volume /dev/sda not found
# pvremove /dev/sda1
Labels on physical volume "/dev/sda1" successfully wiped
# pvcreate /dev/sda1
Physical volume "/dev/sda1" successfully created
# vgcreate myvg /dev/sda1
Volume group "myvg" successfully created
# lvcreate -L 10G -n mydata myvg
Logical volume "mydata" created
# lvs
LV VG Attr LSizePool Origin Data%Move Log Cpy%Sync Convert
mydatamyvg -wi-a----- 10.00g
lv_root vg_centos6 -wi-ao---- 17.57g
lv_swap vg_centos6 -wi-ao----1.94g
格式化此逻辑卷
# mke2fs -t ext4 /dev/mapper/myvg-mydata
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
655360 inodes, 2621440 blocks
131072 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 33 mounts or
180 days, whichever comes first.Use tune2fs -c or -i to override. 2、NFS Server上创建mysql用户
# groupadd -g 3306 mysql
# useradd -u 3306 -g mysql -s /sbin/nologin -Mmysql
# id mysql
uid=3306(mysql) gid=3306(mysql) groups=3306(mysql) 3、设置开机自动挂载此逻辑卷
# mkdir /mydata
# vim /etc/fstab
# tail -1 /etc/fstab
/dev/myvg/mydata/mydataext4 defaults0 0
# mount -a
# mount
/dev/mapper/vg_centos6-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sdb1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/dev/mapper/myvg-mydata on /mydata type ext4 (rw) 4、准备数据目录
# mkdir /mydata/data
# chown -R mysql.mysql /mydata/data
# ll -d /mydata/data
drwxr-xr-x 2 mysql mysql 4096 Jan3 10:03 /mydata/data 5、导出数据目录
# vim /etc/exports
# cat /etc/exports
/mydata172.16.0.0/16(rw)
# exportfs -arv
exporting 172.16.0.0/16:/mydata 6、手动挂载此数据目录，测试两个节点是否能进行读写操作
node1
先在node1上添加mysql用户
# hostname
node1.network.com
# groupadd -g 3306 mysql
# useradd -u 3306 -g mysql -s /sbin/nologin -M mysql
# mkdir /mydata
# mount -t nfs 172.16.1.102:/mydata /mydata
# ll /mydata/
total 20
drwxr-xr-x 2 mysql mysql4096 Jan3 10:03 data
drwx------ 2 rootroot16384 Jan3 09:56 lost+found
切换到mysql用户测试能否读写
# su - mysql
su: warning: cannot change directory to /home/mysql: No such file or directory
This account is currently not available.
# usermod -s /bin/bash mysql # 临时性修改
# su - mysql
su: warning: cannot change directory to /home/mysql: No such file or directory
-bash-3.2$ pwd
/root
-bash-3.2$ cd /mydata/data/
-bash-3.2$ touch a
-bash-3.2$ ls
a
-bash-3.2$ rm a
-bash-3.2$ exit
logout
# usermod -s /sbin/nologin mysql # 将默认shell还改为/sbin/nologin
# umount /mydata/ # 卸载完成，node1的测试结束 node2
同样的添加mysql用户
# hostname
node2.network.com
# groupadd -g 3306 mysql
# useradd -u 3306 -g mysql -M mysql
# mkdir /mydata
# mount -t nfs 172.16.1.102:/mydata /mydata/
# ll /mydata/
total 20
drwxr-xr-x 2 mysql mysql4096 Jan3 10:19 data
drwx------ 2 rootroot16384 Jan3 09:56 lost+found
# su - mysql
su: warning: cannot change directory to /home/mysql: No such file or directory
-bash-3.2$ pwd
/root
-bash-3.2$ cd /mydata/data/
-bash-3.2$ touch hello
-bash-3.2$ ls
hello
-bash-3.2$ rm hello
-bash-3.2$ exit
logout
# usermod -s /sbin/nologin mysql
# umount /mydata/ 7、安装并初始化mysql(选择一个节点即可，这里以node1为例)
解压并初始化mysql
# tar xf mysql-5.5.28-linux2.6-x86_64.tar.gz -C /usr/local/
# cd /usr/local/
# ln -sv mysql-5.5.28-linux2.6-x86_64 mysql
create symbolic link `mysql' to `mysql-5.5.28-linux2.6-x86_64'
# cd mysql
# chown -R root.mysql ./*
# ll
total 132
drwxr-xr-x2 root mysql4096 Jan7 19:31 bin
-rw-r--r--1 root mysql 17987 Aug 292012 COPYING
drwxr-xr-x4 root mysql4096 Jan7 19:31 data
drwxr-xr-x2 root mysql4096 Jan7 19:30 docs
drwxr-xr-x3 root mysql4096 Jan7 19:31 include
-rw-r--r--1 root mysql7604 Aug 292012 INSTALL-BINARY
drwxr-xr-x3 root mysql4096 Jan7 19:31 lib
drwxr-xr-x4 root mysql4096 Jan7 19:31 man
drwxr-xr-x 10 root mysql4096 Jan7 19:31 mysql-test
-rw-r--r--1 root mysql2552 Aug 292012 README
drwxr-xr-x2 root mysql4096 Jan7 19:31 scripts
drwxr-xr-x 27 root mysql4096 Jan7 19:31 share
drwxr-xr-x4 root mysql4096 Jan7 19:31 sql-bench
drwxr-xr-x2 root mysql4096 Jan7 19:31 support-files
# mount -t nfs 172.16.1.102:/mydata/ /mydata/# 挂载数据目录
初始化mysql出错
# scripts/mysql_install_db --user=mysql --datadir=/mydata/data
chown: changing ownership of `/mydata/data': Operation not permitted
Cannot change ownership of the database directories to the 'mysql'
user.Check that you have the necessary permissions and try again.
管理员权限是被压缩的了，所以此时我们需要在NFS共享目录上设置no_root_squash参数
# vim /etc/exports
# cat /etc/exports
/mydata172.16.0.0/16(rw,no_root_squash) # 其实这是很危险的操作，应只能运行指定的几个节点才能挂载
# exportfs -arv
exporting 172.16.0.0/16:/mydata
此时再在node1上初始化mysql
# scripts/mysql_install_db --user=mysql --datadir=/mydata/data
Installing MySQL system tables...
OK
Filling help tables...
OK
To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system
PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
./bin/mysqladmin -u root password 'new-password'
./bin/mysqladmin -u root -h node1.network.com password 'new-password'
Alternatively you can run:
./bin/mysql_secure_installation
which will also give you the option of removing the test
databases and anonymous user created by default.This is
strongly recommended for production servers.
See the manual for more instructions.
You can start the MySQL daemon with:
cd . ; ./bin/mysqld_safe &
You can test the MySQL daemon with mysql-test-run.pl
cd ./mysql-test ; perl mysql-test-run.pl
Please report any problems with the ./bin/mysqlbug script!
给mysql增加配置文件以及服务脚本
# ll /mydata/data/
total 12
drwx------ 2 mysql root4096 Jan3 10:52 mysql
drwx------ 2 mysql mysql 4096 Jan3 10:52 performance_schema
drwx------ 2 mysql root4096 Jan3 10:52 test
# ls
binCOPYINGdatadocsincludeINSTALL-BINARYlibmanmysql-testREADMEscriptssharesql-benchsupport-files
# cp support-files/my-large.cnf /etc/my.cnf
修改mysql的配置文件
# vim /etc/my.cnf
# grep -A 1 "datadir" /etc/my.cnf
datadir = /mydata/data
innodb_file_per_table = 1
拷贝服务启动脚本，并设置开机不自动启动
# cp support-files/mysql.server /etc/rc.d/init.d/mysqld
# chkconfig --add mysqld
# chkconfig mysqld off 8、手动连接mysql，测试是否能读写mysql
# service mysqld start
Starting MySQL......
# /usr/local/mysql/bin/mysql
Welcome to the MySQL monitor.Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.5.28-log MySQL Community Server (GPL)
Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
+--------------------+
4 rows in set (0.01 sec)
mysql> CREATE DATABASE testdb;
Query OK, 1 row affected (0.10 sec)
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
| testdb |
+--------------------+
5 rows in set (0.01 sec)
mysql> \q
Bye
# service mysqld stop
Shutting down MySQL..
# chkconfig mysqld off # 确保mysqld服务不能开机自启动
# chkconfig --list mysqld
mysqld 0:off1:off2:off3:off4:off5:off6:off
卸载之前挂载的mysql数据目录
# umount /mydata/
# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw) 9、配置另外节点上的mysql
我这里直接从node1节点上拷贝过来
# hostname
node2.network.com
# scpnode1:~/mysql-5.5.28-linux2.6-x86_64.tar.gz /tmp/
mysql-5.5.28-linux2.6-x86_64.tar.gz 100%178MB 9.4MB/s 00:19
# cd /tmp/
# ls
mysql-5.5.28-linux2.6-x86_64.tar.gz
# tar xf mysql-5.5.28-linux2.6-x86_64.tar.gz -C /usr/local/
# cd /usr/local/
# ln -sv mysql-5.5.28-linux2.6-x86_64 mysql
create symbolic link `mysql' to `mysql-5.5.28-linux2.6-x86_64'
# cd mysql
# chown -R root.mysql ./*
# ll
total 132
drwxr-xr-x2 root mysql4096 Jan7 20:10 bin
-rw-r--r--1 root mysql 17987 Aug 292012 COPYING
drwxr-xr-x4 root mysql4096 Jan7 20:10 data
drwxr-xr-x2 root mysql4096 Jan7 20:09 docs
drwxr-xr-x3 root mysql4096 Jan7 20:10 include
-rw-r--r--1 root mysql7604 Aug 292012 INSTALL-BINARY
drwxr-xr-x3 root mysql4096 Jan7 20:10 lib
drwxr-xr-x4 root mysql4096 Jan7 20:10 man
drwxr-xr-x 10 root mysql4096 Jan7 20:10 mysql-test
-rw-r--r--1 root mysql2552 Aug 292012 README
drwxr-xr-x2 root mysql4096 Jan7 20:10 scripts
drwxr-xr-x 27 root mysql4096 Jan7 20:10 share
drwxr-xr-x4 root mysql4096 Jan7 20:10 sql-bench
drwxr-xr-x2 root mysql4096 Jan7 20:10 support-files
mysql数据目录在node1节点上已经初始化过了，所以此时不需要再进行初始化，只需要拷贝配置文件和服务启动脚本
# scp node1:/etc/rc.d/init.d/mysqld /etc/rc.d/init.d/
mysqld 100% 10KB10.4KB/s 00:01
# scp node1:/etc/my.cnf /etc/
my.cnf 100% 4713 4.6KB/s 00:00
确保mysqld服务不能开机自启动
# chkconfig --add mysqld
# chkconfig mysqld off
# chkconfig --list mysqld
mysqld 0:off1:off2:off3:off4:off5:off6:off
再挂载数据目录，连上去，看是否能访问到之前node1节点上创建的一个测试库testdb
# mount -t nfs 172.16.1.102:/mydata/ /mydata/
# ll /mydata/data/
total 28704
-rw-rw---- 1 mysql mysql 18874368 Jan3 11:33 ibdata1
-rw-rw---- 1 mysql mysql5242880 Jan3 11:33 ib_logfile0
-rw-rw---- 1 mysql mysql5242880 Jan3 10:58 ib_logfile1
drwx------ 2 mysql root 4096 Jan3 10:52 mysql
-rw-rw---- 1 mysql mysql 298 Jan3 11:02 mysql-bin.000001
-rw-rw---- 1 mysql mysql 213 Jan3 11:33 mysql-bin.000002
-rw-rw---- 1 mysql mysql 38 Jan3 11:31 mysql-bin.index
-rw-rw---- 1 mysql root 3846 Jan3 11:33 node1.network.com.err
drwx------ 2 mysql mysql 4096 Jan3 10:52 performance_schema
drwx------ 2 mysql root 4096 Jan3 10:52 test
drwx------ 2 mysql mysql 4096 Jan3 11:32 testdb
启动mysql
# service mysqld start
Starting MySQL...
# /usr/local/mysql/bin/mysql
Welcome to the MySQL monitor.Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.5.28-log MySQL Community Server (GPL)
Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
| testdb | # 在node1节点上创建的testdb库是可以看到的
+--------------------+
5 rows in set (0.07 sec)
mysql> DROP DATABASE testdb;
Query OK, 0 rows affected (0.14 sec)
mysql> \q
Bye
关闭mysql服务，并卸载数据目录
# service mysqld stop
Shutting down MySQL..
# umount /mydata/
到此为止，基本配置已完成，再来配置heartbeat即可 10、在各节点上启动heartbeat服务
首先在两个节点上启动heartbeat服务
# service heartbeat start
Starting High-Availability services:
# ssh node2 'service heartbeat start'
Starting High-Availability services:
查看当前高可用集群各节点状态
# crm_mon
Refresh in 12s...
============
Last updated: Thu Jan7 20:31:25 2016
Current DC: node1.network.com (e719d448-df8b-4d58-ae0e-020f67bbbb49) # 当前DC为node1
2 Nodes configured.
0 Resources configured.
============
Node: node1.network.com (e719d448-df8b-4d58-ae0e-020f67bbbb49): online
Node: node2.network.com (734de3f1-c370-46d7-b89d-69145618e8a9): online
启动hb_gui
# hb_gui &
14906 首先进入图形配置界面，输入密码
http://s4.运维网.com/wyfs02/M00/79/54/wKioL1aOXEqyFvkJAAIkJFYAiNI362.jpg
成功进入图形配置界面
http://s5.运维网.com/wyfs02/M01/79/55/wKiom1aOXCTy1XjlAAJIbIMfGB4410.jpg
接下来配置mysql高可用的各个资源即可，VIP、Filesystem和mysqld服务
11、配置mysql高可用
(1)、配置mysql_service组和mysqlip
http://s2.运维网.com/wyfs02/M01/79/56/wKiom1aOYdrRO4HxAALkvNp5ljE795.jpg
选择类型为组
http://s3.运维网.com/wyfs02/M02/79/56/wKiom1aOYdvyVzNZAAJ3EEhNVmQ564.jpg
给组命名，我这里为mysql_service
http://s4.运维网.com/wyfs02/M00/79/56/wKiom1aOYd6AHic5AAK8HSNfEGY860.jpg
配置第一个资源mysqlip
http://s2.运维网.com/wyfs02/M01/79/54/wKioL1aOYgqhXiIfAAS4wn9MRuI151.jpg
(2)、配置mysqlstore
http://s5.运维网.com/wyfs02/M02/79/56/wKiom1aOY2nQRcaXAAP9XnElJ3E086.jpg
http://s4.运维网.com/wyfs02/M00/79/54/wKioL1aOY5Wwhiu9AATDqmCsDnA001.jpg
(3)、配置mysqld
http://s2.运维网.com/wyfs02/M01/79/56/wKiom1aOZBCQ0c2SAAPE7A86HC0921.jpg
这里可以选择LSB的mysqld，也可以选择ocf的mysql，自行根据需要，我这里选择的是mysqld
http://s1.运维网.com/wyfs02/M02/79/54/wKioL1aOZDzwF6lwAAS-U3w0x8o906.jpg
(4)、启动mysql_service组资源
http://s1.运维网.com/wyfs02/M01/79/54/wKioL1aOZS7xfq3AAAP2jRxpRIY547.jpg
启动成功，显示如下
http://s3.运维网.com/wyfs02/M02/79/56/wKiom1aOZTbiJZ83AANOG6tn-Do528.jpg
12、测试mysql服务的高可用性
此时mysql服务运行在node1节点上，连上node1，然后授权远程访问用户
# /usr/local/mysql/bin/mysql
Welcome to the MySQL monitor.Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.28-log MySQL Community Server (GPL)
Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'redhat';
Query OK, 0 rows affected (0.07 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)
mysql> \q
Bye
此时在node2节点上做测试，看是否能够远程访问mysql
# /usr/local/mysql/bin/mysql -uroot -h 172.16.1.110 -p
Enter password:
Welcome to the MySQL monitor.Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.5.28-log MySQL Community Server (GPL)
Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
+--------------------+
4 rows in set (0.06 sec)
mysql> CREATE DATABASE mydb;
Query OK, 1 row affected (0.03 sec)
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mydb |
| mysql |
| performance_schema |
| test |
+--------------------+
5 rows in set (0.02 sec)
mysql> \q
Bye 此时让node1节点处于standby状态
http://s3.运维网.com/wyfs02/M01/79/56/wKiom1aOZ8KwK-qNAANzGOissP8937.jpg
此时可以看到资源已成功转移至node2节点上了
http://s2.运维网.com/wyfs02/M02/79/55/wKioL1aOaEGgolvDAANKrT7gQ_0988.jpg
再来测试是否能连上msyql
# /usr/local/mysql/bin/mysql -uroot -h 172.16.1.110 -p
Welcome to the MySQL monitor.Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.5.28-log MySQL Community Server (GPL)
Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mydb |
| mysql |
| performance_schema |
| test |
+--------------------+
5 rows in set (0.14 sec)
mysql> \q
Bye 到此为止，一个简单的heartbeat + NFS + mysql的高可用服务搭建完成
页:
[1]