keepalived_lvs-dr实现discuz负载均衡和高可用
前言：前一篇文章中由于前端调度器由于只有一台，如果损坏了整个系统将无法对外提供服务，因此它成为了系统的SPOF，需要对其做高可用，此时暂不考虑后端存储故障的情况。经过两个小时的配置，终于将此配置出来，下面将配置流程写下来
由于我的电脑使用的是虚拟机，所以在配置DR模型的时候，需要使用两块网卡，一块网卡用于桥接外网，另外一块使用NAT模式进行实验测试！！！(此处无需关注链接外网的网卡，本地测试NAT模式很实用)
高可用模型：
主备模型：即一台主机位MASTER，另一台主机位BACKUP,只有当MASTER主机宕机时，BACKUP主机取而代之成为MASTER才会开始工作，因此在单主模型中，始终有一台主机不工作，会造成资源浪费。
双主模型：两个单主模型叠加，两台主机互为主备，在前段DNS解析时加上双A记录，即可实现两台主机同时工作，而且实现了高可用，且兼备负载均衡。
此处我们使用主备模型，两模型之间切换过于简单，此处不描述主主模型博客！
拓扑图：
各个主机的IP地址
Host VS1
eno16777736 10.0.0.201/8
gateway: 10.0.0.254
Host VS2 eno16777736 10.0.0.203/8
gateway: 10.0.0.254
Host RS1
eno16777736 10.0.0.101/8 (RIP1)
lo:0 10.0.0.111/32 (VIP) gateway: 10.0.0.254
Host RS2
eno16777736 10.0.0.102/8 (RIP2)
lo:0 10.0.0.111/32 (VIP)
gateway: 10.0.0.254
Host DB
eno16777736 10.0.0.202/8
gateway: 10.0.0.254
时间同步：
1
2
# ntpdate cn.pool.ntp.org
# hwclock --sysohc
安装软件：HostDB
安装二进制mariadb-5.5.46
安装NFS
1
# yum install -y nfs-utils
HostRS1安装nginx，注意nginx属于epel源
1
# yum install-y nginx php-fpm php-mbstring php-mysql nfs-utils mariadb
HostRS2安装nginx，注意nginx属于epel源
1
# yum install-y nginx php-fpm php-mbstring php-mysql nfs-utils mariadb
Host VS1
安装keepalived是为了实现高可用，ipvsadm用来查看状态 http用来提供sorry server
1
# yum install -y keepalived ipvsadm httpd
Host VS2
安装keepalived是为了实现高可用，ipvsadm用来查看状态 http用来提供sorry server
1
# yum install -y keepalived ipvsadm httpd
HostDB
完成安全初始化后，创建discuz数据库和discuz用户，并授权其可以远程操作数据库
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# mysql_secure_installation
# mysql -u root -p
Enter password:
Welcome to the MariaDB monitor.Commands end with ; or g.
Your MariaDB connection id is 3
Server version: 5.5.46-MariaDB-log MariaDB Server
Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
MariaDB [(none)]> create database discuz;
Query OK, 1 row affected (0.00 sec)
MariaDB [(none)]> create user 'discuz'@'localhost' identified by 'magedu';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> grant all privileges on discuz.* to 'discuz'@'%' identified by 'magedu';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)
配置NFS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# mkdir /nfshare/
# ls -dl /nfshare/
drwxr-xr-x 2 root root 6 May 9 17:01 /nfshare/
# echo "/nfshare/ 10.0.0.101(rw,no_root_squash,async) 10.0.0.102(rw,no_root_squash,async)" > /etc/exports
# systemctl start rpcbind <--如果是CentOS-6要使用/etc/init.d/rpcbind start
Starting rpcbind:
# /etc/init.d/nfs start
Starting NFS services:
Starting NFS quotas:
Starting NFS mountd:
Starting NFS daemon:
Starting RPC idmapd:
# systemctl enable rpcbind<--如果是6 chkconfig rpcbind on
# systemctl enable rpcbind<--如果是6 chkconfig nfs on
# chkconfig rpcbind --list
rpcbind 0:off 1:off 2:on 3:on 4:on 5:on 6:off
# chkconfig nfs --list
nfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off
# showmount -e 127.0.0.1
Export list for 127.0.0.1:
/nfshare 172.18.71.102,172.18.71.101 此步骤如果出现问题，请查看《lvs-dr负载均衡Discuz》
解压discuz的程序包至/nfsshare/目录
1
2
3
4
# mkdir /nfshare/discuz
# unzip /Discuz_X3.2_SC_UTF8.zip -d /nfshare/discuz/
# ls /nfshare/discuz/
readmeuploadutility
Host RS1
我们如果要为RS配置VIP，为了不产生IP冲突，我们要先禁止RS响应VIP的ARP请求
1
2
3
4
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
在本地回环接口别名上配置VIP,并增添路由
1
2
# ip addr add 10.0.0.111/32 broadcast 10.0.0.111 dev lo labal lo:0
# route add -host 10.0.0.111 lo:0
注意:CentOS-7最小安装，默认没有route命令，需要安装net-tools(不用google了，遇到这个问题的都特么是外国人。没错，很坑爹)
测试链接HostDB上的mariadb
1
# mysql -h 10.0.0.202 -u discuz -p
挂载HostDB上的NFS共享存储目录，属主属组改为apache
1
2
3
4
5
6
7
8
9
10
# showmount -e 10.0.0.202
Export list for 10.0.0.202:
/nfshare 10.0.0.102,10.0.0.101
# mkdir /htdocs
# ls -ld /htdocs/
drwxr-xr-x 2 root root 6 May 9 17:05 /htdocs/
# mount -t nfs 10.0.0.202:/nfshare /htdocs
# ls /htdocs/
discuz
# chown -R apache:apache /htdocs/discuz/
启动php-fpm
1
# systemctl start php-fpm.service
修改nginx的配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
http {
log_formatmain'$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log/var/log/nginx/access.logmain;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
include /etc/nginx/conf.d/*.conf;
server {
listen80;
server_name www.rs1.com;
# Load configuration files for the default server block.
include/etc/nginx/default.d/*.conf;
location / {
root/htdocs/discuz/upload;
index index.html index.htm index.php;
}
location ~ .php$ {
root/htdocs/discuz/upload;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include /etc/nginx/fastcgi.conf;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
}
# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# nginx
客户端浏览器访问http://10.0.0.101/install/,在这里就将discuz安装完成
Host RS2
与Host RS1的配置相同，此处不再复制
Host VS1
修改keepalived配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from admin@iyunv.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_mcast_group4 224.0.71.18
}
vrrp_instance VI_1 {
state MASTER
interface eno16777736
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.111/32 dev eno16777736 label eno16777736:0
}
}
virtual_server 10.0.0.111 80 {
delay_loop 6
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 10.0.0.101 80 {
weight 1
HTTP_GET {
url {
path /install/
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.0.0.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
启动哈个服务
Host VS2修改keepalived配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from admin@iyunv.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_mcast_group4 224.0.71.18
}
vrrp_instance VI_1 {
state BACKUP
interface eno16777736
virtual_router_id 52
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.111/32 dev eno16777736 label eno16777736:0
}
}
virtual_server 10.0.0.111 80 {
delay_loop 6
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 10.0.0.101 80 {
weight 2
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.0.0.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
至此，地址已经可以飘起来了，如果遇到什么麻烦请联系我QQ:2137626
为HostVS1 和HostVS2 提供Sorry页面，并启动服务
1
2
3
4
# echo "服务维护中,请稍后访问." > /var/www/html/index.html
#sed -i "s@^#ServerName.*@ServerName admin.ws.com@" /etc/httpd/conf/httpd.conf
# /etc/init.d/httpd start
Starting httpd:
测试验证
通过客户端浏览器强制刷新10.0.0.111在HostVS1和2上都可以通过ipvsadm查看，两个后端RS主机轮询的很开心
1
2
3
4
5
6
7
ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP10.0.0.111:http rr
-> 10.0.0.101:http Route 1 3 6
-> 10.0.0.102:http Route 1 3 0
页:
[1]