通过ansible部署高可用LNAMMKP架构

hghgfg

通过ansible部署高可用LNAMMKP架构

主/备模式高可用keepalived+{nginx(proxy)|lvs}
两台主机：httpd+php+Discuz!+phpMyAdmin
一台主机：memcached用来缓存php的session；
一台主机：mysql-server或mariadb-server；

实验规划：
  这里我规划用备用模式高可用两台keepalived+nginx(proxy)服务器作为前端代理，两台主机httpd+php+MariaDB+Discuz作为后端真实机，memcached放在第一台director上用来缓存php的session，最后用一台ansible主机实现对所有主机的部署调用。

ansible主机：172.16.1.7
director1(nginx+keepalived+memcached)：172.16.1.10，虚拟ip：172.16.1.100
director2(nginx+keepalived)：172.16.1.5
RS1(Apache+php+MariaDB)：172.16.1.3
RS2(Apache+php+MariaDB)：172.16.1.6

拓扑图如下：


注意：
  1.把所有节点的防火墙和SElinux关闭，避免对实验干扰。
  2.同步所有节点的时间。

当部署完所有的应用后，生成如下所以的文件和子文件。

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69

[iyunv@zj07 ansible]# tree . ├── ansible.cfg ├── files │   └── nginx.conf ├── hosts ├── playbooks │   ├── amp.yml │   ├── first.retry │   ├── first.yml │   ├── ngx.retry │   └── ngx.yml ├── roles │   ├── amp │   │   ├── default │   │   ├── files │   │   │   └── db.sh │   │   ├── handlers │   │   ├── tasks │   │   │   └── main.yml │   │   ├── templates │   │   └── vars │   ├── keepalived │   │   ├── default │   │   ├── files │   │   ├── handlers │   │   │   └── main.yml │   │   ├── tasks │   │   │   └── main.yml │   │   ├── templates │   │   │   └── keepalived.conf.j2 │   │   └── vars │   ├── memcached │   │   ├── default │   │   ├── files │   │   │   └── Discuz_X3.2_SC_UTF8.zip │   │   ├── handlers │   │   │   └── main.yml │   │   ├── tasks │   │   │   └── main.yml │   │   ├── templates │   │   │   ├── php.ini.j2 │   │   │   └── sessstore.php.j2 │   │   └── vars │   ├── memcached1 │   │   ├── default │   │   ├── files │   │   ├── handlers │   │   ├── tasks │   │   │   └── main.yml │   │   ├── templates │   │   └── vars │   └── nginx │       ├── default │       ├── files │       │   ├── default.conf.j2 │       │   └── ip_forward.sh │       ├── handlers │       │   └── main.yml │       ├── tasks │       │   └── main.yml │       ├── templates │       │   ├── nginx.conf.j2 │       │   └── nginx.repo.j2 │       └── vars ├── zrs.retry └── zrs.yml 36 directories, 29 files

下面开始部署
-------------------------------------------------------------------------------------------
配置ansible主机
1.yum -y install ansible
2.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

vim /etc/ansible/hosts [knsrvs] 172.16.1.10  STATE=MASTER     PRI=100 172.16.1.5  STATE=BACKUP     PRI=98 [websrvs] 172.16.1.3 172.16.1.6 [memcached1] 172.16.1.10 [memcached] 172.16.1.3 172.16.1.6

3.建立免秘钥登陆，发给所以主机
ssh-keygen -t rsa -f .ssh/id_rsa -P ''
ssh-copy-id -i .ssh/id_rsa.pub root@172.16.1.10
ssh-copy-id -i .ssh/id_rsa.pub root@172.16.1.5
ssh-copy-id -i .ssh/id_rsa.pub root@172.16.1.3
ssh-copy-id -i .ssh/id_rsa.pub root@172.16.1.6
ansible all -m ping测试连通性

4.在ansible主机上安装nginx和keepalived和php-fpm，以便ansible过程中需要使用其配置文件
-------------------------------------------------------------------------------------------
keepalived配置
1.mkdir /etc/ansible/roles/keepalived/{files,tasks,templates,handlers,vars,default} -pv
2.vim /etc/ansible/roles/keepalived/tasks/main.yml

3.vim /etc/ansible/roles/keepalived/handlers/main.yml

1 2	- name: restart keepalived server   service: name=keepalived state=restarted

4.复制刚才安装的keepalived主配置文件到指定目录,并修改如下
vim /etc/ansible/roles/keepalived/template/keepalived.conf.j2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

! Configuration File for keepalived global_defs {    notification_email { root@localhost    }    notification_email_from keepalived@localhost    smtp_server 127.0.0.1    smtp_connect_timeout 30    router_id node1    vrrp_mcast_group4 224.0.43.100 } vrrp_script ngx_server {     script "killal -0 nginx"     interval 1     weight -5 } vrrp_instance VI_1 {     state {{ STATE }}     interface eno16777736     virtual_router_id 43     priority {{ PRI }}     advert_int 1     track_script ngx_server     authentication {     auth_type PASS         auth_pass zrs66zrs     }     virtual_ipaddress {         172.16.1.100/32 brd 172.16.1.100  dev eno16777736 label eno16777736:0     } }

-------------------------------------------------------------------------------------------
nginx的配置：
1.mkdir /etc/ansible/roles/nginx/{files,tasks,templates,handlers,vars,default} -pv
2.vim /etc/ansible/roles/nginx/tasks/main.yml

3.vim /etc/ansible/roles/nginx/templates/nginx.repo.j2

1 2 3 4 5

[nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0 enabled=1

复制刚才安装的nginx主配置文件到指定目录,并修改如下
vim /etc/ansible/roles/nginx/templates/nginx.conf.j2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

user  nginx; worker_processes  1; error_log  /var/log/nginx/error.log warn; pid        /var/run/nginx.pid; events {     worker_connections  1024; } http {     include       /etc/nginx/mime.types;     default_type  application/octet-stream;     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '                       '$status $body_bytes_sent "$http_referer" '                       '"$http_user_agent" "$http_x_forwarded_for"';     access_log  /var/log/nginx/access.log  main;     upstream amp {                   server 172.16.1.3;             server 172.16.1.6;     }    server {     listen 80;     location / {         proxy_pass http://amp;         proxy_set_header X-Real-IP $remote_addr;     } }     sendfile        on;     #tcp_nopush     on;     keepalive_timeout  65;     #gzip  on;     include /etc/nginx/conf.d/*.conf; }

4.复制刚才安装的nginx主配置文件到指定目录,并修改如下
vim /etc/ansible/roles/nginx/files/default.conf.j2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

server {     listen       80 default_server;     server_name  localhost;     include /etc/nginx/default.d/*.conf;     location / {         root   /usr/share/nginx/html;         proxy_pass http://amp;         index  index.html index.htm;     }     error_page  404              /404.html;     location = /404.html {         root   /usr/share/nginx/html;     }     error_page   500 502 503 504  /50x.html;     location = /50x.html {         root   /usr/share/nginx/html;     } }

5.创建ip_forward脚本，开启前端代理服务器的核心转发功能
vim /etc/ansible/roles/nginx/files/ip_forward.sh

1 2 3

#!/bin/bash # echo > 1 /proc/sys/net/ipv4/ip_forward

-------------------------------------------------------------------------------------------
amp的配置：

1.mkdir /etc/ansible/roles/amp/{files,tasks,templates,handlers,vars,default} -pv

2.vim /etc/ansible/roles/amp/tasks/main.yml

3.创建testdb数据库，和授权用户访问
vim /etc/ansible/roles/amp/files/db.sh

1 2 3 4 5 6

#!/bin/bash # mysql -u root -e "         create database testdb;         grant all on testdb.* TO 'testuser'@'localhost' IDENTIFIED BY '123456';         FLUSH PRIVILEGES;"

-------------------------------------------------------------------------------------------
memcached配置
memcached缓存服务器为第一台调度器即172.16.1.10服务器，所以单独配置为memcached1
1.mkdir /etc/ansible/roles/memcached1/{files,tasks,templates,handlers,vars,default} -pv
2.vim /etc/ansible/roles/memcached1/tasks/main.yml

后端服务器上的配置
3.mkdir /etc/ansible/roles/memcached/{files,tasks,templates,handlers,vars,default} -pv
4.vim /etc/ansible/roles/memcached/tasks/main.yml

说明：
第二个任务：在本机安装php-fpm即可出现并修改/etc/php.ini该配置文件中的[Session]段中的缓存路径为如下，
session.save_handler = memcache
session.save_handler = "tcp://172.16.1.10:11211"

第四个任务：是建立一个缓存测试页面
vim /etc/ansible/roles/memcached/templates/sessstore.php

1 2 3 4 5 6 7 8 9 10

<?php $mem = new Memcache; $mem->connect("172.16.1.10", 11211)  or die("Could not connect"); $version = $mem->getVersion(); echo "Server's version: ".$version."<br/>\n"; $mem->set('hellokey', 'Hello World', 0, 600) or die("Failed to save data at the memcached server"); echo "Store data in the cache (data will expire in 600 seconds)<br/>\n"; $get_result = $mem->get('hellokey'); echo "$get_result is from memcached server.";          ?>

第五个任务：将Discuz安装包放置在该目录下，传送到后端主机的指定目录。
第六个任务：解压该压缩包并赋予制定用户的权限
-------------------------------------------------------------------------------------------
创建主配置文件，并且调用roles：
vim /etc/ansible/zrs.yml

配置完成
先测试运行一下ansible-playbook --check zrs.yml，因为有的主机已经安装了某些软件，所以测试显示的是changed或者ok或者skipping

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120

[iyunv@zj07 ansible]# ansible-playbook --check zrs.yml PLAY [knsrvs] ************************************************************************** TASK [Gathering Facts] ***************************************************************** ok: [172.16.1.10] ok: [172.16.1.5] TASK [keepalived : install keepalived] ************************************************* changed: [172.16.1.5] changed: [172.16.1.10] TASK [keepalived : install conf file] ************************************************** changed: [172.16.1.10] changed: [172.16.1.5] TASK [keepalived : start keepalived] *************************************************** changed: [172.16.1.10] changed: [172.16.1.5] TASK [nginx : copy nginx repo] ********************************************************* changed: [172.16.1.5] ok: [172.16.1.10] TASK [nginx : install nginx] *********************************************************** ok: [172.16.1.10] changed: [172.16.1.5] TASK [nginx : copy conf file] ********************************************************** changed: [172.16.1.10] changed: [172.16.1.5] TASK [nginx : reload nginx] ************************************************************ changed: [172.16.1.10] changed: [172.16.1.5] TASK [nginx : start nginx] ************************************************************* changed: [172.16.1.5] ok: [172.16.1.10] TASK [nginx : ip_forward] ************************************************************** changed: [172.16.1.5] changed: [172.16.1.10] RUNNING HANDLER [keepalived : restart keepalived server] ******************************* changed: [172.16.1.5] changed: [172.16.1.10] PLAY [websrvs] ************************************************************************* TASK [Gathering Facts] ***************************************************************** ok: [172.16.1.6] ok: [172.16.1.3] TASK [amp : install apache-php-mysql some package on CentOS 6] ************************* skipping: [172.16.1.3] => (item=[]) skipping: [172.16.1.6] => (item=[]) TASK [amp : install apache-php-mysql some package on CentOS 7] ************************* ok: [172.16.1.6] => (item=[u'httpd', u'mariadb-server', u'php-fpm', u'php-mysql']) changed: [172.16.1.3] => (item=[u'httpd', u'mariadb-server', u'php-fpm', u'php-mysql']) TASK [amp : start apm server] ********************************************************** skipping: [172.16.1.3] => (item=httpd) skipping: [172.16.1.3] => (item=mysqld) skipping: [172.16.1.6] => (item=httpd) skipping: [172.16.1.6] => (item=mysqld) TASK [amp : start apm server] ********************************************************** changed: [172.16.1.6] => (item=httpd) changed: [172.16.1.3] => (item=httpd) changed: [172.16.1.6] => (item=mariadb) changed: [172.16.1.3] => (item=mariadb) PLAY [memcached1] ********************************************************************** TASK [Gathering Facts] ***************************************************************** ok: [172.16.1.10] TASK [memcached1 : install memcached] ************************************************** ok: [172.16.1.10] TASK [memcached1 : start memcached] **************************************************** changed: [172.16.1.10] PLAY [memcached] *********************************************************************** TASK [Gathering Facts] ***************************************************************** ok: [172.16.1.6] ok: [172.16.1.3] TASK [memcached : install memcached php session] *************************************** ok: [172.16.1.3] ok: [172.16.1.6] TASK [memcached : transfer php session conf] ******************************************* changed: [172.16.1.3] changed: [172.16.1.6] TASK [memcached : reload httpd] ******************************************************** changed: [172.16.1.6] changed: [172.16.1.3] TASK [memcached : copy sessstore file] ************************************************* changed: [172.16.1.6] changed: [172.16.1.3] TASK [memcached : copy discuz file] **************************************************** changed: [172.16.1.6] changed: [172.16.1.3] TASK [memcached : tar discuz] ********************************************************** changed: [172.16.1.6] changed: [172.16.1.3] PLAY RECAP ***************************************************************************** 172.16.1.10                : ok=14   changed=8    unreachable=0    failed=0    172.16.1.3                 : ok=10   changed=7    unreachable=0    failed=0    172.16.1.5                 : ok=11   changed=10   unreachable=0    failed=0    172.16.1.6                 : ok=10   changed=6    unreachable=0    failed=0

测试没问题，查看缓存页面是否成功，显示没问题。

开始运行
ansible-playbook zrs.yml

客户端查看，成功。