LVS+keepalived+LNMP架构

21fdfsd

DR模型原理：

客户端请求负载均衡器，负载均衡器根据调度算法，从后端realserver集群中选择一台机器，然后更改请求报文MAC地址，将目的MAC地址改为后端服务器的MAC地址，源MAC改为自己的MAC地址。后端服务器收到请求报文，然后处理，最后直接响应给客户端。

注意问题：

1 2 3 4 5 6

1.director和realserver都是同一个网段。 2.director和realserver都配置VIP地址。 3.director和realserver都配置route add VIP路由 4.director的vip配置在eth0:0上 5.realserver的vip配置在lo:0上。 6.realserver禁止响应arp数据包。

一：架构图



二：Realserver安装软件

1 2	192.168.1.119 192.168.1.121

1）nginx-1.6.1.tar.gz安装
http://www.iyunv.com/thread-98772-1-1.html


2）php-5.5.4.tar.gz安装
A:安装依赖包

1 2 3 4 5 6 7 8

cd /home/mcc/tools yum -y install libmcrypt-devel mhash-devel libxslt-devel  \ libjpeg  libjpeg-devel  libpng  libpng-devel  \ freetype  freetype-devel  libxml2  libxml2-devel  \ zlib  zlib-devel  glibc glibc-devel glib2 glib2-devel bzip2 \ bzip2-devel ncurses ncurses-devel curl curl-devel  \ e2fsprogs  e2fsprogs-devel  krb5  krb5-devel  libidn \ libidn-devel  openssl  openssl-devel  bzip2-devel  libcurl-devel

B:安装第三方依赖包

1 2 3

wget http://www.atomicorp.com/installers/atomic sh ./atomic yum  install  php-mcrypt  libmcrypt  libmcrypt-devel   -y

C:编译安装

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

./configure \         --prefix=/app/php  \         --with-config-file-path=/app/php/etc \         --enable-sockets \         --with-openssl \         --with-mhash   \         --with-mysql \         --with-mysqli \         --with-iconv-dir \         --with-freetype-dir \         --with-jpeg-dir \         --with-png-dir \         --with-zlib \         --with-curl \         --with-libxml-dir=/usr \         --enable-xml \         --with-xmlrpc \         --with-bz2 \         --enable-fpm  \         --enable-zip \         --enable-soap \         --enable-sysvsem \         --enable-inline-optimization \         --with-gd \         --with-gettext \         --with-mcrypt \         --disable-rpath \         --enable-mbstring \         --enable-ftp  \         --enable-bcmath \         --enable-shmop \         --enable-pcntl \         --enable-gd-native-ttf \         --without-pear make   &&  make   install

1 2 3 4 5

cp  sapi/fpm/init.d.php-fpm   /etc/rc.d/init.d/php-fpm chmod +x /etc/rc.d/init.d/php-fpm chkconfig --add php-fpm chkconfig php-fpm on cp  /app/php/etc/php-fpm.conf.default   /app/php/etc/php-fpm.conf

D:配置php-fpm.conf

1 2 3 4

pm.max_children = 150 pm.start_servers = 8 pm.min_spare_servers = 5 pm.max_spare_servers = 10

E:启动php-fpm

1	service  php-fpm  start

F:配置nginx，整合php环境
两台机器nginx配置相同

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

worker_processes 1; worker_rlimit_nofile 655360; events {         use  epoll;         worker_connections 1024; } http{     include       mime.types;     default_type  application/octet-stream;     sendfile          on;     tcp_nopush        on;     tcp_nodelay       on;     server_tokens     off;        keepalive_timeout 65; server {         listen        80;         server_name   localhost;         location /  {                 root  html;                 index index.html  index.htm index.php;         }         location ~ .*\.(html|htm|gif|jpg|jpeg|bmp|ico|txt|js|css)$ {             root      html;             expires   3d;         }         location ~ \.php$ {                 root   html;                 fastcgi_pass   127.0.0.1:9000;                 fastcgi_index  index.php;                 fastcgi_param  SCRIPT_FILENAME   $document_root$fastcgi_script_name;                 include        fastcgi_params;         }         error_page 500  502  503  504  /50x.html;         location = /50.html {             root   html;         } } }

F：代码上线，先上一台机器，安装完成后在把安装完成的discuz包拷贝到另外一台机器上。

1 2 3

192.168.1.121 cd  /app/nginx/html rz

安装时：不要用VIP访问。用192.168.1.121访问。


三：配置Realserver的LVS功能
启动脚本：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

#!/bin/bash # # chkconfig: - 90 10 # description: LVS DR RealServer . /etc/rc.d/init.d/functions VIP=192.168.1.200 case $1 in start)         /sbin/ifconfig lo down         /sbin/ifconfig lo up         /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP up         /sbin/route add -host $VIP dev lo:0         echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore         echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore         echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce         echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce         sysctl  -p 2>&1 >/dev/null         echo "RealServer Start OK!" ;; stop)         /sbin/ifconfig lo:0 down         /sbin/route del $VIP 2>&1 >/dev/null         echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore         echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore         echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce         echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce         sysctl  -p 2>&1 >/dev/null         echo "RealServer Stop OK!" ;; *)         printf "Usage: $0 {start|stop}\n"         exit 1 esac

2) 拷贝realserver脚本到/etc/init.d/

1	cp  realserver  /etc/ini.d/

3)使用

1 2	server realserver stop   server realserver start

4)开启启动

1 2	chkconfig –add    realserver chkconfig    realserver  on

启动问题：

1 2 3 4 5 6 7

[iyunv@localhost mcc]# ./realserver  start error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key error: "net.bridge.bridge-nf-call-iptables" is an unknown key error: "net.bridge.bridge-nf-call-arptables" is an unknown key 解决： modprobe bridge lsmod | grep  bridge

四：Director安装软件

1 2	192.168.1.116 192.168.1.117

安装ipvsadm-1.24.tar.gz

1 2 3 4 5 6 7

mkdir   /home/mcc/tools wget -c http://www.linuxvirtualserver.or ... ipvsadm-1.24.tar.gz ln -s /usr/src/kernels/2.6.* /usr/src/linux tar xf ipvsadm-1.24.tar.gz cd ipvsadm-1.24 make make  install

安装keepalived-1.2.12.tar.gz
http://www.iyunv.com/thread-7897-1-1.html


配置文件:

1 2	192.168.1.116  master 192.168.1.117  bakcup

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51

主配置文件: ! Configuration File for keepalived global_defs {    notification_email {     yujianglei@singulax.com    }    notification_email_from 15614119390@163.com    smtp_server smtp.163.com    smtp_connect_timeout 30    router_id LVS } vrrp_instance VI_1 {     state MASTER     interface eth0     virtual_router_id 10     priority 100     advert_int 1     authentication {         auth_type PASS         auth_pass jidian123     }     virtual_ipaddress {         192.168.1.200     } } virtual_server 192.168.1.200 80 {     delay_loop 6     lb_algo rr     lb_kind DR     nat_mask 255.255.255.0     persistence_timeout 50     protocol TCP     real_server 192.168.1.121 80 {         weight 1         TCP_CHECK {             connect_timeout  10             nb_get_retry 3             delay_before_retry 3             connect_port 80         }     }     real_server 192.168.1.119 80 {         weight 1         TCP_CHECK {             connect_timeout 10             nb_get_retry 3             delay_before_retry 3             connect_port 80         }     } }

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51

从配置文件: ! Configuration File for keepalived global_defs {    notification_email {     yujianglei@singulax.com    }    notification_email_from 15614119390@163.com    smtp_server smtp.163.com    smtp_connect_timeout 30    router_id LVS } vrrp_instance VI_1 {     state BACKUP     interface eth0     virtual_router_id 10     priority 99     advert_int 1     authentication {         auth_type PASS         auth_pass jidian123     }     virtual_ipaddress {         192.168.1.200     } } virtual_server 192.168.1.200 80 {     delay_loop 6     lb_algo rr     lb_kind DR     nat_mask 255.255.255.0     persistence_timeout 50     protocol TCP     real_server 192.168.1.121 80 {         weight 1         TCP_CHECK {             connect_timeout 10             nb_get_retry 3             delay_before_retry 3             connect_port 80         }     }     real_server 192.168.1.119 80 {         weight 1         TCP_CHECK {             connect_timeout 10             nb_get_retry 3             delay_before_retry 3             connect_port 80         }     } }

五：配置Director的LVS功能
启动脚本：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

#!/bin/bash # VIP=192.168.1.200 RIP1=192.168.1.119 RIP2=192.168.1.121 PORT=80 case $1 in start)         /sbin/ifconfig eth0:0 $VIP broadcast $VIP  netmask 255.255.255.255 up         /sbin/route  add -host $VIP dev eth0:0         echo "1"  >/proc/sys/net/ipv4/ip_forward         /sbin/iptables  -F         /sbin/ipvsadm   -C         /sbin/ipvsadm   -A -t $VIP:$PORT -s rr         /sbin/ipvsadm   -a -t $VIP:$PORT -r $RIP1 -g         /sbin/ipvsadm   -a -t $VIP:$PORT -r $RIP2 -g         touch  /var/lock/subsys/director.lock         echo "Director Start OK!" ;; stop)         /sbin/ifconfig eth0:0 down         /sbin/ipvsadm -C         echo "0"  >/proc/sys/net/ipv4/ip_forward         rm  -fr  /var/lock/subsys/director.lock         echo "Director Stop OK!" ;; status)         if [ -e /var/lock/subsys/director.lock ];then                 echo "Director is Running!"         else                 echo "Director is Down!"         fi ;; *)         printf "Usage: $0 {start|stop}\n"         exit 1 esac

---

一：安装MySQL需要依赖的包

1	yum install ncurses-devel -y

二：建立数据库启动账号

1 2 3

[iyunv@localhost ~]# cat /etc/passwd | grep  mysql [iyunv@localhost ~]# groupadd mysql [iyunv@localhost ~]# useradd -s /sbin/nologin -g mysql -M mysql

三：获取MySQL软件

1	dev.mysql.com获取mysql-5.5.45.tar.gz、cmake-2.8.8.tar.gz

四：上传MySQL软件到服务器
1.建立软件包目录

1 2 3

mkdir  -p /home/mcc/tools cd /home/mcc/tools rz

五：创建MySQL实例的数据文件目录和日志目录

1 2 3

mkdir -p /mydata55/data mkdir -p /mydata55/{bin_log,error_log,relay_log} tree /mydata55

六：创建MySQL实例的安装文件目录

1	mkdir /app55/mysql-5.5.45 -pv

七：安装MySQL软件
1.解压安装cmake-2.8.8.tar.gz软件包

1 2 3 4 5

tar xf cmake-2.8.8.tar.gz cd cmake-2.8.8 ./configure gmake gmake install

2.解压编译安装mysql-5.5.45.tar.gz软件包

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

tar xf mysql-5.5.45.tar.gz cd mysql-5.5.45 cmake . -DCMAKE_INSTALL_PREFIX=/app55/mysql-5.5.45  \ -DMYSQL_DATADIR=/mydata55/data  \ -DMYSQL_UNIX_ADDR=/mydata55/mysql.sock  \ -DDEFAULT_CHARSET=utf8  \ -DDEFAULT_COLLATION=utf8_general_ci  -DEXTRA_CHARSETS=all  \ -DENABLED_LOCAL_INFILE=ON  \ -DWITH_INNOBASE_STORAGE_ENGINE=1  \ -DWITH_FEDERATED_STORAGE_ENGINE=1  \ -DWITH_BLACKHOLE_STORAGE_ENGINE=1  \ -DWITHOUT_EXAMPLE_STORAGE_ENGINE=1  \ -DWITHOUT_PARTITION_STORAGE_ENGINE=1  \ -DWITH_FAST_MUTEXES=1  \ -DENABLED_LOCAL_INFILE=1 \ -DWITH_READLINE=1 \ -DWITH_EMBEDDED_SERVER=1 \ -DWITH_DEBUG=0 make &&  make  install

2.生成不带版本号的软连接/app/mysql,即mysql安装路径

1 2	cd /app55 ln -s  mysql-5.5.45  mysql

八：准备配置文件

1 2	[iyunv@localhost app55]# cp /app55/mysql/support-files/my-small.cnf /etc/my.cnf cp: overwrite `/etc/my.cnf'? y

1	vi  /etc/my.cnf

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84

[client] port = 3306 socket = /mydata55/mysql.sock default-character-set=utf8 [mysql] #No-auto-rehash default-character-set=utf8 [mysqld] user = mysql port = 3306 socket = /mydata55/mysql.sock character-set-server=utf8 basedir = /app55/mysql datadir = /mydata55/data open_files_limit = 1024 back_log = 600 max_connections = 800 max_connect_errors = 3000 table_cache = 614 external-locking = FALSE max_allowed_packet = 8M sort_buffer_size = 1M join_buffer_size = 1M thread_cache_size = 100 thread_concurrency = 2 query_cache_size = 2M query_cache_limit = 1M query_cache_min_res_unit = 2k default-storage-engine=InnoDB #default_table_type = InnoDB thread_stack = 192k transaction_isolation = READ-COMMITTED tmp_table_size = 2M max_heap_table_size = 2M long_query_time = 1 #log_long_format log-error = /mydata55/error_log/3306.err pid-file = /mydata55/3306.pid #二进制日志 log-bin = /mydata55/bin_log/mysql-bin log-bin-index = /mydata55/bin_log/mysql-bin.index binlog_format = mixed relay-log = /mydata55/relay_log/relay-bin relay-log-info-file = /mydata55/ relay_log /relay-log.info binlog_cache_size = 1M max_binlog_cache_size = 1M max_binlog_size = 2M expire_logs_days = 10 #MyISAM引擎 key_buffer_size = 16M read_buffer_size = 1M #read_md_buffer_size = 1M bulk_insert_buffer_size = 1M myisam_sort_buffer_size = 1M myisam_max_sort_file_size = 10G #myisam_max_extra_sort_file_size = 10G myisam_repair_threads = 1 myisam_recover lower_case_table_names = 1 skip-name-resolve slave-skip-errors = 1032,1062 replicate-ignore-db = mysql server-id = 1 #InnoDB引擎 innodb_additional_mem_pool_size = 4M innodb_buffer_pool_size = 32M innodb_data_file_path = ibdata1:128M:autoextend innodb_file_io_threads = 4 innodb_thread_concurrency = 8 innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 2M innodb_log_file_size = 4M innodb_log_files_in_group = 3 innodb_max_dirty_pages_pct = 90 innodb_lock_wait_timeout = 120 innodb_file_per_table = 1 [mysqldump] quick max_allowed_packet = 2M default-character-set=utf8 [mysqld_safe] log-error = /mydata55/error_log/3306.err pid-file = /mydata55/3306.pid default-character-set=utf8

九：配置环境变量

1 2 3

vi /etc/profile export PATH=/app55/mysql/bin:$PATH source  /etc/profile

提示：
在配置环境变量时，一定要将安装路径配置在PATH前面，否则系统中万一存在rpm安装的mysql。那么系统就会去寻找/usr/local/bin/下关于mysql的有关命令。

十：更改权限

1 2	chown  -R  mysql.mysql   /mydata55 chmod  -R  1777 /tmp

十一：初始化数据库

1 2	cd  /app55/mysql/scripts ./mysql_install_db --basedir=/app55/mysql --datadir=/mydata55/data  --user=mysql

十二：数据库启动脚本

1 2	cd /app55/mysql/support-files cp mysql.server /etc/init.d/mysqld

十三：授权，加密，优化

1 2 3 4 5 6 7 8

/etc/init.d/mysqld start mysql -uroot -p -e "grant all privileges on *.* to admin@'localhost' identified by 'jidian123' with grant option" mysql -uroot -p -e "grant all privileges on *.* to admin@'%' identified by 'jidian123' with grant option" mysql -uroot -p -e "flush  privileges;" mysql -uroot -p -e "drop database test;" mysql -uroot -p -e "delete from mysql.user where User='root'" mysql -uadmin -p'jidian123' -e "delete from mysql.user where User=''" mysql -uadmin -p'jidian123' -e "flush  privileges;"

十三：报错解决

1、启动报错日志：

1 2 3 4

150819  3:12:24 [ERROR] Plugin 'InnoDB' init function returned error. 150819  3:12:24 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 150819  3:12:24 [ERROR] Unknown/unsupported storage engine: InnoDB 150819  3:12:24 [ERROR] Aborting

解决方案：

1 2	移除数据目录下的ib_logfile0  ib_logfile1 mv  /mydata55/data/ib_logfile*   /home

重新启动：

1	/etc/init.d/mysqld  start

2、导入报错日志

1 2	ERROR 1665 (HY000): Cannot execute statement: impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine limited to row-based logging.

【报错原因】
innodb的事务隔离级别是read commited或者read uncommited模式时，binlog不可以使用statement模式。

解决：

1	set global binlog_format=mixed