nginx+keepalived 高可用兼负载均衡集群
版本作者联系方式最后更新
v0.3purplegrapepurplegrape4@gmail.com2013-04-10
Nginx是一个高性能的web服务器，同时也是一个优秀的反向代理服务器，本文利用两台Dell R720 构建一个高可用兼负载均衡的Linux web集群。
原理
通过nginx分别搭建两个web服务器，监听在本地非80端口；然后利用nginx构建一个包含两个节点的负载均衡池；最后通过keepalived实现负载均衡池的高可用。两个节点同时运行时，备用节点可以承载一半的前端流量，一个节点宕机后，负载均衡器通过健康检查，将失效节点踢出集群。
环境：CentOS 6.4 x86_64
软件版本
nginx1.2.7
keepalived1.2.7
php-fpm5.4.13
主机名IP地址
web01192.168.122.10
web02192.168.122.20
VIP192.168.122.30
安装nginx、keepalived和php-fpm等（编译过程略）
yum install nginx keepalived php-fpm php-gd php-xml php-mysql php-pecl-memcached php-pecl-sphinx watchodg -ychkconfig nginx onchkconfig keepalived onchkconfig php-fpm onchkconfig watchdog on
配置nginx
nginx主配置文件 /etc/nginx/nginx.conf
user nginx;worker_processes 2;worker_rlimit_nofile 65535;error_log /var/log/nginx/error.log warn;pid /var/run/nginx.pid;events {use epoll;worker_connections 2048;}http {include /etc/nginx/mime.types;default_type application/octet-stream;log_format main '$remote_addr - $remote_user [$time_local] "$request" ''$status $body_bytes_sent "$http_referer" ''"$http_user_agent" "$http_x_forwarded_for"';#access_log /var/log/nginx/access.log main;sendfile on;server_tokens off;#tcp_nopush on;keepalive_timeout 65;gzip on;gzip_static on;gzip_disable "msie6";gzip_http_version 1.1;gzip_vary on;gzip_comp_level 6;gzip_proxied any;gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x$gzip_buffers 16 8k;client_max_body_size 20m;client_body_buffer_size 128k;
proxy_hide_header Vary;proxy_connect_timeout 600;proxy_read_timeout 600;proxy_send_timeout 600;proxy_buffer_size 16k;proxy_buffers 4 64k;proxy_busy_buffers_size 128k;#proxy_temp_file_write_size 128k
proxy_temp_path /dev/shm/tmp;proxy_cache_path /dev/shm/proxycache/ levels=1:2 keys_zone=shmcache:10m inactive=2h max_size=500m;
include /etc/nginx/conf.d/*.conf;}
网站主配置文件/etc/nginx/conf.d/default.conf
upstream webservers {ip_hash ;server 192.168.122.10:88 max_fails=3 fail_timeout=3s weight=2 ;server 192.168.122.20:88 max_fails=3 fail_timeout=3s weight=2 ;}server {listen 80 ;server_name localhost ;access_log /var/log/nginx/web01.access.log main;location / {proxy_redirect off;proxy_pass http://webservers ;
proxy_cache shmcache;proxy_cache_valid 200 302 1d;proxy_cache_valid 404 1h;proxy_cache_valid any 10m;proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
proxy_set_header Host $host;proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;proxy_next_upstream error timeout invalid_header http_500 http_502 http_504;}location /status {stub_status on;access_log off;allow 192.168.122.0/24;}}server {listen 192.168.122.10:88;server_name localhost;root /usr/share/nginx/html;index index.html index.htm index.php;#charset koi8-r;location / {if (!-e $request_filename) {rewrite ^/(.*)$ /index.php?q=$1 last;}}#error_page 404 /404.html;# redirect server error pages to the static page /50x.html#error_page 500 502 503 504 /50x.html;# proxy the PHP scripts to Apache listening on 127.0.0.1:80##location ~ \.php$ {# proxy_pass http://127.0.0.1;#}# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000#location ~ \.php$ {#try_files $uri = 404;fastcgi_pass 127.0.0.1:9000;#fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;fastcgi_index index.php;fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;fastcgi_buffer_size 128k;fastcgi_buffers 256 16k;fastcgi_busy_buffers_size 256k;fastcgi_temp_file_write_size 256k;fastcgi_read_timeout 240;include fastcgi_params;}if ($fastcgi_script_name ~ \..*\/.*php) {return 403;}# deny access to hiden file . (filename begin with ".")location ~ /\. {access_log off;log_not_found off;deny all;}# deny access to bakup file .(any filename end with "~" )location ~ ~$ {access_log off;log_not_found off;deny all;}# cache image filelocation ~* \.(jpg|jpeg|gif|png|css|js|ico|xml|swf)$ {expires 1d;}# don't log robots and favionlocation = /robots.txt { access_log off; log_not_found off; }location = /favicon.ico { access_log off; log_not_found off; }# deny access to .htaccess files, if Apache's document root# concurs with nginx's one#location ~ /\.ht {deny all;}}
配置php-fpmphp-fpm全称php fastcgi process manager，用于管理php的fastcgi进程，自从php5.3.3之后集成进了php源码中。默认的php-fpm用户是apache，我们需要修改成nginx，代码：
sed -i 's/apache/nginx/g' /etc/php-fpm.d/www.conf
配置keepalivedkeepalived是一个高可用软件，通过vrrp心跳来检测对方是否存活。
keepalived主配置文件
global_defs {notification_email {root@localhost}notification_email_from keepalived01@web01.test.orgsmtp_server 127.0.0.1smtp_connect_timeout 30}vrrp_instance VI_1 {state MASTERinterface eth0virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.122.30 label eth0:1}}
配置watchdog
watchdog是个健壮可靠的内核模块，即使在高负载的系统中仍然可以存活。它能够执行检测脚本，定时检查nginx和keepalived进程是否存在，如果不存在可以重新开启进程。
watchdog主配置文件/etc/watchdog.conf
min-memory = 1repair-binary = /etc/watchdog.d/repair.shtest-binary = /etc/watchdog.d/test.shtest-timeout = 5watchdog-device = /dev/watchdogadmin = rootinterval = 10logtick = 1realtime = yespriority = 1pidfile = /var/run/syslogd.pidpidfile = /var/run/nginx.pidpidfile = /var/run/keepalived.pidpidfile = /var/run/php-fpm/php-fpm.pid
测试脚本，权限751
#!/usr/bin/env bash#author:purplegrape#desc: shell script to check if keepalived/nginx/php-fpm down.
keepalived_proc=`pgrep keepalived |wc -l`nginx_proc=`pgrep nginx|wc -l`php_fpm_porc=`pgrep php-fpm|wc -l`
if [$keepalived_proc == 0 ];thenexit 1fi
if [$nginx_proc == 0 ];thenexit 1fi
if [$php_fpm_proc == 0 ];thenexit 1fi
修复脚本,权限751
#!/usr/bin/env bash#author:purplegrape#desc: shell script to repair keepalived/nginx/php-fpm.
keepalived_proc=`pgrep keepalived |wc -l`nginx_proc=`pgrep nginx|wc -l`php_fpm_proc=`pgrep php-fpm|wc -l`
if [$keepalived_proc == 0 ];thenwall “keepalived is dead”/etc/init.d/keepalived restartsleep 30/etc/init.d/nginx restartfi
if [$nginx_proc == 0 ];thenif [$keepalived_proc != 0 ];thenwall “nginx is dead”/etc/init.d/nginx restartfifi
if [ $php_fpm_proc == 0 ]; thenwall “php-fpm is dead”/etc/init.d/php-fpm restartfi
测试打开浏览器，访问地址http://192.168.122.30
男人靠的住，母猪能上树! 穿别人的鞋，走自己的路，让他们找去吧。 啥时硬件也可以COPY就好了! 帮帮顶顶！！ 爱护环境，人人有病。 内练一口气，外练一口屁。
页:
[1]