设为首页 收藏本站

运维网

 找回密码
 立即注册
查看: 1568|回复: 0

[经验分享] nginx+keepalived+tomcat配置高可用web集群

[复制链接]
rtre
累计签到：1 天
连续签到：1 天
发表于 2015-9-25 10:56:02 | 显示全部楼层 |阅读模式

基本架构：

角色
ip
安装软件
作用
主机名
nginx主
192.168.247.129
nginx+keepalived
反向代理
nginxmaster.com
nginx备
192.168.247.130
nginx+keepalived
反向代理
nginxsalve.com
tomcat1
192.168.247.128
tomcat
web服务器
tomcat1.com
tomcat2
192.168.247.131
tomcat
web服务器
tomcat2.com
nfs主
192.168.247.132
nfs
低端共享存储
nfsmaster.com
nfs备
192.168.247.133
nfs
低端共享存储
nfsslave.com
虚拟ip
192.168.247.150







集群部署：

    tomcat安装部署：

        编译安装tomcat：（主，备）

            安装JDK，运行java环境：（主，备）
                卸载centos自带的JDK：
1
2
rpm -qa | grep java
yum -y remove java-1.6.0-openjdk-1.6.0.0-1.7.b09.el5



                下载安装JDK：

1
2
##下载地址：http://www.oracle.com/technetwor ... nloads-1880260.html
##下载最新的（64位）：jdk-7u79-linux-x64.rpm



1
2
3
4
##在/usr下建立一个java目录，以备将java程序安装在此目录下
mkdir /usr/java
cd /usr/local/src
rpm -ivh jdk-7u79-linux-x64.rpm        ##安装后，在/usr/java目录下就会生成一个新的目录jdk1.7.0_79，该目录下存放的是安装后的文件



1
2
3
4
5
6
7
##设置环境变量：
vi /etc/profile  ##在最后添加：
export JAVA_HOME=/usr/java/jdk1.7.0_79
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

source /etc/profile



1
2
3
4
5
##验证是否安装成功：
java -version         ##出现如下则正常
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)




            安装tomcat7：（主，备）
1
2
3
4
软件：apache-tomcat-7.0.29.tar.gz
cd /usr/local/src/
tar -zxf apache-tomcat-7.0.42.tar.gz
cp -a apache-tomcat-7.0.42 /usr/local/tomcat7/



1
2
3
4
5
6
7
8
/usr/local/tomcat7/bin/startup.sh       ##启动tomcat，出现如下则正常
Using CATALINA_BASE:   /usr/local/tomcat7
Using CATALINA_HOME:   /usr/local/tomcat7
Using CATALINA_TMPDIR: /usr/local/tomcat7/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_79
Using CLASSPATH:       /usr/local/tomcat7/bin/bootstrap.jar:/usr/local/tomcat7/bin/tomcat-juli.jar

##设置开机自动启动



1
##访问下：    ip:8080   看是否正常



        配置tomcat：（主，备）

            配置server.xml文件：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
cd /usr/local/tomcat7/conf/
vi server.xml
##找到：
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
##将其注释掉

##添加如下（已优化）：
    <Connector  port="8080"   

                protocol="HTTP/1.1"  

                maxHttpHeaderSize="8192"  

                maxThreads="1000"  

                minSpareThreads="100"  

                maxSpareThreads="1000"  

                minProcessors="100"  

                maxProcessors="1000"  

                enableLookups="false"  

                compression="on"  

                compressionMinSize="2048"  

                compressableMimeType="text/html,text/xml,text/javascript,text/css,text/plain"  

                connectionTimeout="20000"  

                URIEncoding="utf-8"  

                acceptCount="1000"  

                redirectPort="8443"  

                disableUploadTimeout="true"/>
##配置优化：http://ihuangweiwei.iteye.com/blog/1233941


##找到：Valve标签
##将pattern参数的值修改为：
"%{X-Real-IP}i %l %u %t &quot;%r&quot; %s %b"        ##{X-Real-IP}i，获取的是真实客户端ip，不是代理机ip
##每个参数的含义：http://twb.iteye.com/blog/182100
##nginx前端代理tomcat取真实客户端IP：http://www.cnphp6.com/archives/60908



1
2
3
4
5
6
7
8
9
10
11
12
13
##重新启动tomcat：
[iyunv@tomcat1 ~]# /usr/local/tomcat7/bin/shutdown.sh
Using CATALINA_BASE:   /usr/local/tomcat7
Using CATALINA_HOME:   /usr/local/tomcat7
Using CATALINA_TMPDIR: /usr/local/tomcat7/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_79
Using CLASSPATH:       /usr/local/tomcat7/bin/bootstrap.jar:/usr/local/tomcat7/bin/tomcat-juli.jar
[iyunv@tomcat1 ~]# /usr/local/tomcat7/bin/startup.sh
Using CATALINA_BASE:   /usr/local/tomcat7
Using CATALINA_HOME:   /usr/local/tomcat7
Using CATALINA_TMPDIR: /usr/local/tomcat7/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_79
Using CLASSPATH:       /usr/local/tomcat7/bin/bootstrap.jar:/usr/local/tomcat7/bin/tomcat-juli.jar







    反向代理的部署：

        编译安装nginx：（主，备）
            安装nginx的依赖包：

1
yum -y install gcc gcc-c++ autoconf automake zlib zlib-devel openssl openssl-devel pcre-devel



            编译安装pcre，使nginx支持rewrite重写：

1
2
3
4
5
6
cd /usr/local/src/
wget tar -zxf pcre-8.35.tar.gz
cd pcre-8.35
./configure
make
make install



            安装nginx：

1
2
3
4
5
6
7
8
cd /usr/local/src/
wget http://nginx.org/download/nginx-1.6.2.tar.gz
tar -zxf nginx-1.6.2.tar.gz
cd nginx-1.6.2
./configure  --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_gzip_static_module
make
make install
###注意--with-http_ssl_module，后面要用ssl



        配置nginx：（主，备）

            创建nginx用户：

1
useradd -s /sbin/nologin nginx



        修改文件打开数：

1
2
ulimit -n 65535
##设置开机自动启动



        生成ssl证书（模拟）
1
2
3
4
5
6
7
8
9
10
11
12
13
##创建证书存放目录：
cd /usr/local/nginx/
mkdir ca
cd ca
##创建服务器私钥：
openssl genrsa -des3 -out server.key 1024
##创建签名请求的证书（CSR）：
openssl req -new -key server.key -out server.csr
##在加载SSL支持的Nginx并使用上述私钥时除去必须的口令
cp server.key server.key.org
openssl rsa -in server.key.org -out server.key
##最后标记证书使用上述私钥和CSR：
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt



        修改配置文件：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
# mv /usr/local/nginx/conf/nginx.conf /usr/local/nginx/conf/nginx.conf.old
# vi /usr/local/nginx/conf/nginx.conf
user nginx nginx;
worker_processes  2;
error_log  logs/error.log;
pid        logs/nginx.pid;
worker_rlimit_nofile 65535;

events {
    use epoll;
    worker_connections  65535;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" $http_x_forwarded_for';

    server_names_hash_bucket_size 128;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;

    sendfile       on;

    #tcp_nopush     on;
    tcp_nodelay    on;

    client_header_timeout 30;
    client_body_timeout 30;
    send_timeout   30;
    client_max_body_size 100M;

    keepalive_timeout  60;

    proxy_connect_timeout  100;
    proxy_send_timeout  100;
    proxy_read_timeout  100;
    proxy_buffer_size  16k;
    proxy_buffers  4 32k;
    proxy_busy_buffers_size  64k;

    gzip  on;
    gzip_min_length 1k;
    gzip_buffers 4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 2;
    gzip_types text/plain application/x-javascript text/css application/xml text/javascript;
    gzip_vary on;

    upstream web1{
      ip_hash;
      server 192.168.247.128:8080 max_fails=3 fail_timeout=30s;
      server 192.168.247.131:8080 max_fails=3 fail_timeout=30s;
    }


    server {
        listen       443;
        server_name  www.scj.com;
        root /opt/nginx/www/www.scj.com;
        index index.html index.htm index.jsp;

        ssl on;
        ssl_certificate /usr/local/nginx/ca/server.crt;
        ssl_certificate_key /usr/local/nginx/ca/server.key;

        access_log  logs/www.scj.com/access.log main;
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        location ~ .*\.jsp$ {
          proxy_pass http://web1;
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_next_upstream http_502 http_504 error timeout invalid_header;
        }

        location ~ .*\.(gif|jpg|png|swf|bmp|jpeg)$ {
            expires 10d;
        }

        location ~ .*\.(js|css)?$ {
            expires 1h;
        }
    }

    #server {
       #listen 80 default;
       #server_name _;
       #return 500;
    #}
}



1
2
3
##创建日志目录和根目录
mkdir /usr/local/nginx/logs/www.scj.com        ##日志目录
mkdir -p /opt/nginx/www/www.scj.com            ##创建根目录



1
2
3
4
5
6
7
8
9
10
11
12
13
##启动nginx：
# /usr/local/nginx/sbin/nginx

# ps -ef | grep nginx
root       7890      1  0 08:00 ?        00:00:00 nginx: master process /usr/local/nginx/sbin/nginx
nginx      7891   7890  0 08:00 ?        00:00:00 nginx: worker process      
nginx      7892   7890  0 08:00 ?        00:00:00 nginx: worker process      
root       7896   7798  0 08:00 pts/0    00:00:00 grep nginx

# netstat  -tlnpa | grep nginx
tcp        0      0 0.0.0.0:443                 0.0.0.0:*                   LISTEN      7890/nginx

##设置开机自动启动







    部署keepalived，实现自动切换：

        安装keepalived：（主，备）

1
2
3
4
5
6
7
cd /usr/local/src/
wget http://www.keepalived.org/software/keepalived-1.2.15.tar.gz
tar -zxf keepalived-1.2.15.tar.gz
cd keepalived-1.2.15
./configure --prefix=/usr/local/keepalived
make
make install



        拷贝文件：（主，备）

1
2
3
4
5
6
7
8
cp -a /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/
cp -a /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived/
cp -a /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp -a /usr/local/keepalived/sbin/keepalived /usr/sbin/

##注意： /etc/sysconfig/keepalived 和 /etc/keepalived/keepalived.conf 的路径一定要正确，
##因为在执行/etc/init.d/keepalived这个启动脚本时，会读取/etc/sysconfig/keepalived 和 /etc/keepalived/keepalived.conf 这两个文件



        修改配置文件：
            nginx主：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
[iyunv@nginxmaster ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.old
[iyunv@nginxmaster ~]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
    notification_email {
      732233048@qq.com
    }
    notification_email_from root@localhost
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id nginx_ha
}

vrrp_instance VI_1 {
    state master
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    #nopreempt

    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.247.150
    }
}

virtual_server 192.168.247.150 443 {
    delay_loop 6
    #lb_algo wrr
    #lb_kind DR
    #persistence_timeout 50
    protocol TCP
    real_server 192.168.247.129 443 {
        #weight 3
        notify_down /etc/keepalived/killkeepalived.sh
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 443
        }
    }
}




           nginx备：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
[iyunv@nginxslave ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.old
[iyunv@nginxslave ~]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
    notification_email {
      732233048@qq.com
    }
    notification_email_from root@localhost
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id nginx_ha
}

vrrp_instance VI_1 {
    state backup
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    #nopreempt

    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.247.150
    }
}

virtual_server 192.168.247.150 443 {
    delay_loop 6
    #lb_algo wrr
    #lb_kind DR
    #persistence_timeout 50
    protocol TCP
    real_server 192.168.247.130 443 {
        #weight 3
        notify_down /etc/keepalived/killkeepalived.sh
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 443
        }
    }
}



            创建/etc/keepalived/killkeepalived.sh脚本：（主，备）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
vi /etc/keepalived/killkeepalived.sh
#!/bin/bash
# check nginx server status

nginx_status=`netstat  -tlnpa | grep 443 | wc -l`

if [ $nginx_status -eq 0 ];then
  /usr/local/nginx/sbin/nginx
  sleep 1
  nginx_status=`netstat  -tlnpa | grep 443 | wc -l`
  if [ $nginx_status -eq 0 ];then
    /etc/init.d/keepalived stop
  fi
fi

chmod 755 /etc/keepalived/killkeepalived.sh




        启动keepalived：

            nginx主：

1
2
[iyunv@nginxmaster ~]# /etc/init.d/keepalived start
Starting keepalived:                                       [  OK  ]



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
##查看日志：
Sep 23 20:16:27 nginxmaster Keepalived[2909]: Starting Keepalived v1.2.15 (09/23,2015)
Sep 23 20:16:27 nginxmaster Keepalived[2910]: Starting Healthcheck child process, pid=2912
Sep 23 20:16:27 nginxmaster Keepalived[2910]: Starting VRRP child process, pid=2913
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Netlink reflector reports IP 192.168.247.129 added
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Netlink reflector reports IP fe80::20c:29ff:fee0:d236 added
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Registering Kernel netlink reflector
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Registering Kernel netlink command channel
Sep 23 20:16:27 nginxmaster Keepalived_vrrp[2913]: Registering gratuitous ARP shared channel
Sep 23 20:16:27 nginxmaster kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
Sep 23 20:16:27 nginxmaster kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
Sep 23 20:16:27 nginxmaster kernel: IPVS: ipvs loaded.
Sep 23 20:16:27 nginxmaster Keepalived_healthcheckers[2912]: Netlink reflector reports IP 192.168.247.129 added
Sep 23 20:16:27 nginxmaster Keepalived_healthcheckers[2912]: Netlink reflector reports IP fe80::20c:29ff:fee0:d236 added
Sep 23 20:16:27 nginxmaster Keepalived_healthcheckers[2912]: Registering Kernel netlink reflector
Sep 23 20:16:27 nginxmaster Keepalived_healthcheckers[2912]: Registering Kernel netlink command channel
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: Opening file '/etc/keepalived/keepalived.conf'.
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: Configuration is using : 63272 Bytes
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: Using LinkWatch kernel netlink reflector...
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep 23 20:16:28 nginxmaster Keepalived_vrrp[2913]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: Opening file '/etc/keepalived/keepalived.conf'.
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: Configuration is using : 11705 Bytes
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: IPVS: Scheduler or persistence engine not found
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: IPVS: No such process
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: Using LinkWatch kernel netlink reflector...
Sep 23 20:16:28 nginxmaster Keepalived_healthcheckers[2912]: Activating healthchecker for service [192.168.247.129]:443
Sep 23 20:16:28 nginxmaster kernel: IPVS: Scheduler module ip_vs_ not found
Sep 23 20:16:31 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 23 20:16:32 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 23 20:16:32 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 23 20:16:32 nginxmaster Keepalived_healthcheckers[2912]: Netlink reflector reports IP 192.168.247.150 added
Sep 23 20:16:32 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 23 20:16:37 nginxmaster Keepalived_vrrp[2913]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150



            nginx备：

1
2
[iyunv@nginxslave ~]# /etc/init.d/keepalived start
Starting keepalived:                                       [  OK  ]



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
##查看日志：
Sep 24 09:43:29 nginxslave Keepalived[9481]: Starting Keepalived v1.2.15 (09/24,2015)
Sep 24 09:43:29 nginxslave Keepalived[9482]: Starting Healthcheck child process, pid=9484
Sep 24 09:43:29 nginxslave Keepalived[9482]: Starting VRRP child process, pid=9485
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Netlink reflector reports IP 192.168.247.130 added
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Netlink reflector reports IP fe80::20c:29ff:fe1e:8eab added
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Registering Kernel netlink reflector
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Registering Kernel netlink command channel
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Registering gratuitous ARP shared channel
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Opening file '/etc/keepalived/keepalived.conf'.
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Configuration is using : 63262 Bytes
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: Using LinkWatch kernel netlink reflector...
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep 24 09:43:29 nginxslave Keepalived_vrrp[9485]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Netlink reflector reports IP 192.168.247.130 added
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Netlink reflector reports IP fe80::20c:29ff:fe1e:8eab added
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Registering Kernel netlink reflector
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Registering Kernel netlink command channel
Sep 24 09:43:29 nginxslave kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
Sep 24 09:43:29 nginxslave kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
Sep 24 09:43:29 nginxslave kernel: IPVS: ipvs loaded.
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Opening file '/etc/keepalived/keepalived.conf'.
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Configuration is using : 11695 Bytes
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: IPVS: Scheduler or persistence engine not found
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: IPVS: No such process
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Using LinkWatch kernel netlink reflector...
Sep 24 09:43:29 nginxslave Keepalived_healthcheckers[9484]: Activating healthchecker for service [192.168.247.130]:443
Sep 24 09:43:29 nginxslave kernel: IPVS: Scheduler module ip_vs_ not found




        查看vip绑定在哪台机器上：

1
2
3
4
5
6
7
8
9
10
11
12
[iyunv@nginxmaster ~]#ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:e0:d2:36 brd ff:ff:ff:ff:ff:ff
    inet 192.168.247.129/24 brd 192.168.247.255 scope global eth0
    inet 192.168.247.150/32 scope global eth0
    inet6 fe80::20c:29ff:fee0:d236/64 scope link
       valid_lft forever preferred_lft forever



1
2
3
4
5
6
7
8
9
10
11
[iyunv@nginxslave ~]#ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:1e:8e:ab brd ff:ff:ff:ff:ff:ff
    inet 192.168.247.130/24 brd 192.168.247.255 scope global eth0
    inet6 fe80::20c:29ff:fe1e:8eab/64 scope link
       valid_lft forever preferred_lft forever



1
##由上发现vip150绑定在nginxmaster，正常







测试：
    测试一：把nginx主的nginx服务stop掉
1
[iyunv@nginxmaster ~]# /usr/local/nginx/sbin/nginx -s stop



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
##查看日志：
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: TCP connection to [192.168.247.129]:443 failed !!!
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: Removing service [192.168.247.129]:443 from VS [192.168.247.150]:443
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: IPVS: Service not defined
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: Executing [/etc/keepalived/killkeepalived.sh] for service [192.168.247.129]:443 in VS [192.168.247.150]:443
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: Lost quorum 1-0=1 > 0 for VS [192.168.247.150]:443
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: Remote SMTP server [127.0.0.1]:25 connected.
Sep 23 23:38:00 nginxmaster Keepalived_healthcheckers[3990]: SMTP alert successfully sent.
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: TCP connection to [192.168.247.129]:443 success.
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: Adding service [192.168.247.129]:443 to VS [192.168.247.150]:443
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: IPVS: Service not defined
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: Gained quorum 1+0=1 <= 1 for VS [192.168.247.150]:443
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: Remote SMTP server [127.0.0.1]:25 connected.
Sep 23 23:38:06 nginxmaster Keepalived_healthcheckers[3990]: SMTP alert successfully sent

##keepalived会先把129移除，然后执行/etc/keepalived/killkeepalived.sh脚本，nginx服务正常启动，又把129加入到集群中



    测试二：把nginx主的keepalived服务stop掉

1
2
[iyunv@nginxmaster ~]# /etc/init.d/keepalived stop
Stopping keepalived:                                       [  OK  ]



1
2
3
4
5
6
7
8
##查看nginx主的日志：
Sep 23 23:41:05 nginxmaster Keepalived[3988]: Stopping Keepalived v1.2.15 (09/23,2015)
Sep 23 23:41:05 nginxmaster Keepalived_vrrp[3991]: VRRP_Instance(VI_1) sending 0 priority
Sep 23 23:41:05 nginxmaster Keepalived_vrrp[3991]: VRRP_Instance(VI_1) removing protocol VIPs.
Sep 23 23:41:05 nginxmaster Keepalived_healthcheckers[3990]: Netlink reflector reports IP 192.168.247.150 removed
Sep 23 23:41:05 nginxmaster Keepalived_healthcheckers[3990]: Removing service [192.168.247.129]:443 from VS [192.168.247.150]:443
Sep 23 23:41:05 nginxmaster Keepalived_healthcheckers[3990]: IPVS: Service not defined
Sep 23 23:41:05 nginxmaster Keepalived_healthcheckers[3990]: IPVS: No such service



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
##查看nginx备日志：
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: TCP connection to [192.168.247.130]:443 success.
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: Adding service [192.168.247.130]:443 to VS [192.168.247.150]:443
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: IPVS: Service not defined
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: Gained quorum 1+0=1 <= 1 for VS [192.168.247.150]:443
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: Remote SMTP server [127.0.0.1]:25 connected.
Sep 24 12:57:11 nginxslave Keepalived_healthcheckers[10012]: SMTP alert successfully sent.
Sep 24 12:58:05 nginxslave dhclient[906]: DHCPREQUEST on eth0 to 192.168.247.254 port 67 (xid=0x10a73226)
Sep 24 12:58:05 nginxslave dhclient[906]: DHCPACK from 192.168.247.254 (xid=0x10a73226)
Sep 24 12:58:07 nginxslave dhclient[906]: bound to 192.168.247.130 -- renewal in 705 seconds.
Sep 24 13:05:23 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 24 13:05:24 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 24 13:05:24 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 24 13:05:24 nginxslave Keepalived_healthcheckers[10012]: Netlink reflector reports IP 192.168.247.150 added
Sep 24 13:05:24 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 24 13:05:29 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150

##nginx备变成master，绑定vip 150



    测试三：把nginx主的nginx服务stop，并不再让它成功启动
1
2
##删掉日志目录，nginx便不能启动了
[iyunv@nginxmaster logs]# rm -rf /usr/local/nginx/logs/www.scj.com/



1
2
##关闭nginx服务
[iyunv@nginxmaster ~]# /usr/local/nginx/sbin/nginx -s stop



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
##查看nginx主日志：
Sep 23 23:44:33 nginxmaster Keepalived_vrrp[4079]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: TCP connection to [192.168.247.129]:443 failed !!!
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: Removing service [192.168.247.129]:443 from VS [192.168.247.150]:443
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: IPVS: Service not defined
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: Executing [/etc/keepalived/killkeepalived.sh] for service [192.168.247.129]:443 in VS [192.168.247.150]:443
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: Lost quorum 1-0=1 > 0 for VS [192.168.247.150]:443
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: Remote SMTP server [127.0.0.1]:25 connected.
Sep 23 23:44:38 nginxmaster Keepalived_healthcheckers[4078]: SMTP alert successfully sent.
Sep 23 23:44:39 nginxmaster Keepalived[4076]: Stopping Keepalived v1.2.15 (09/23,2015)
Sep 23 23:44:39 nginxmaster Keepalived_vrrp[4079]: VRRP_Instance(VI_1) sending 0 priority
Sep 23 23:44:39 nginxmaster Keepalived_vrrp[4079]: VRRP_Instance(VI_1) removing protocol VIPs.
Sep 23 23:44:39 nginxmaster Keepalived_healthcheckers[4078]: Netlink reflector reports IP 192.168.247.150 removed
Sep 23 23:44:39 nginxmaster Keepalived_healthcheckers[4078]: IPVS: No such service

##nginx服务不能启动了，然后把keepalived服务关掉



1
2
3
4
5
6
7
8
9
10
11
12
##查看nginx备日志
Sep 24 13:08:57 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 24 13:08:58 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 24 13:08:58 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 24 13:08:58 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 24 13:08:58 nginxslave Keepalived_healthcheckers[10012]: Netlink reflector reports IP 192.168.247.150 added
Sep 24 13:09:03 nginxslave Keepalived_vrrp[10013]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.247.150
Sep 24 13:09:52 nginxslave dhclient[906]: DHCPREQUEST on eth0 to 192.168.247.254 port 67 (xid=0x10a73226)
Sep 24 13:09:52 nginxslave dhclient[906]: DHCPACK from 192.168.247.254 (xid=0x10a73226)
Sep 24 13:09:54 nginxslave dhclient[906]: bound to 192.168.247.130 -- renewal in 800 seconds.

##nginx备变为master，并绑定vip 150



    测试四：把nginx备的nginx服务stop

1
##查看nginx备的nginx服务会不会又正常启动



    测试五：把nginx备的keepalived服务stop

1
##nginx主不会有任何变化



    测试六：把nginx备的nginx服务stop掉，并不再让它正常启动

1
2
##nginx备的keepalived服务会被关掉
##nginx主不会有任何变化




注意：
    修复后，一定要记得把keepalived服务启动起来








运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表，该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有，请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意，请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息，一经发现立即删除。若您因此触犯法律，一切后果自负，我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集，我们仅提供一个展示、介绍、观摩学习的平台，我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责，亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏，不得用于商业或者其他用途，否则，一切后果均由您自己承担，我们对此不承担任何法律责任
7、如涉及侵犯版权等问题，请您及时通知我们，我们将立即采取措施予以解决
8、联系人Email：admin@iyunv.com 网址：www.yunweiku.com

所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其承担任何法律责任,如涉及侵犯版权等问题,请您及时通知我们,我们将立即处理,联系人Email:kefu@iyunv.com,QQ:1061981298 本贴地址：https://www.yunweiku.com/thread-118582-1-1.html 上篇帖子： 更改掉nginx默认的用户和用户组 下篇帖子： Nginx配置高亮
回复

使用道具 举报

返回列表
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

扫码加入运维网微信交流群X

扫码加入运维网微信交流群

扫描二维码加入运维网微信交流群，最新一手资源尽在官方微信交流群！快快加入我们吧...

扫描微信二维码查看详情

客服 E-mail:iyunvcom@gmail.com

本站由青云提供云计算服务

运维网--中国最专业的运维工程师交流社区

豫ICP备20007574号-1 Copyright © 2012-2025

关于我们

发展历程

联系我们

关注我们

广告服务

广告案例

隐私条款

免责声明

用户登录

用户注册

版主守则

申请版主

手机版

金币获取

帮助中心

每日签到

Good Good

Study

Day Day

UP


客服E-mail:kefu@iyunv.com 客服QQ：1061981298


QQ群⑦：运维网交流群⑦ QQ群⑧：运维网交流群⑧ k8s群：运维网kubernetes交流群


提醒：禁止发布任何违反国家法律、法规的言论与图片等内容；本站内容均来自个人观点与网络等信息，非本站认同之观点.


本站大部分资源是网友从网上搜集分享而来，其版权均归原作者及其网站所有，我们尊重他人的合法权益，如有内容侵犯您的合法权益，请及时与我们联系进行核实删除！



合作伙伴: 青云cloud

快速回复 返回顶部 返回列表