Sun Aug 14 11:51:14 2016 - [warning] Got error on MySQL select ping: 2006 (MySQL server has gone away) Sun Aug 14 11:51:14 2016 - [info] Executing SSH check script: exit 0 Sun Aug 14 11:51:14 2016 - [info] HealthCheck: SSH to 192.168.1.103 is reachable. Sun Aug 14 11:51:15 2016 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111) Sun Aug 14 11:51:15 2016 - [warning] Connection failed 2 time(s).. Sun Aug 14 11:51:16 2016 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111) Sun Aug 14 11:51:16 2016 - [warning] Connection failed 3 time(s).. Sun Aug 14 11:51:17 2016 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111) Sun Aug 14 11:51:17 2016 - [warning] Connection failed 4 time(s).. Sun Aug 14 11:51:17 2016 - [warning] Master is not reachable from health checker! Sun Aug 14 11:51:17 2016 - [warning] Master 192.168.1.103(192.168.1.103:3306) is not reachable! Sun Aug 14 11:51:17 2016 - [warning] SSH is reachable. Sun Aug 14 11:51:17 2016 - [info] Connecting to a master server failed. Reading configuration file /etc/masterha_default.cnf and /etc/mha/app1.cnf again, and trying to connect to all servers to check server status.. Sun Aug 14 11:51:17 2016 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping. Sun Aug 14 11:51:17 2016 - [info] Reading application default configuration from /etc/mha/app1.cnf.. Sun Aug 14 11:51:17 2016 - [info] Reading server configuration from /etc/mha/app1.cnf.. Sun Aug 14 11:51:17 2016 - [info] GTID failover mode = 1 Sun Aug 14 11:51:17 2016 - [info] Dead Servers: Sun Aug 14 11:51:17 2016 - [info] 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:51:17 2016 - [info] Alive Servers: Sun Aug 14 11:51:17 2016 - [info] 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:51:17 2016 - [info] Alive Slaves: Sun Aug 14 11:51:17 2016 - [info] 192.168.1.104(192.168.1.104:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:51:17 2016 - [info] GTID ON Sun Aug 14 11:51:17 2016 - [info] Replicating from 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:51:17 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:51:17 2016 - [info] Checking slave configurations.. Sun Aug 14 11:51:17 2016 - [info] read_only=1 is not set on slave 192.168.1.104(192.168.1.104:3306). Sun Aug 14 11:51:17 2016 - [info] Checking replication filtering settings.. Sun Aug 14 11:51:17 2016 - [info] Replication filtering check ok. Sun Aug 14 11:51:17 2016 - [info] Master is down! Sun Aug 14 11:51:17 2016 - [info] Terminating monitoring script. Sun Aug 14 11:51:17 2016 - [info] Got exit code 20 (Master dead). Sun Aug 14 11:51:17 2016 - [info] MHA::MasterFailover version 0.56. Sun Aug 14 11:51:17 2016 - [info] Starting master failover. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] * Phase 1: Configuration Check Phase.. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] GTID failover mode = 1 Sun Aug 14 11:51:17 2016 - [info] Dead Servers: Sun Aug 14 11:51:17 2016 - [info] 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:51:17 2016 - [info] Checking master reachability via MySQL(double check)... Sun Aug 14 11:51:17 2016 - [info] ok. Sun Aug 14 11:51:17 2016 - [info] Alive Servers: Sun Aug 14 11:51:17 2016 - [info] 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:51:17 2016 - [info] Alive Slaves: Sun Aug 14 11:51:17 2016 - [info] 192.168.1.104(192.168.1.104:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:51:17 2016 - [info] GTID ON Sun Aug 14 11:51:17 2016 - [info] Replicating from 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:51:17 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:51:17 2016 - [info] Starting GTID based failover. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] ** Phase 1: Configuration Check Phase completed. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] * Phase 2: Dead Master Shutdown Phase.. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] Forcing shutdown so that applications never connect to the current master.. Sun Aug 14 11:51:17 2016 - [info] Executing master IP deactivation script: Sun Aug 14 11:51:17 2016 - [info] /etc/mha/master_ip_failover --orig_master_host=192.168.1.103 --orig_master_ip=192.168.1.103 --orig_master_port=3306 --command=stopssh --ssh_user=root
Disabling the VIP on old master if the server is still UP: 192.168.1.103 Sun Aug 14 11:51:17 2016 - [info] done. Sun Aug 14 11:51:17 2016 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master. Sun Aug 14 11:51:17 2016 - [info] * Phase 2: Dead Master Shutdown Phase completed. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] * Phase 3: Master Recovery Phase.. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] * Phase 3.1: Getting Latest Slaves Phase.. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] The latest binary log file/position on all slaves is bin.000014:231
Sun Aug 14 11:51:17 2016 - [info] Latest slaves (Slaves that received> Sun Aug 14 11:51:17 2016 - [info] 192.168.1.104(192.168.1.104:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:51:17 2016 - [info] GTID ON Sun Aug 14 11:51:17 2016 - [info] Replicating from 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:51:17 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:51:17 2016 - [info] The oldest binary log file/position on all slaves is bin.000014:231 Sun Aug 14 11:51:17 2016 - [info] Oldest slaves: Sun Aug 14 11:51:17 2016 - [info] 192.168.1.104(192.168.1.104:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:51:17 2016 - [info] GTID ON Sun Aug 14 11:51:17 2016 - [info] Replicating from 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:51:17 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] * Phase 3.3: Determining New Master Phase.. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] Searching new master from slaves.. Sun Aug 14 11:51:17 2016 - [info] Candidate masters from the configuration file: Sun Aug 14 11:51:17 2016 - [info] 192.168.1.104(192.168.1.104:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:51:17 2016 - [info] GTID ON Sun Aug 14 11:51:17 2016 - [info] Replicating from 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:51:17 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:51:17 2016 - [info] Non-candidate masters:
Sun Aug 14 11:51:17 2016 - [info] Searching from candidate_master slaves which have received the latest> Sun Aug 14 11:51:17 2016 - [info] New master is 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:51:17 2016 - [info] Starting master failover.. Sun Aug 14 11:51:17 2016 - [info] From: 192.168.1.103(192.168.1.103:3306) (current master) +--192.168.1.104(192.168.1.104:3306)
To: 192.168.1.104(192.168.1.104:3306) (new master) Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] * Phase 3.3: New Master Recovery Phase.. Sun Aug 14 11:51:17 2016 - [info] Sun Aug 14 11:51:17 2016 - [info] Waiting all logs to be applied.. Sun Aug 14 11:51:17 2016 - [info] done. Sun Aug 14 11:51:17 2016 - [info] Getting new master's binlog name and position.. Sun Aug 14 11:51:17 2016 - [info] bin.000015:231 Sun Aug 14 11:51:17 2016 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.1.104', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='xxx'; Sun Aug 14 11:51:17 2016 - [info] Master Recovery succeeded. File:Pos:Exec_Gtid_Set: bin.000015, 231, 1683955a-6102-11e6-8b6f-080027ca1592:1-8, 1c3a7f53-6102-11e6-8b6f-08002722cc6d:1-10 Sun Aug 14 11:51:17 2016 - [info] Executing master IP activate script: Sun Aug 14 11:51:17 2016 - [info] /etc/mha/master_ip_failover --command=start --ssh_user=root --orig_master_host=192.168.1.103 --orig_master_ip=192.168.1.103 --orig_master_port=3306 --new_master_host=192.168.1.104 --new_master_ip=192.168.1.104 --new_master_port=3306 --new_master_user='root' --new_master_password='111111' IN SCRIPT TEST====/sbin/ifconfig eth1:1 down==/sbin/ifconfig eth1:1 192.168.1.108;/sbin/arping -I eth1 -c 3 -s 192.168.1.108 192.168.1.1 >/dev/null 2>&1=== Enabling the VIP - 192.168.1.108 on the new master - 192.168.1.104 Sun Aug 14 11:51:20 2016 - [info] OK. Sun Aug 14 11:51:20 2016 - [info] ** Finished master recovery successfully. Sun Aug 14 11:51:20 2016 - [info] * Phase 3: Master Recovery Phase completed. Sun Aug 14 11:51:20 2016 - [info] Sun Aug 14 11:51:20 2016 - [info] * Phase 4: Slaves Recovery Phase.. Sun Aug 14 11:51:20 2016 - [info] Sun Aug 14 11:51:20 2016 - [info] Sun Aug 14 11:51:20 2016 - [info] * Phase 4.1: Starting Slaves in parallel.. Sun Aug 14 11:51:20 2016 - [info] Sun Aug 14 11:51:20 2016 - [info] All new slave servers recovered successfully. Sun Aug 14 11:51:20 2016 - [info] Sun Aug 14 11:51:20 2016 - [info] * Phase 5: New master cleanup phase.. Sun Aug 14 11:51:20 2016 - [info] Sun Aug 14 11:51:20 2016 - [info] Resetting slave info on the new master.. Sun Aug 14 11:51:20 2016 - [info] 192.168.1.104: Resetting slave info succeeded. Sun Aug 14 11:51:20 2016 - [info] Master failover to 192.168.1.104(192.168.1.104:3306) completed successfully. Sun Aug 14 11:51:20 2016 - [info]
----- Failover Report -----
app1: MySQL Master failover 192.168.1.103(192.168.1.103:3306) to 192.168.1.104(192.168.1.104:3306) succeeded
Master 192.168.1.103(192.168.1.103:3306) is down!
Check MHA Manager logs at lab1:/etc/mha/manager.log for details.
Started automated(non-interactive) failover. Invalidated master IP address on 192.168.1.103(192.168.1.103:3306) Selected 192.168.1.104(192.168.1.104:3306) as a new master. 192.168.1.104(192.168.1.104:3306): OK: Applying all logs succeeded. 192.168.1.104(192.168.1.104:3306): OK: Activated master IP address. 192.168.1.104(192.168.1.104:3306): Resetting slave info succeeded. Master failover to 192.168.1.104(192.168.1.104:3306) completed successfully. 从日志中可以看到，master切换至192.168.1.104 slave上，并且VIP也随之绑定到192.168.1.104 对应的网卡设备上。 2.主库服务器当机 （1）把192.168.1.03与192.168.1.104复制关系重新建立： change master to master_host='192.168.1.104',master_user='repl',master_password='111111',master_auto_position=1; start slave; (2)重新启动MHA： nohup masterha_manager --conf=/etc/mha/app1.cnf < /dev/null >/etc/mha/app1.log 2>&1 & （3）直接关闭192.168.1.104服务器，查看切换情况： Shutdown –h now 对应MHA manager日志：
Sun Aug 14 11:59:09 2016 - [info] MHA::MasterMonitor version 0.56. Sun Aug 14 11:59:09 2016 - [info] GTID failover mode = 1 Sun Aug 14 11:59:09 2016 - [info] Dead Servers: Sun Aug 14 11:59:09 2016 - [info] Alive Servers: Sun Aug 14 11:59:09 2016 - [info] 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:09 2016 - [info] 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:59:09 2016 - [info] Alive Slaves: Sun Aug 14 11:59:09 2016 - [info] 192.168.1.103(192.168.1.103:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:59:09 2016 - [info] GTID ON Sun Aug 14 11:59:09 2016 - [info] Replicating from 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:09 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:59:09 2016 - [info] Current Alive Master: 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:09 2016 - [info] Checking slave configurations.. Sun Aug 14 11:59:09 2016 - [info] read_only=1 is not set on slave 192.168.1.103(192.168.1.103:3306). Sun Aug 14 11:59:09 2016 - [info] Checking replication filtering settings.. Sun Aug 14 11:59:09 2016 - [info] binlog_do_db= , binlog_ignore_db= Sun Aug 14 11:59:09 2016 - [info] Replication filtering check ok. Sun Aug 14 11:59:09 2016 - [info] GTID (with auto-pos) is supported. Skipping all SSH and Node package checking. Sun Aug 14 11:59:09 2016 - [info] Checking SSH publickey authentication settings on the current master.. Sun Aug 14 11:59:09 2016 - [info] HealthCheck: SSH to 192.168.1.104 is reachable. Sun Aug 14 11:59:09 2016 - [info] 192.168.1.104(192.168.1.104:3306) (current master) +--192.168.1.103(192.168.1.103:3306)
Sun Aug 14 11:59:09 2016 - [info] Checking master_ip_failover_script status: Sun Aug 14 11:59:09 2016 - [info] /etc/mha/master_ip_failover --command=status --ssh_user=root --orig_master_host=192.168.1.104 --orig_master_ip=192.168.1.104 --orig_master_port=3306
Checking the Status of the script.. OK Sun Aug 14 11:59:12 2016 - [info] OK. Sun Aug 14 11:59:12 2016 - [warning] shutdown_script is not defined. Sun Aug 14 11:59:12 2016 - [info] Set master ping interval 1 seconds. Sun Aug 14 11:59:12 2016 - [warning] secondary_check_script is not defined. It is highly recommended setting it to check master reachability from two or more routes. Sun Aug 14 11:59:12 2016 - [info] Starting ping health check on 192.168.1.104(192.168.1.104:3306).. Sun Aug 14 11:59:12 2016 - [info] Ping(SELECT) succeeded, waiting until MySQL doesn't respond.. Sun Aug 14 11:59:23 2016 - [warning] Got error on MySQL select ping: 2006 (MySQL server has gone away) Sun Aug 14 11:59:23 2016 - [info] Executing SSH check script: exit 0 Sun Aug 14 11:59:23 2016 - [warning] HealthCheck: SSH to 192.168.1.104 is NOT reachable. Sun Aug 14 11:59:24 2016 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111) Sun Aug 14 11:59:24 2016 - [warning] Connection failed 2 time(s).. Sun Aug 14 11:59:25 2016 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111) Sun Aug 14 11:59:25 2016 - [warning] Connection failed 3 time(s).. Sun Aug 14 11:59:26 2016 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111) Sun Aug 14 11:59:26 2016 - [warning] Connection failed 4 time(s).. Sun Aug 14 11:59:26 2016 - [warning] Master is not reachable from health checker! Sun Aug 14 11:59:26 2016 - [warning] Master 192.168.1.104(192.168.1.104:3306) is not reachable! Sun Aug 14 11:59:26 2016 - [warning] SSH is NOT reachable. Sun Aug 14 11:59:26 2016 - [info] Connecting to a master server failed. Reading configuration file /etc/masterha_default.cnf and /etc/mha/app1.cnf again, and trying to connect to all servers to check server status.. Sun Aug 14 11:59:26 2016 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping. Sun Aug 14 11:59:26 2016 - [info] Reading application default configuration from /etc/mha/app1.cnf.. Sun Aug 14 11:59:26 2016 - [info] Reading server configuration from /etc/mha/app1.cnf.. Sun Aug 14 11:59:27 2016 - [info] GTID failover mode = 1 Sun Aug 14 11:59:27 2016 - [info] Dead Servers: Sun Aug 14 11:59:27 2016 - [info] 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:27 2016 - [info] Alive Servers: Sun Aug 14 11:59:27 2016 - [info] 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:59:27 2016 - [info] Alive Slaves: Sun Aug 14 11:59:27 2016 - [info] 192.168.1.103(192.168.1.103:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:59:27 2016 - [info] GTID ON Sun Aug 14 11:59:27 2016 - [info] Replicating from 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:27 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:59:27 2016 - [info] Checking slave configurations.. Sun Aug 14 11:59:27 2016 - [info] read_only=1 is not set on slave 192.168.1.103(192.168.1.103:3306). Sun Aug 14 11:59:27 2016 - [info] Checking replication filtering settings.. Sun Aug 14 11:59:27 2016 - [info] Replication filtering check ok. Sun Aug 14 11:59:27 2016 - [info] Master is down! Sun Aug 14 11:59:27 2016 - [info] Terminating monitoring script. Sun Aug 14 11:59:27 2016 - [info] Got exit code 20 (Master dead). Sun Aug 14 11:59:27 2016 - [info] MHA::MasterFailover version 0.56. Sun Aug 14 11:59:27 2016 - [info] Starting master failover. Sun Aug 14 11:59:27 2016 - [info] Sun Aug 14 11:59:27 2016 - [info] * Phase 1: Configuration Check Phase.. Sun Aug 14 11:59:27 2016 - [info] Sun Aug 14 11:59:27 2016 - [info] GTID failover mode = 1 Sun Aug 14 11:59:27 2016 - [info] Dead Servers: Sun Aug 14 11:59:27 2016 - [info] 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:27 2016 - [info] Checking master reachability via MySQL(double check)... Sun Aug 14 11:59:28 2016 - [info] ok. Sun Aug 14 11:59:28 2016 - [info] Alive Servers: Sun Aug 14 11:59:28 2016 - [info] 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:59:28 2016 - [info] Alive Slaves: Sun Aug 14 11:59:28 2016 - [info] 192.168.1.103(192.168.1.103:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:59:28 2016 - [info] GTID ON Sun Aug 14 11:59:28 2016 - [info] Replicating from 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:28 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:59:28 2016 - [info] Starting GTID based failover. Sun Aug 14 11:59:28 2016 - [info] Sun Aug 14 11:59:28 2016 - [info] ** Phase 1: Configuration Check Phase completed. Sun Aug 14 11:59:28 2016 - [info] Sun Aug 14 11:59:28 2016 - [info] * Phase 2: Dead Master Shutdown Phase.. Sun Aug 14 11:59:28 2016 - [info] Sun Aug 14 11:59:28 2016 - [info] Forcing shutdown so that applications never connect to the current master.. Sun Aug 14 11:59:28 2016 - [info] Executing master IP deactivation script: Sun Aug 14 11:59:28 2016 - [info] /etc/mha/master_ip_failover --orig_master_host=192.168.1.104 --orig_master_ip=192.168.1.104 --orig_master_port=3306 --command=stop
Disabling the VIP on old master if the server is still UP: 192.168.1.104 Sun Aug 14 11:59:34 2016 - [info] done. Sun Aug 14 11:59:34 2016 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master. Sun Aug 14 11:59:34 2016 - [info] * Phase 2: Dead Master Shutdown Phase completed. Sun Aug 14 11:59:34 2016 - [info] Sun Aug 14 11:59:34 2016 - [info] * Phase 3: Master Recovery Phase.. Sun Aug 14 11:59:34 2016 - [info] Sun Aug 14 11:59:34 2016 - [info] * Phase 3.1: Getting Latest Slaves Phase.. Sun Aug 14 11:59:34 2016 - [info] Sun Aug 14 11:59:34 2016 - [info] The latest binary log file/position on all slaves is bin.000015:231
Sun Aug 14 11:59:34 2016 - [info] Latest slaves (Slaves that received> Sun Aug 14 11:59:34 2016 - [info] 192.168.1.103(192.168.1.103:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:59:34 2016 - [info] GTID ON Sun Aug 14 11:59:34 2016 - [info] Replicating from 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:34 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:59:34 2016 - [info] The oldest binary log file/position on all slaves is bin.000015:231 Sun Aug 14 11:59:34 2016 - [info] Oldest slaves: Sun Aug 14 11:59:34 2016 - [info] 192.168.1.103(192.168.1.103:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:59:34 2016 - [info] GTID ON Sun Aug 14 11:59:34 2016 - [info] Replicating from 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:34 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:59:34 2016 - [info] Sun Aug 14 11:59:34 2016 - [info] * Phase 3.3: Determining New Master Phase.. Sun Aug 14 11:59:34 2016 - [info] Sun Aug 14 11:59:34 2016 - [info] Searching new master from slaves.. Sun Aug 14 11:59:34 2016 - [info] Candidate masters from the configuration file: Sun Aug 14 11:59:34 2016 - [info] 192.168.1.103(192.168.1.103:3306) Version=5.6.30-enterprise-commercial-advanced-log (oldest major version between slaves) log-bin:enabled Sun Aug 14 11:59:34 2016 - [info] GTID ON Sun Aug 14 11:59:34 2016 - [info] Replicating from 192.168.1.104(192.168.1.104:3306) Sun Aug 14 11:59:34 2016 - [info] Primary candidate for the new Master (candidate_master is set) Sun Aug 14 11:59:34 2016 - [info] Non-candidate masters:
Sun Aug 14 11:59:34 2016 - [info] Searching from candidate_master slaves which have received the latest> Sun Aug 14 11:59:34 2016 - [info] New master is 192.168.1.103(192.168.1.103:3306) Sun Aug 14 11:59:34 2016 - [info] Starting master failover.. Sun Aug 14 11:59:34 2016 - [info] From: 192.168.1.104(192.168.1.104:3306) (current master) +--192.168.1.103(192.168.1.103:3306)
To: 192.168.1.103(192.168.1.103:3306) (new master) Sun Aug 14 11:59:34 2016 - [info] Sun Aug 14 11:59:34 2016 - [info] * Phase 3.3: New Master Recovery Phase.. Sun Aug 14 11:59:34 2016 - [info] Sun Aug 14 11:59:34 2016 - [info] Waiting all logs to be applied.. Sun Aug 14 11:59:34 2016 - [info] done. Sun Aug 14 11:59:34 2016 - [info] Getting new master's binlog name and position.. Sun Aug 14 11:59:34 2016 - [info] bin.000015:231 Sun Aug 14 11:59:34 2016 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.1.103', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='xxx'; Sun Aug 14 11:59:34 2016 - [info] Master Recovery succeeded. File:Pos:Exec_Gtid_Set: bin.000015, 231, 1683955a-6102-11e6-8b6f-080027ca1592:1-8, 1c3a7f53-6102-11e6-8b6f-08002722cc6d:1-10 Sun Aug 14 11:59:34 2016 - [info] Executing master IP activate script: Sun Aug 14 11:59:34 2016 - [info] /etc/mha/master_ip_failover --command=start --ssh_user=root --orig_master_host=192.168.1.104 --orig_master_ip=192.168.1.104 --orig_master_port=3306 --new_master_host=192.168.1.103 --new_master_ip=192.168.1.103 --new_master_port=3306 --new_master_user='root' --new_master_password='111111'
Enabling the VIP - 192.168.1.108 on the new master - 192.168.1.103 Sun Aug 14 11:59:37 2016 - [info] OK. Sun Aug 14 11:59:37 2016 - [info] ** Finished master recovery successfully. Sun Aug 14 11:59:37 2016 - [info] * Phase 3: Master Recovery Phase completed. Sun Aug 14 11:59:37 2016 - [info] Sun Aug 14 11:59:37 2016 - [info] * Phase 4: Slaves Recovery Phase.. Sun Aug 14 11:59:37 2016 - [info] Sun Aug 14 11:59:37 2016 - [info] Sun Aug 14 11:59:37 2016 - [info] * Phase 4.1: Starting Slaves in parallel.. Sun Aug 14 11:59:37 2016 - [info] Sun Aug 14 11:59:37 2016 - [info] All new slave servers recovered successfully. Sun Aug 14 11:59:37 2016 - [info] Sun Aug 14 11:59:37 2016 - [info] * Phase 5: New master cleanup phase.. Sun Aug 14 11:59:37 2016 - [info] Sun Aug 14 11:59:37 2016 - [info] Resetting slave info on the new master.. Sun Aug 14 11:59:37 2016 - [info] 192.168.1.103: Resetting slave info succeeded. Sun Aug 14 11:59:37 2016 - [info] Master failover to 192.168.1.103(192.168.1.103:3306) completed successfully. Sun Aug 14 11:59:37 2016 - [info]
----- Failover Report -----
app1: MySQL Master failover 192.168.1.104(192.168.1.104:3306) to 192.168.1.103(192.168.1.103:3306) succeeded
Master 192.168.1.104(192.168.1.104:3306) is down! Check MHA Manager logs at lab1:/etc/mha/manager.log for details. Started automated(non-interactive) failover. Invalidated master IP address on 192.168.1.104(192.168.1.104:3306) Selected 192.168.1.103(192.168.1.103:3306) as a new master. 192.168.1.103(192.168.1.103:3306): OK: Applying all logs succeeded. 192.168.1.103(192.168.1.103:3306): OK: Activated master IP address. 192.168.1.103(192.168.1.103:3306): Resetting slave info succeeded. Master failover to 192.168.1.103(192.168.1.103:3306) completed successfully.
QQ群⑧：
窥视卡
雷达卡
发表于 2018-10-4 07:36:34
提升卡
置顶卡
沉默卡
喧嚣卡
变色卡
千斤顶
显身卡