mysql 主从复制以及binlog 测试

###mysql查看binlog日志内容

 

https://blog.csdn.net/nuli888/article/details/52106910

mysql的binlog日志位置可经过show variables like '%datadir%';查看，直接打开没法查看，要看其内容2个办法：


一、登陆到mysql查看binlog
只查看第一个binlog文件的内容
mysql> show binlog events;


查看指定binlog文件的内容
mysql> show binlog events in 'mysql-bin.000002';

mysql> show binlog events in 'mysql-bin.000001';
+------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+
| Log_name         | Pos  | Event_type  | Server_id | End_log_pos | Info                                                      |
+------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+
| mysql-bin.000001 |    4 | Format_desc |       195 |         106 | Server ver: 5.1.73-log, Binlog ver: 4                     |
| mysql-bin.000001 |  106 | Query       |       195 |         198 | use `hadoop`; delete from user where id=3                 |
| mysql-bin.000001 |  198 | Intvar      |       195 |         226 | INSERT_ID=4                                               |
| mysql-bin.000001 |  226 | Query       |       195 |         332 | use `hadoop`; INSERT INTO user (id,name)VALUES (NULL,1)   |
| mysql-bin.000001 |  332 | Query       |       195 |         424 | use `hadoop`; delete from user where id=3                 |
| mysql-bin.000001 |  424 | Intvar      |       195 |         452 | INSERT_ID=5                                               |
| mysql-bin.000001 |  452 | Query       |       195 |         560 | use `hadoop`; INSERT INTO user (id,name)VALUES (NULL,222) |
| mysql-bin.000001 |  560 | Query       |       195 |         660 | use `hadoop`; DELETE FROM `user` WHERE (`id`='1')         |
| mysql-bin.000001 |  660 | Intvar      |       195 |         688 | INSERT_ID=6                                               |
| mysql-bin.000001 |  688 | Query       |       195 |         795 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 |  795 | Intvar      |       195 |         823 | INSERT_ID=7                                               |
| mysql-bin.000001 |  823 | Query       |       195 |         930 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 |  930 | Intvar      |       195 |         958 | INSERT_ID=8                                               |
| mysql-bin.000001 |  958 | Query       |       195 |        1065 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 | 1065 | Intvar      |       195 |        1093 | INSERT_ID=9                                               |
| mysql-bin.000001 | 1093 | Query       |       195 |        1200 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 | 1200 | Query       |       195 |        1300 | use `hadoop`; DELETE FROM `user` WHERE (`id`='9')         |
| mysql-bin.000001 | 1300 | Query       |       195 |        1400 | use `hadoop`; DELETE FROM `user` WHERE (`id`='8')         |
| mysql-bin.000001 | 1400 | Query       |       195 |        1500 | use `hadoop`; DELETE FROM `user` WHERE (`id`='7')         |
| mysql-bin.000001 | 1500 | Query       |       195 |        1600 | use `hadoop`; DELETE FROM `user` WHERE (`id`='4')         |
| mysql-bin.000001 | 1600 | Query       |       195 |        1700 | use `hadoop`; DELETE FROM `user` WHERE (`id`='5')         |
| mysql-bin.000001 | 1700 | Query       |       195 |        1800 | use `hadoop`; DELETE FROM `user` WHERE (`id`='6')         |
| mysql-bin.000001 | 1800 | Intvar      |       195 |        1828 | INSERT_ID=10                                              |
| mysql-bin.000001 | 1828 | Query       |       195 |        1935 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('555')  |
| mysql-bin.000001 | 1935 | Intvar      |       195 |        1963 | INSERT_ID=11                                              |
| mysql-bin.000001 | 1963 | Query       |       195 |        2070 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('666')  |
| mysql-bin.000001 | 2070 | Intvar      |       195 |        2098 | INSERT_ID=12                                              |
| mysql-bin.000001 | 2098 | Query       |       195 |        2205 | use `hadoop`; INSERT INTO `user` (`name`) VALUES ('777')  |
+------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+

查看当前正在写入的binlog文件
mysql> show master status\G

mysql> show master status\G
*************************** 1. row ***************************
            File: mysql-bin.000002
        Position: 106
    Binlog_Do_DB:
Binlog_Ignore_DB: mysql,information_schema,performance_schema
1 row in set (0.00 sec)

获取binlog文件列表
mysql> show binary logs;

mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |      3548 |
| mysql-bin.000002 |       106 |
+------------------+-----------+
2 rows in set (0.00 sec)

二、用mysqlbinlog工具查看
基于开始/结束时间
[root@hd3 ~]# mysqlbinlog --start-datetime='2016-08-02 00:00:00' --stop-datetime='2016-08-03 23:01:01' -d hadoop /var/lib/mysql/mysql-bin.000001


基于pos值，注：hadoop是库名，/var/lib/mysql/mysql-bin.000001是二进制文件路径
[root@hd3 ~]# mysqlbinlog --start-position=2098 --stop-position=2205 -d hadoop /var/lib/mysql/mysql-bin.000001

/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#160803 17:49:51 server id 195  end_log_pos 106         Start: binlog v 4, server v 5.1.73-log created 160803 17:49:51 at startup
# Warning: this binlog is either in use or was not closed properly.
ROLLBACK/*!*/;
BINLOG '
P76hVw/DAAAAZgAAAGoAAAABAAQANS4xLjczLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA/vqFXEzgNAAgAEgAEBAQEEgAAUwAEGggAAAAICAgC
'/*!*/;
# at 2098
#160803 18:53:56 server id 195  end_log_pos 2205        Query   thread_id=1481  exec_time=115   error_code=0
use `hadoop`/*!*/;
SET TIMESTAMP=1470221636/*!*/;
SET @@session.pseudo_thread_id=1481/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=1, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=0/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
INSERT INTO `user` (`name`) VALUES ('777')
/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;

 

 

#############12

 

查看是否开启
mysql> show variables like 'log_bin%';


二、5.7及之后的版本
在5.6的版本状况下须要多添加一个参数
server-id=123456        #123456是惟一的值就好

 

############sample 主从复制准备  5.6 版本以前 老的方式 

配置主数据库：
my.cnf：
server-id       = 1
log-bin 
重启数据库
登陆并查看：
[root@Mysql-server ~]# mysql -uroot -p199429
mysql> show variables like ‘log_bin‘;  
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_bin       | ON    |
+---------------+-------+
mysql> show variables like ‘server_id‘;
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 1     |
+---------------+-------+
创建主从复制帐号：
mysql> grant replication slave on *.* to ‘rep‘@‘192.1.1.%‘ identified by ‘199429‘;
mysql> select user,host from mysql.user;
+-----------+-----------+
| user      | host      |
+-----------+-----------+
| root      | 127.0.0.1 |
| bbs       | 192.1.1.% |
| keer      | 192.1.1.% |
| rep       | 192.1.1.% |
| wordpress | 192.1.1.% |
| root      | localhost |
+-----------+-----------+
实现对主数据库锁表只读：
mysql> flush table with read lock;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables like ‘%timeout%‘;
+----------------------------+----------+
| Variable_name              | Value    |
+----------------------------+----------+
| connect_timeout            | 10       |
| delayed_insert_timeout     | 300      |
| innodb_lock_wait_timeout   | 50       |
| innodb_rollback_on_timeout | OFF      |
| interactive_timeout        | 28800    |#####
| lock_wait_timeout          | 31536000 |
| net_read_timeout           | 30       |
| net_write_timeout          | 60       |
| slave_net_timeout          | 3600     |
| wait_timeout               | 28800    |#####
+----------------------------+----------+
10 rows in set (0.00 sec)
查看主库状态：
mysql> show master status;
+-------------------------+----------+--------------+------------------+
| File                    | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------------+----------+--------------+------------------+
| Mysql-server-bin.000001 |      962 |              |                  |
+-------------------------+----------+--------------+------------------+
1 row in set (0.00 sec)
新开窗口备份导出数据：
[root@Mysql-server ~]# mkdir -p /server/backup/ 
[root@Mysql-server ~]# mysqldump -uroot -p199429 --events -A -B |gzip >/server/backup/mysql_bak.$(date +%F).sql.gz
在此查看主库状态是否变化：
mysql> show master status;
+-------------------------+----------+--------------+------------------+
| File                    | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------------+----------+--------------+------------------+
| Mysql-server-bin.000001 |      962 |              |                  |
+-------------------------+----------+--------------+------------------+
1 row in set (0.00 sec)



从数据库：
配置文件my.cnf
server-id       = 2  ####保证惟一性
重启从数据库
登陆从数据库：
[root@Mysql-server_02 backup]# mysql -uroot -p199429 -S /data/3306/mysql.sock 
mysql> show variables like ‘log_bin‘;
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_bin       | OFF   |
+---------------+-------+
1 row in set (0.00 sec)
mysql> show variables like ‘server_id‘;
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id     | 3     |
+---------------+-------+
恢复MySQLdump的数据
[root@Mysql-server_02 backup]# cd /server/backup/
[root@Mysql-server_02 backup]# gzip -d mysql_bak.2017-03-23.sql.gz
[root@Mysql-server_02 backup]# mysql -uroot -p‘199429‘ -S /data/3306/mysql.sock <mysql_bak.2017-03-23.sql 

登陆数据库配置复制参数
[root@Mysql-server_02 backup]# mysql -uroot -p‘199429‘ -S /data/3306/mysql.sock
mysql> CHANGE MASTER TO
    -> MASTER_HOST=‘192.1.1.11‘,
    -> MASTER_PORT=3306,
    -> MASTER_USER=‘rep‘,
    -> MASTER_PASSWORD=‘199429‘,
    -> MASTER_LOG_FILE=‘Mysql-server-bin.000001‘,
    -> MASTER_LOG_POS=962;
费登陆状态执行方法：
[root@Mysql-server_02 backup]# mysql -uroot -p‘199429‘ -S /data/3306/mysql.sock<< EOF
CHANGE MASTER TO
    MASTER_HOST=‘192.1.1.11‘,
    MASTER_PORT=3306,
    MASTER_USER=‘rep‘,
    MASTER_PASSWORD=‘199429‘,
    MASTER_LOG_FILE=‘Mysql-server-bin.000001‘,
    MASTER_LOG_POS=962;
EOF
实际修改从库中的master.info文件
[root@Mysql-server_02 backup]# cat /data/3306/data/master.info 
18
mysql-server-bin.000001
962
192.1.1.11
rep
199429
3306
60
0
，。。。。。。。。

启动主从复制：
mysql> start slave;                                               
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.1.1.11
                  Master_User: rep
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: Mysql-server-bin.000001
          Read_Master_Log_Pos: 962
               Relay_Log_File: relay-bin.000002
                Relay_Log_Pos: 260
        Relay_Master_Log_File: Mysql-server-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 962
              Relay_Log_Space: 410
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
1 row in set (0.00 sec)


测试主从复制功能（省略）

 

 

 

#############sampe 2  测试经过   5.6 版本以前 老的方式 

 

https://www.linuxidc.com/Linux/2016-08/134669.htm

mysql主从复制
mysql支持单向 双向 链式级联 实时 异步复制，在复制过程当中，一台服务器充当主服务器（Master）,而一个或多个其余服务器充当从服务器(Slave)

mysql主从复制的应用场景
一、主从服务器互为备份
二、主从服务器读写分离分担网站压力

读写分离
中大型公司：经过程序(php,java)
测试环境：代理软件(mysql-proxy,amoeba)
门户网站：分布式dbproxy（读写分离，hash负载均衡，健康检查）
主从同步实践操做（多实例环境） 
  
一、主库上面设置server-id值并开启binlog参数 
[root@CentOS03 ~]# egrep "log-bin|server-id" /data/3306/my.cnf  
log-bin = /data/3306/mysql-bin
server-id = 1 
  
  
检查实际配置效果 
[root@centos03 ~]# mysql -uroot -p123456 -S /data/3306/mysql.sock -e "show variables like 'log_bin';"  
+---------------+-------+ 
| Variable_name | Value | 
+---------------+-------+ 
| log_bin      | ON    | 
+---------------+-------+ 
  
二、创建用于同步的帐号 
mysql>  grant replication slave  on *.* to rep@'172.16.80.%' identified by '123456'; 
说明：replication slave 是mysql同步的必须权限，此处不要受权all 
  
mysql> flush privileges; 
  
查看受权后的结果 
mysql> show grants for rep@'172.16.80.%'; 
+--------------------------------------------------------------------------------------------------------------------------+ 
| Grants for rep@172.16.80.%                                                                                              | 
+--------------------------------------------------------------------------------------------------------------------------+ 
| GRANT REPLICATION SLAVE ON *.* TO 'rep'@'172.16.80.%' IDENTIFIED BY PASSWORD '*6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9' | 
+--------------------------------------------------------------------------------------------------------------------------+ 
1 row in set (0.00 sec) 
  
  
三、锁表，导出数据库 
mysql> flush table with read lock;    #该窗口不能断，新开一个窗口作数据库导出操做 
Query OK, 0 rows affected (0.00 sec) 
  
mysql> show master status; 
+------------------+----------+--------------+------------------+ 
| File            | Position | Binlog_Do_DB | Binlog_Ignore_DB | 
+------------------+----------+--------------+------------------+ 
| mysql-bin.000002 |      332 |              |                  | 
+------------------+----------+--------------+------------------+ 
1 row in set (0.00 sec) 
  
[root@centos03 ~]# mysqldump -uroot -p123456 -S /data/3306/mysql.sock -A -B --events --master-data=2 > /opt/rep.sql #导出全部数据库 
[root@centos03 ~]# vim /opt/rep.sql 
-- CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000002', MASTER_LOG_POS=332;  #能够看到该语句的记录位置和上面show master status是同样的，注释状态 
  
  
四、数据库导出后，解锁 
mysql> show master status;  #再次查看位置点，以验证上面的锁表操做是否有效 
+------------------+----------+--------------+------------------+ 
| File            | Position | Binlog_Do_DB | Binlog_Ignore_DB | 
+------------------+----------+--------------+------------------+ 
| mysql-bin.000002 |      332 |              |                  | 
+------------------+----------+--------------+------------------+ 
1 row in set (0.00 sec) 
  
mysql> unlock tables; 
Query OK, 0 rows affected (0.00 sec) 
  
  
五、从库上面 
[root@centos03 ~]# egrep "log-bin|server-id" /data/3307/my.cnf      
#log-bin = /data/3307/mysql-bin    #log-bin无需开启 
server-id = 3  #server-id的值不能和主库上面的值同样 
  
[root@centos03 ~]# mysql -uroot -p123456 -S /data/3307/mysql.sock < /opt/rep.sql  #导入从主库备份的数据库 
  
[root@centos03 ~]# mysql -uroot -phello123 -S /data/3307/mysql.sock              
mysql> change master to \ 
    -> master_host='172.16.80.118',\ 
    -> master_user='rep',\ 
    -> master_password='123456',\ 
    -> master_log_file='mysql-bin.000002',\ 
    -> master_log_pos=332; 
Query OK, 0 rows affected (0.03 sec) 
  
验证一下 
[root@centos03 ~]# cat /data/3307/data/master.info  
18 
mysql-bin.000002 
332 
172.16.80.118 
rep 
123456 
3306 
60 
0 
  
mysql> start slave;      
Query OK, 0 rows affected (0.01 sec) 
  
mysql> show slave status\G;    #观察Slave_IO和Slave_SQL 这两个线程的状态是不是yes 
*************************** 1. row *************************** 
              Slave_IO_State: Waiting for master to send event 
                  Master_Host: 172.16.80.118 
                  Master_User: rep 
                  Master_Port: 3306 
                Connect_Retry: 60 
              Master_Log_File: mysql-bin.000002 
          Read_Master_Log_Pos: 332 
              Relay_Log_File: relay-bin.000002 
                Relay_Log_Pos: 253 
        Relay_Master_Log_File: mysql-bin.000002 
            Slave_IO_Running: Yes 
            Slave_SQL_Running: Yes 
              Replicate_Do_DB:  
          Replicate_Ignore_DB: mysql 
          Replicate_Do_Table:  
      Replicate_Ignore_Table:  
      Replicate_Wild_Do_Table:  
  Replicate_Wild_Ignore_Table:  
                  Last_Errno: 0 
                  Last_Error:  
                Skip_Counter: 0 
          Exec_Master_Log_Pos: 332 
              Relay_Log_Space: 403 
              Until_Condition: None 
              Until_Log_File:  
                Until_Log_Pos: 0 
          Master_SSL_Allowed: No 
          Master_SSL_CA_File:  
          Master_SSL_CA_Path:  
              Master_SSL_Cert:  
            Master_SSL_Cipher:  
              Master_SSL_Key:  
        Seconds_Behind_Master: 0 
Master_SSL_Verify_Server_Cert: No 
                Last_IO_Errno: 0 
                Last_IO_Error:  
              Last_SQL_Errno: 0 
              Last_SQL_Error:  
  Replicate_Ignore_Server_Ids:  
            Master_Server_Id: 1 
1 row in set (0.00 sec)

六、登陆主库建立数据库，看是否会同步到从库上面 
[root@centos03 ~]# mysql -uroot -p123456 -S /data/3306/mysql.sock  
mysql> create database martin;  
Query OK, 1 row affected (0.01 sec) 
  
  
观察从库，能够看到已经同步过来 
mysql> show databases; 
+--------------------+ 
| Database          | 
+--------------------+ 
| information_schema | 
| martin            | 
| mysql              | 
| performance_schema | 
| test              | 
+--------------------+ 
5 rows in set (0.00 sec) 
  
  
[root@centos03 ~]# cd /data/3307 
[root@centos03 3307]# ls 
data  my.cnf  mysql  mysqld.pid  mysql_martin3307.err  mysql.sock  relay-bin.000001  relay-bin.000002  relay-bin.index  relay-log.info 
[root@centos03 3307]# cat data/master.info  
18 
mysql-bin.000002 
419 
172.16.80.118 
rep 
123456 
3306 
60 
0 
  
[root@centos03 3307]# mysqlbinlog  relay-bin.000002 
/*!\C utf8 *//*!*/; 
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=33/*!*/; 
SET @@session.lc_time_names=0/*!*/; 
SET @@session.collation_database=DEFAULT/*!*/; 
create database martin

mysql主从复制原理总结：

一、异步同步方式

二、逻辑同步模式，多种模式，默认是经过sql语句执行

三、主库经过记录bin-log实现对从库的同步，bin-log记录数据库更新的语句

四、主库一个IO线程，从库一个IO线程和一个SQL线程

五、从库关键文件master.info  relay-log  relay-info 

六、若是从库还须要作级联从库，从库须要打开log-bin和log-slave-updates参数

 

监控mysql主从状态（这里咱们简单监控从库上面io和sql线程yes总数为不是2就认为主从出现问题了）

在客户端编写脚本

[root@centos03 tools]# cat /tmp/mysql-replication.sh 

#!/bin/bash

/application/mysql/bin/mysql -uroot -p123456 -e 'show slave status\G' -S /data/3307/mysql.sock|grep -Ei "Slave_IO_Running|Slave_SQL_Running"|awk '{print $2}'|grep -c Yes

 

修改zabbix-agent配置文件

UnsafeUserParameters=1

UserParameter=mysql.replication,/tmp/mysql-replication.sh

 

在服务器端添加监控项--触发器--图形

 

 

 

正常状况下的图形以下

此时咱们模拟主库挂掉

[root@centos03 tools]# /data/3306/mysql stop

 

本文永久更新连接地址：http://www.linuxidc.com/Linux/2016-08/134669.htm

 

 

#### 5.6 版本能够参考  GTID 方式+mysqldump搭建准备复制模式  测试经过(数据库比较小推荐这种方式)

https://dev.mysql.com/doc/refman/5.6/en/replication-gtids-howto.html

 

------=========master----------
1、 启用gtid:

一、 备份my.cnf文件：
cp /mydb/mysql/app/mysql/my.cnf /mydb/mysql/app/mysql/my.cnf0502
二、master端vi /mydb/mysql/app/mysql/my.cnf加入如下参数：
#GTID
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates = 1

log_bin=mysql-bin

 

2.2 create rep user

二、创建用于同步的帐号
mysql> grant replication slave on *.* to rep@'172.16.80.%' identified by '123456';
说明：replication slave 是mysql同步的必须权限，此处不要受权all

 

2、 停应用后处理会话
一、 应用中止

mysql> SET @@global.read_only = ON;

二、 以下查询若有结果则kill掉：
mysql>select concat('kill connection ', id, ';') from information_schema.processlist;

 

3、重启

shell> mysqladmin -uusername -p shutdown

 

mysqld_safe  --defaults-file=/**/**/my.cnf &

 

 

2.2 use coyp or Mysqldump backup db. 

##Mysqldump方式在备份过程当中会锁myisam，搭建的时候须要注意主库的活动事物状况。、

##若是是大库，不适用直接拷贝，使用mysqldump比较好。

### master node :

mysqldump -uroot -p123456 -S /data/3306/mysql.sock -A -B --events --master-data=2 > /opt/rep.sql #导出全部数据库 

 

 

4、验证参数

mysql> show variables like 'log_bin%';
mysql> show variables like 'enforce%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | ON |
+--------------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'log_slave_updates';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| log_slave_updates | ON |
+-------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'gtid_mode';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_mode | ON |
+---------------+-------+
1 row in set (0.00 sec)

--============slave：

1、 启用gtid并修改server_id

一、备份my.cnf文件：
cp /mydb/mysql/app/mysql/my.cnf /mydb/mysql/app/mysql/my.cnf0502
二、slave端vi /mydb/mysql/app/mysql/my.cnf加入如下参数：
#GTID
gtid_mode = on
enforce_gtid_consistency = 1
log_slave_updates = 1

log_bin=mysql-bin

三、vi /mydb/mysql/app/mysql/my.cnf：
server_id=6

2、重启

shell> mysqladmin -uusername -p shutdown

 

mysqld_safe  --defaults-file=/**/**/my.cnf &

 

 mysql -uroot -p123456 -S /data/3307/mysql.sock < /opt/rep.sql  #导入从主库备份的数据库 

 

3、验证参数

mysql> show variables like 'log_bin%';

mysql> show variables like 'enforce%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | ON |
+--------------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'log_slave_updates';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| log_slave_updates | ON |
+-------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'gtid_mode';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_mode | ON |
+---------------+-------+
1 row in set (0.00 sec)

4、change master到指定源
change master to
master_host='xxx',
master_user='root',
master_password='',
master_port=xxx,
master_auto_position = 1;

5、启动slave同步
一、start slave；

 2. show slave status\G

 

6.Master node

mysql> SET @@global.read_only = off;

 

 

############## 5.6 版本能够参考  GTID 方式+ mysqlbackup 搭建准备复制模式  测试经过( 数据库比较大 推荐这种方式)

目前个人环境都仍是mysql5.6的，因此选择用3.12的版本

mysqlbackup企业版在mysql的开源社区是下载不了的，须要有metalink的帐号才能下载。（早期是能够下载的，估计后来用户的体验不错，须要受权购买oracle的服务才能够了）

目前个人环境都仍是mysql5.6的，因此选择用3.12的版本（3.12的版本修复了几个重要的bug，建议用这个版本）

其余设置命令使用的快捷方式：ln -s /opt/mysql/meb-3.12/bin/mysqlbackup /usr/bin/mysqlbackup

3.1.3 建立单独的备份用户

其余业务分开帐号的使用，数据库做为重要的系统，通常都会每一年变动管理密码的，这里单独使用一个避免后期还须要变动；
备份用户所须要的基本权限

GRANT CREATE, INSERT, DROP, UPDATE ON mysql.backup_progress TO 'mysqlbackup'@'localhost' identified by '123456';;
GRANT CREATE, INSERT, SELECT, DROP, UPDATE ON mysql.backup_history TO 'mysqlbackup'@'localhost';
GRANT REPLICATION CLIENT ON *.* TO 'mysqlbackup'@'localhost';
GRANT SUPER ON *.* TO 'mysqlbackup'@'localhost';
GRANT LOCK TABLES, SELECT, CREATE, DROP, FILE ON *.* TO 'mysqlbackup'@'localhost';
GRANT reload on *.* to 'mysqlbackup'@'localhost';

 

###machine 1 primay database:
全备的脚本

#####
[root@db02 backup]# mysqlbackup --user=mysqlbackup --password=123456 --backup-dir=/tmp/dba --with-timestamp --socket /db/mysql/data/mysqltmp/mysql_3307.sock backup

####output
180518 08:26:24 mysqlbackup: INFO: MEB logfile created at /tmp/dba/2018-05-18_08-26-24/meta/MEB_2018-05-18.08-26-24_backup.log

--------------------------------------------------------------------
Server Repository Options:
--------------------------------------------------------------------
datadir = /db/mysql/data_3307/
innodb_data_home_dir =
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /db/mysql/data_3307/
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = 16384
innodb_checksum_algorithm = innodb
innodb_undo_directory = /db/mysql/data_3307/
innodb_undo_tablespaces = 0
innodb_undo_logs = 128

--------------------------------------------------------------------
Backup Config Options:
--------------------------------------------------------------------
datadir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = 16384
innodb_checksum_algorithm = innodb
innodb_undo_directory = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_undo_tablespaces = 0
innodb_undo_logs = 128

mysqlbackup: INFO: Unique generated backup id for this is 15266031843454002

mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
180518 08:26:26 mysqlbackup: INFO: Full Backup operation starts with following threads
1 read-threads 6 process-threads 1 write-threads
180518 08:26:26 mysqlbackup: INFO: System tablespace file format is Antelope.
180518 08:26:26 mysqlbackup: INFO: Starting to copy all innodb files...
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/ibdata1 (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Found checkpoint at lsn 1626047.
180518 08:26:26 mysqlbackup: INFO: Starting log scan from lsn 1625600.
180518 08:26:26 mysqlbackup: INFO: Copying log...
180518 08:26:26 mysqlbackup: INFO: Log copied, lsn 1626047.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/innodb_index_stats.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/innodb_table_stats.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/slave_master_info.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/slave_relay_log_info.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql/slave_worker_info.ibd (Antelope file format).
180518 08:26:26 mysqlbackup: INFO: Completing the copy of innodb files.
180518 08:26:26 mysqlbackup: INFO: Starting to copy Binlog files...
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000001.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000002.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000003.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000004.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000005.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000006.
180518 08:26:26 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000007.
180518 08:26:27 mysqlbackup: INFO: Preparing to lock tables: Connected to mysqld server.
180518 08:26:27 mysqlbackup: INFO: Starting to lock all the tables...
180518 08:26:27 mysqlbackup: INFO: All tables are locked and flushed to disk
180518 08:26:27 mysqlbackup: INFO: Copying /db/mysql/data_3307/mysql-bin.000008.
180518 08:26:28 mysqlbackup: INFO: Completed the copy of binlog files...
180518 08:26:28 mysqlbackup: INFO: Opening backup source directory '/db/mysql/data_3307/'
180518 08:26:28 mysqlbackup: INFO: Starting to backup all non-innodb files in
subdirectories of '/db/mysql/data_3307/'
180518 08:26:28 mysqlbackup: INFO: Copying the database directory 'mysql'
180518 08:26:28 mysqlbackup: INFO: Copying the database directory 'peng'
180518 08:26:28 mysqlbackup: INFO: Copying the database directory 'performance_schema'
180518 08:26:29 mysqlbackup: INFO: Copying the database directory 'test'
180518 08:26:29 mysqlbackup: INFO: Copying the database directory 'tmp'
180518 08:26:29 mysqlbackup: INFO: Completing the copy of all non-innodb files.
180518 08:26:30 mysqlbackup: INFO: A copied database page was modified at 1626047.
(This is the highest lsn found on page)
Scanned log up to lsn 1626047.
Was able to parse the log up to lsn 1626047.
Maximum page number for a log record 0
180518 08:26:30 mysqlbackup: INFO: All tables unlocked
180518 08:26:30 mysqlbackup: INFO: All MySQL tables were locked for 2.818 seconds.
180518 08:26:30 mysqlbackup: INFO: Reading all global variables from the server.
180518 08:26:30 mysqlbackup: INFO: Completed reading of all global variables from the server.
180518 08:26:30 mysqlbackup: INFO: Creating server config files server-my.cnf and server-all.cnf in /tmp/dba/2018-05-18_08-26-24
180518 08:26:30 mysqlbackup: INFO: Full Backup operation completed successfully.
180518 08:26:30 mysqlbackup: INFO: Backup created in directory '/tmp/dba/2018-05-18_08-26-24'
180518 08:26:30 mysqlbackup: INFO: MySQL binlog position: filename mysql-bin.000008, position 1659

-------------------------------------------------------------
Parameters Summary
-------------------------------------------------------------
Start LSN : 1625600
End LSN : 1626047
-------------------------------------------------------------

mysqlbackup completed OK!
[mysql@localhost dba]$

#######

##[root@db02 backup]# mysqlbackup --backup-dir=/backup/2017-09-18_13-49-11 apply-log #由于在备份期间数据库还在读写，把这期间的log进行应用，达到数据的一致性

mysqlbackup --backup-dir=/tmp/dba/2018-05-18_08-26-24 apply-log

####

180518 08:30:51 mysqlbackup: INFO: MEB logfile created at /tmp/dba/2018-05-18_08-26-24/meta/MEB_2018-05-18.08-30-51_apply_log.log

--------------------------------------------------------------------
Backup Config Options:
--------------------------------------------------------------------
datadir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = 16384
innodb_checksum_algorithm = innodb

mysqlbackup: INFO: Creating 14 buffers each of size 65536.
180518 08:30:51 mysqlbackup: INFO: Apply-log operation starts with following threads
1 read-threads 1 process-threads
mysqlbackup: INFO: Using up to 100 MB of memory.
180518 08:30:51 mysqlbackup: INFO: ibbackup_logfile's creation parameters:
start lsn 1625600, end lsn 1626047,
start checkpoint 1626047.
InnoDB: Doing recovery: scanned up to log sequence number 1626047
mysqlbackup: INFO: InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
mysqlbackup: INFO: InnoDB: Setting log file size to 536870912
InnoDB: Progress in MB: 100 200 300 400 500
mysqlbackup: INFO: InnoDB: Setting log file size to 536870912
InnoDB: Progress in MB: 100 200 300 400 500
mysqlbackup: INFO: InnoDB: Setting log file size to 536870912
InnoDB: Progress in MB: 100 200 300 400 500
180518 08:30:55 mysqlbackup: INFO: We were able to parse ibbackup_logfile up to
lsn 1626047.
mysqlbackup: INFO: Last MySQL binlog file position 0 1659, file name mysql-bin.000008:1659
180518 08:30:55 mysqlbackup: INFO: The first data file is '/tmp/dba/2018-05-18_08-26-24/datadir/ibdata1'
and the new created log files are at '/tmp/dba/2018-05-18_08-26-24/datadir'
180518 08:30:55 mysqlbackup: INFO: Apply-log operation completed successfully.
180518 08:30:55 mysqlbackup: INFO: Full backup prepared for recovery successfully.

mysqlbackup completed OK!
########

从库还原 （in standby db） ：

 

machine 2:

mysqlbackup --force --socket=/var/lib/mysql/mysql_3307.sock --datadir=/mysqldb/data_3307 --backup-dir=/tmp/dba/2018-05-18_08-26-24 copy-back

[root@localhost 2018-05-18_08-26-24]# mysqlbackup --force --socket=/var/lib/mysql/mysql_3307.sock --datadir=/mysqldb/data_3307 --backup-dir=/tmp/dba/2018-05-18_08-26-24 copy-back
MySQL Enterprise Backup version 3.12.0 Linux-3.8.13-35.2.1.el7uek.x86_64-x86_64 [2015/03/10]
Copyright (c) 2003, 2015, Oracle and/or its affiliates. All Rights Reserved.

mysqlbackup: INFO: Starting with following command line ...
mysqlbackup --force --socket=/var/lib/mysql/mysql_3307.sock
--datadir=/mysqldb/data_3307 --backup-dir=/tmp/dba/2018-05-18_08-26-24
copy-back

mysqlbackup: INFO:
IMPORTANT: Please check that mysqlbackup run completes successfully.
At the end of a successful 'copy-back' run mysqlbackup
prints "mysqlbackup completed OK!".

180518 10:18:57 mysqlbackup: INFO: MEB logfile created at /tmp/dba/2018-05-18_08-26-24/meta/MEB_2018-05-18.10-18-57_copy_back.log

mysqlbackup: WARNING: If you restore to a server of a different version, the innodb_data_file_path parameter might have a different default. In that case you need to add 'innodb_data_file_path=ibdata1:12M:autoextend' to the target server configuration.
mysqlbackup: WARNING: If you restore to a server of a different version, the innodb_log_files_in_group parameter might have a different default. In that case you need to add 'innodb_log_files_in_group=3' to the target server configuration.
mysqlbackup: WARNING: If you restore to a server of a different version, the innodb_log_file_size parameter might have a different default. In that case you need to add 'innodb_log_file_size=536870912' to the target server configuration.
--------------------------------------------------------------------
Server Repository Options:
--------------------------------------------------------------------
datadir = /mysqldb/data_3307
innodb_data_home_dir = /mysqldb/data_3307
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /mysqldb/data_3307
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = Null
innodb_checksum_algorithm = innodb

--------------------------------------------------------------------
Backup Config Options:
--------------------------------------------------------------------
datadir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_data_file_path = ibdata1:12M:autoextend
innodb_log_group_home_dir = /tmp/dba/2018-05-18_08-26-24/datadir
innodb_log_files_in_group = 3
innodb_log_file_size = 536870912
innodb_page_size = 16384
innodb_checksum_algorithm = innodb

mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
180518 10:18:57 mysqlbackup: INFO: Copy-back operation starts with following threads
1 read-threads 1 write-threads
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/ibdata1.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/innodb_index_stats.ibd.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/innodb_table_stats.ibd.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/slave_master_info.ibd.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/slave_relay_log_info.ibd.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql/slave_worker_info.ibd.
180518 10:18:57 mysqlbackup: INFO: Starting to copy Binlog files...
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000001.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000002.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000003.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000004.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000005.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000006.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000007.
180518 10:18:57 mysqlbackup: INFO: Copying /tmp/dba/2018-05-18_08-26-24/datadir/mysql-bin.000008.
180518 10:18:58 mysqlbackup: INFO: Completed the copy of binlog files...
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'mysql'
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'peng'
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'performance_schema'
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'test'
180518 10:18:58 mysqlbackup: INFO: Copying the database directory 'tmp'
180518 10:18:58 mysqlbackup: INFO: Completing the copy of all non-innodb files.
180518 10:18:58 mysqlbackup: INFO: Copying the log file 'ib_logfile0'
mysqlbackup: Progress in MB: 200 400
180518 10:19:12 mysqlbackup: INFO: Copying the log file 'ib_logfile1'
mysqlbackup: Progress in MB: 600 800 1000
180518 10:19:15 mysqlbackup: INFO: Copying the log file 'ib_logfile2'
mysqlbackup: Progress in MB: 1200 1400
180518 10:19:19 mysqlbackup: INFO: Creating server config files server-my.cnf and server-all.cnf in /mysqldb/data_3307
180518 10:19:19 mysqlbackup: INFO: Copy-back operation completed successfully.
180518 10:19:19 mysqlbackup: INFO: Finished copying backup files to '/mysqldb/data_3307'

mysqlbackup completed OK! with 3 warnings

#############

in standby database 

chown -R mysql:mysql /mysqldb/data_3307
mysqld_safe --defaults-file=/mysqldb/data_3307/my.cnf &

  

mysql> show master status;
mysql> reset master;
mysql> show master status;

 

4、在slave端进行GTID_PURGED，执行backup_gtid_executed.sql中的SET @@GLOBAL.GTID_PURGED语句：

mysql> SET @@GLOBAL.GTID_PURGED='4e461dbc-54a7-11e8-8589-005056a41701:1-9';

 

mysql> change master to
-> master_host='10.241.95.221',
-> master_user='rep',
-> master_password='123456',
-> master_port=3307,
-> master_auto_position = 1;

 

6、启动slave同步

1.start slave；

2. show slave status\G

####

 

#####more  info about gtids

17.1.3.2 Setting Up Replication Using GTIDs
This section describes a process for configuring and starting GTID-based replication in MySQL 5.6. This is a “cold start” procedure that assumes either that you are starting the replication master for the first time, or that it is possible to stop it; for information about provisioning replication slaves using GTIDs from a running master, see Section 17.1.3.3, “Using GTIDs for Failover and Scaleout”.

The key steps in this startup process for the simplest possible GTID replication topology—consisting of one master and one slave—are as follows:

If replication is already running, synchronize both servers by making them read-only.

Stop both servers.

Restart both servers with GTIDs, binary logging, and slave update logging enabled, and with statements that are unsafe for GTID-based replication disabled. In addition, the servers should be started in read-only mode, and the slave SQL and I/O threads should be prevented from starting on the slave.

The mysqld options necessary to start the servers as described are discussed in the example that follows later in this section.

Instruct the slave to use the master as the replication data source and to use auto-positioning. The SQL statements needed to accomplish this step are described in the example that follows later in this section.

Take a new backup. Binary logs containing transactions without GTIDs cannot be used on servers where GTIDs are enabled, so backups taken before this point cannot be used with your new configuration.

Start the slave, then disable read-only mode again on both servers, so that they can accept updates.

In the following example, two servers are already running as master and slave, using MySQL's “classic” file-based replication protocol.

Most of the steps that follow require the use of the MySQL root account or another MySQL user account that has the SUPER privilege. mysqladmin shutdown requires either the SUPER privilege or the SHUTDOWN privilege.

Step 1: Synchronize the servers. Make the servers read-only. To do this, enable the read_only system variable by executing the following statement on both servers:

mysql> SET @@global.read_only = ON;
Wait for all ongoing transactions to commit or roll back. Then, allow the slave to catch up with the master. It is extremely important that you make sure the slave has processed all updates before continuing.

If you use binary logs for anything other than replication, for example to do point in time backup and restore, wait until you do not need the old binary logs containing transactions without GTIDs. Ideally, wait for the server to purge all binary logs, and wait for any existing backup to expire.

Important
It is important to understand that logs containing transactions without GTIDs cannot be used on servers where GTIDs are enabled. Before proceeding, you must be sure that transactions without GTIDs do not exist anywhere in the topology.

Step 2: Stop both servers. Stop each server using mysqladmin as shown here, where username is the user name for a MySQL user having sufficient privileges to shut down the server:

shell> mysqladmin -uusername -p shutdown
Then supply this user's password at the prompt.

Step 3: Restart both servers with GTIDs enabled. To enable binary logging with global transaction identifiers, each server must be started with GTID mode, binary logging, slave update logging enabled, and with statements that are unsafe for GTID-based replication disabled. In addition, you should prevent unwanted or accidental updates from being performed on either server by starting both in read-only mode. This means that both servers must be started with (at least) the options shown in the following invocation of mysqld_safe:

shell> mysqld_safe --gtid_mode=ON --log-bin --log-slave-updates --enforce-gtid-consistency &
Note
Prior to MySQL 5.6.9, --enforce-gtid-consistency was named --disable-gtid-unsafe-statements.

In addition, you should start the slave with the --skip-slave-start option along with the other server options specified in the example just shown.

Note
--gtid-mode is not a boolean, but an enumeration. Use one of the values ON or OFF only, when setting this option. Using a numeric value such as 0 or 1 can lead to unexpected results.

For more information about the --gtid-mode and --enforce-gtid-consistency server options, see Section 17.1.4.5, “Global Transaction ID Options and Variables”.

Depending on your configuration, supply additional options to mysqld_safe or other mysqld startup script.

Step 4: Direct the slave to use the master. Tell the slave to use the master as the replication data source, and to use GTID-based auto-positioning rather than file-based positioning. Execute a CHANGE MASTER TO statement on the slave, using the MASTER_AUTO_POSITION option to tell the slave that transactions will be identified by GTIDs.

You may also need to supply appropriate values for the master's host name and port number as well as the user name and password for a replication user account which can be used by the slave to connect to the master; if these have already been set prior to Step 1 and no further changes need to be made, the corresponding options can safely be omitted from the statement shown here.

mysql> CHANGE MASTER TO
> MASTER_HOST = host,
> MASTER_PORT = port,
> MASTER_USER = user,
> MASTER_PASSWORD = password,
> MASTER_AUTO_POSITION = 1;
Neither the MASTER_LOG_FILE option nor the MASTER_LOG_POS option may be used with MASTER_AUTO_POSITION set equal to 1. Attempting to do so causes the CHANGE MASTER TO statement to fail with an error.

Step 5: Take a new backup. Existing backups that were made before you enabled GTIDs can no longer be used on these servers now that you have enabled GTIDs. Take a new backup at this point, so that you are not left without a usable backup.

For instance, you can execute FLUSH LOGS on the server where you are taking backups. Then either explicitly take a backup or wait for the next iteration of any periodic backup routine you may have set up.

Step 6: Start the slave and disable read-only mode. Start the slave like this:

mysql> START SLAVE;
Allow the master to begin accepting updates once again by running the following statement:

mysql> SET @@global.read_only = OFF;
GTID-based replication should now be running, and you can begin (or resume) activity on the master as before. Section 17.1.3.3, “Using GTIDs for Failover and Scaleout”, discusses creation of new slaves when using GTIDs.

PREV HOME UP NEXT
User Comments
Sign UpLogin You must be logged in to post a comment.

 

 

###############22

https://www.cnblogs.com/1477717815fuming/p/8006143.html

 

公司规模已经造成，用户数据已成为公司的核心命脉，一次老王一不当心把数据库文件删除，经过mysqldump备份策略恢复用了两个小时，在这两小时中，公司业务中断，损失100万，老王作出深入检讨，公司也所以对于数据库的性能和可靠性提出更高要求。要求对数据库进行改造，使其承载力进行提高，故障修复时间减小，有没有能实现的方案呢？

 

数据库常遇到的问题

1、性能问题

一、向上拓展 scale up ：针对单台服务器，提升服务器的硬件性能，好比：内存，cpu等，个体自己 容易达到极限

二、向外拓展 scale out ：多台服务器造成集群，共同完成一件事情

2、可用性问题

一、数据库服务中断

二、误操做数据损坏

三、硬件故障

四、数据库升级测试遭遇bug

五、黑客攻击

 

数据库高可用技术说明

高可用架构对于互联网服务基本是标配，不管是应用服务仍是数据库服务都须要作到高可用。虽然互联网服务号称7*24小时不间断服务，但多多少少有一些时候服务不可用，好比某些时候网页打不开，百度不能搜索或者没法发微博，发微信等。通常而言，衡量高可用作到什么程度能够经过一年内服务不可用时间做为参考，要作到3个9的可用性，一年内只能累计有8个小时不可服务，而若是要作到5个9的可用性，则一年内只能累计5分钟服务中断。因此虽然说每一个公司都说本身的服务是7*24不间断的，但实际上能作到5个9的屈指可数，甚至根本作不到，国内互联网巨头BAT(百度，阿里巴巴，腾讯)都有由于故障致使的停服问题。对于一个系统而言，可能包含不少模块，好比前端应用，缓存，数据库，搜索，消息队列等，每一个模块都须要作到高可用，才能保证整个系统的高可用。对于数据库服务而言，高可用可能更复杂，对用户的服务可用，不只仅是能访问，还须要有正确性保证，所以，对于实现数据库高可用，对互联网公司来讲极其重要！

 

企业级数据库高可用架构图

 

Mysql主从架构技术说明

Mysql内建的复制功能是构建大型，高性能应用程序的基础。将Mysql的数据分布到多个系统上去，这种分布的机制，是经过将Mysql的某一台主机（Master）的数据复制到其它主机（slaves）上，并从新执行一遍来实现的。复制过程当中一个服务器充当主服务器，而一个或多个其它服务器充当从服务器。主服务器将更新写入二进制日志文件，这些日志能够记录发送到从服务器的更新。当一个从服务器链接主服务器时，它通知主服务器从服务器在日志中读取的最后一次成功更新的位置。从服务器接收从那时起发生的任何更新，而后封锁并等待主服务器通知新的更新。

主从复制架构图

 

数据库复制特性

Mysql复制解决的问题

MySQL复制技术有如下一些特色：

(1) 数据分布 (Data distribution )

(2) 负载平衡(load balancing)

(3) 备份(Backups)

(4) 高可用性和容错性 High availabilityand failover

 

Mysql复制如何工做

Mysql的复制功能主要有3个步骤：

(1) 主服务器（master）将改变记录到二进制日志(binarylog)中（这些记录叫作二进制日志事件，binary log events）

(2) 从服务器（slave）将主服务器master的binary logevents拷贝到它的中继日志(relay log)

(3) slave重作中继日志中的事件，将改变反映它本身的数据。

一、该过程的第一部分就是master记录二进制日志。在每一个事务更新数据完成以前，master在二进制日志记录这些改变。MySQL将事务串行的写入二进制日志，在事件写入二进制日志完成后，master通知存储引擎提交事务。此后可接收slave的请求

二、下一步就是slave将master的binary log拷贝到它本身的中继日志。首先，slave开始一个工做线程——I/O线程。I/O线程在master上打开一个普通的链接，而后开始在主节点上binlog dump process（二进制转存线程）。Binlog dump process从master的二进制日志中读取事件，若是已经跟上master，它会睡眠并等待master产生新的事件。I/O线程将这些事件写入中继日志。

三、 SQL slave thread（SQL从线程）处理该过程的最后一步。SQL线程从中继日志读取事件，并重放其中的事件而更新slave的数据，使其与master中的数据一致。只要该线程与I/O线程保持一致，中继日志一般会位于OS的缓存中，因此中继日志的开销很小。

I/O线程：将master数据库二进制日志拉到slave数据库上，并将二进制日志写到中继日志，写完以后，他会睡眠并等待master数据库二进制日志更新，一旦更新，就会写入slave数据库的中继日志中

SQL线程：读取中继日志的事件，并在数据库中执行，写入到内存中，使slave数据库的数据与master数据库中的数据一致

Mysql实现企业级数据库主从复制架构实战

注意：slave数据库只能是可读的，不能是可写的，若是改变了slave数据库的数据，master不能从slave数据库上同步数据，致使主从数据库数据不一致。

实战演练

1、环境准备

centos系统服务器2台、一台用户作Mysql主服务器，一台用于作Mysql从服务器，都在同一个网段中，配置好yum源、防火墙关闭、各节点时钟服务同步、各节点之间能够经过主机名互相通讯

2、准备步骤：

一、iptables -F && setenforce 清空防火墙策略，关闭selinux

二、拿两台服务器都使用yum方式安装Mysql服务，要求版本一致

三、分别启动两台服务器mysql服务，确保服务正常

3、实现步骤:

一、配置master主服务器

对master进行配置，包括打开二进制日志，指定惟一的servr ID。例如，在配置文件加入以下值

vim /etc/my.cnf

server-id=1 #配置server-id，让主服务器有惟一ID号（让从服务器知道他的主服务器是谁）

log-bin=mysql-bin #打开Mysql日志，日志格式为二进制

skip-name-resolve#关闭名称解析，（非必须）

而后重启数据库服务

systemctl restart mariadb

2．建立复制账号

在Master的数据库中创建一个备份账户：每一个slave使用标准的MySQL用户名和密码链接master

。进行复制操做的用户会授予REPLICATION SLAVE权限。（给从服务器受权，让他能从主服务器拷贝二进制日志）

mysql

GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO slave@'192.168.10.%' IDENTIFIED BY 'magedu';

3．查看主服务器状态

在Master的数据库执行show master status，查看主服务器二进制日志状态

四、配置slave从服务器

对slave进行配置，打开中继日志，指定惟一的servr ID，设置只读权限。在配置文件加入以下值

vim /etc/my.cnf

server-id=2 #配置server-id，让从服务器有惟一ID号

relay_log = mysql-relay-bin #打开Mysql日志，日志格式为二进制

read_only = 1 #设置只读权限

log_bin = mysql-bin #开启从服务器二进制日志

log_slave_updates = 1 #使得更新的数据写进二进制日志中

而后重启数据库服务

systemctl restart mariadb

5．启动从服务器复制线程

让slave链接master，并开始重作master二进制日志中的事件。

mysql

CHANGE MASTER TO MASTER_HOST='192.168.10.190',

MASTER_USER='slave',

MASTER_PASSWORD='magedu',

MASTER_LOG_FILE='mysql-bin.000001',

MASTER_LOG_POS=278;

执行start slave；# 启动复制线程。

六、查看从服务器状态

可以使用SHOW SLAVE STATUS\G查看从服务器状态，以下所示，也可用show processlist \G查看前复制状态：

mysql

SHOW SLAVE STATUS\G

Slave_IO_Running: Yes #IO线程正常运行

Slave_SQL_Running: Yes #SQL线程正常运行

7.测试

理想的结果是在主服务器上添加的数据，在从服务器上也会同步

在主服务器上

 

在从服务器上

4、添加新slave服务器

假如master已经运行好久了，想对新安装的slave进行数据同步，甚至它没有master的数据。

此时，有几种方法可使slave从另外一个服务开始，例如，从master拷贝数据，从另外一个slave克隆，从最近的备份开始一个slave。为了加快Slave与master同步，可用如下方式先进行数据同步：

(1)master的某个时刻的数据快照；

(2)数据库的备份数据

(3)master的二进制日志文件。

 

实现主从从架构

也能够搭建主从从架构，让从服务器之间进行复制

就是在从服务器也开启二进制日志，而后从的从I/O线程再将从的二进制日志给拷贝过来写入到本身的relay log中，而后sql线程再读取relay log中的事件，在数据库中执行，写入到内存中。

 

Mysql复制过滤器

复制过滤器：

仅复制有限一个或几个数据库相关的数据，而非全部；由复制过滤器进行；

有两种实现思路：

(1) 主服务器

主服务器仅向二进制日志中记录有关特定数据库相关的写操做；

binlog_do_db=

binlog_ignore_db=

(2) 从服务器

从服务器的SQL THREAD仅重放关注的数据库或表相关的事件，并将其应用于本地；

Replicate_Do_DB=

Replicate_Ignore_DB=

 

企业常见数据库架构

1、单一master和多slave

在实际应用场景中，MySQL复制90%以上都是一个Master复制到一个或者多个Slave的架构模式，主要用于读压力比较大的应用的数据库端廉价扩展解决方案。由于只要Master和Slave的压力不是太大（尤为是Slave端压力）的话，异步复制的延时通常都不多不多。尤为是自从Slave端的复制方式改为两个线程处理以后，更是减少了Slave端的延时问题。而带来的效益是，对于数据实时性要求不是特别高的应用，只须要经过廉价的pcserver来扩展Slave的数量，将读压力分散到多台Slave的机器上面，便可经过分散单台数据库服务器的读压力来解决数据库端的读性能瓶颈，毕竟在大多数数据库应用系统中的读压力仍是要比写压力大不少。这在很大程度上解决了目前不少中小型网站的数据库压力瓶颈问题，甚至有些大型网站也在使用相似方案解决数据库瓶颈。

单一master和多slave架构图

(1) 不一样的slave扮演不一样的做用(例如使用不一样的索引，或者不一样的存储引擎)；

(2) 用一个slave做为备用master，只进行复制；#主服务器挂了以后，可在从服务器执行

1> 在备机上执行STOP SLAVE 和RESET MASTER

2> 查看show slave status \G;

3> 而后修改应用的链接地址。

(3) 用一个远程的slave，用于灾难恢复；

2、互为主从Master-Master(Master-Master in Active-Active Mode)

Master-Master复制的两台服务器，既是master，又是另外一台服务器的slave。这样，任何一方所作的变动，都会经过复制应用到另一方的数据库中。

即：在两台服务器上既执行master的操做又执行slave的操做（注意：两台数据库都必须是可写的）

互为主从复制过程

互为主从：两个节点各自都要开启binlog和relay log；

一、数据不一致；

二、自动增加id；

什么是自增加ID？

对于某些惟一性的字段，能够经过设置自增加ID来实现,自增加ID的数据，表明这个表中存在一条惟一的记录；而自增加id是确定不会重复的；

建立表，设置ID为自增加

create table userInfo (id int PRIMARY KEY AUTO_INCREMENT,name varchar(50) NOT NULL)；

两边插入数据看数据增加

insert into userInfo(name) value('xiao'),('da'),('lao');

定义一个节点使用奇数id

auto_increment_increment=2 #表示自增加字段每次递增的量

auto_increment_offset=1 #表示自增加字段从那个数开始

另外一个节点使用偶数id

auto_increment_increment=2

auto_increment_offset=2

 

配置：

一、server_id必需要使用不一样值；

二、均启用binlog和relay log； read only = 0(由于互为主从，因此必须是可写的)

三、存在自动增加id的表，为了使得id不相冲突，须要定义其自动增加方式；

服务启动后执行以下两步：

四、都受权有复制权限的用户帐号；

五、各把对方指定为主节点；

 

实验：数据库互为主从复制步骤

1.修改mysql配置文件

一台数据库服务器上

vim /etc/my.cnf

server-id = 1

log_bin = mysql_bin

relay_log = relay-log

skip-name-resolve = on

log_slave_updates = 1

auto_increment_increment=2

auto_increment_offset=1

另外一台数据库服务器上

vim /etc/my.cnf

server-id = 2

relay_log = relay-log

log_bin = mysql-log

skip-name-resolve = on

log_slave_updates = 1

auto_increment_increment=2

auto_increment_offset=2

修改完配置文件后，重启数据库服务

systemctl restart mariadb

2．建立复制账号

分别在两台数据库服务器上建立复制帐号

mysql

GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO slave@'192.168.10.%' IDENTIFIED BY 'magedu';

3．启动从服务器复制线程

让slave链接master，并开始重作master二进制日志中的事件。

mysql

CHANGE MASTER TO MASTER_HOST='192.168.10.190',

MASTER_USER='slave',

MASTER_PASSWORD='magedu',

MASTER_LOG_FILE='mysql-bin.000001',

MASTER_LOG_POS=278;

执行start slave；# 启动复制线程。

另外一台数据库服务器也是如此

四、查看从服务器状态

可以使用SHOW SLAVE STATUS\G查看从服务器状态，以下所示，也可用show processlist \G查看前复制态：

mysql

SHOW SLAVE STATUS\G

Slave_IO_Running: Yes #IO线程正常运行

Slave_SQL_Running: Yes #SQL线程正常运行

两台数据库服务器都显示如上结果就ok。

5.建立表，设置ID为自增加,两边插入数据看数据增加

在一台数据库服务器上

mysql

create database dnf;

use dnf;

create table userinfo (id int PRIMARY KEY AUTO_INCREMENT,name varchar(20) NOT NULL);

insert into userinfo (name) values('ni'),('wo'),('ta');

而后查看表，由于是自增加id，从1开始，步长为2，因此添加的数据id为1,3,5

而后在另外一台数据库服务器插入数据，由于是自增加id，从2开始，步长为2，因此新添加的数据id为6,8,10

 

排错：当配置文件中配置中继日志格式不当心配置错了，或者让slave链接master，执行sql语句不当心写错了，都有可能致使start slave；报错，此时能够show slave status\G;会出现一大串信息，里面会提示错误。找到错误之后，重置slave，reset slave;从新设置，而后再start slave；

注意：mysql的错误日志很是重要，能够提供错误信息，从而找到错误缘由。

互为主从容易致使数据不一致，此时咱们能够用两个实例来互为主从

 

三种复制方式

异步复制（Asynchronous replication）

MySQL默认的复制便是异步的，主库在执行完客户端提交的事务后会当即将结果返给给客户端，并不关心从库是否已经接收并处理，这样就会有一个问题，主若是crash掉了，此时主上已经提交的事务可能并无传到从上，若是此时，强行将从提高为主，可能致使新主上的数据不完整

全同步复制（Fully synchronous replication）

指当主库执行完一个事务，全部的从库都执行了该事务才返回给客户端。由于须要等待全部从库执行完该事务才能返回，因此全同步复制的性能必然会收到严重的影响。须要有超时时间。

半同步复制（Semisynchronous replication）

介于异步复制和全同步复制之间，主库在执行完客户端提交的事务后不是马上返回给客户端，而是等待至少一个从库接收到并写到relay log中才返回给客户端。相对于异步复制，半同步复制提升了数据的安全性，同时它也形成了必定程度的延迟，这个延迟最少是一个TCP/IP往返的时间。因此，半同步复制最好在低延时的网络中使用。

 

半同步复制

支持多种插件：/usr/lib64/mysql/plugins/

须要安装方可以使用：

mysql> INSTALL PLUGIN plugin_name SONAME 'shared_library_name';

半同步复制：

semisync_master.so

semisync_slave.so

主节点：

INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';

MariaDB [mydb]> SHOW GLOBAL VARIABLES LIKE 'rpl_semi%';

+------------------------------------+-------+

| Variable_name | Value |

+------------------------------------+-------+

| rpl_semi_sync_master_enabled | OFF |

| rpl_semi_sync_master_timeout | 10000 |

| rpl_semi_sync_master_trace_level | 32 |

| rpl_semi_sync_master_wait_no_slave | ON |

+------------------------------------+-------+

MariaDB [mydb]> SET GLOBAL rpl_semi_sync_master_enabled=ON/1;

stop slave;

start slave;

从节点：

INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';

MariaDB [mydb]> SHOW GLOBAL VARIABLES LIKE 'rpl_semi%';

+---------------------------------+-------+

| Variable_name | Value |

+---------------------------------+-------+

| rpl_semi_sync_slave_enabled | OFF |

| rpl_semi_sync_slave_trace_level | 32 |

+---------------------------------+-------+

MariaDB [mydb]> STOP SLAVE IO_THREAD;

MariaDB [mydb]> SET GLOBAL rpl_semi_sync_slave_enabled = ON ;

MariaDB [mydb]> SHOW GLOBAL VARIABLES LIKE 'rpl_semi%';

MariaDB [mydb]> START SLAVE IO_THREAD;

stop slave;

start slave;

可查看从库错误日志观察是否生效

master错误日志

slave错误日志

 

mysql优化：

1.能够用数据缓存，常见的memcache

2.数据库自己有不少缓存机制，可以使用对应的缓存策略

3.对数据来讲，竟可能使用索引

4.对请求而言，能够实现读写分离，对读请求负载均衡

5.对大数据库或者表，可根据业务逻辑进行分库分表

6.多有的优化，尽量网内存中存放

分库分表

分库：当数据库的数据很是庞大，能够把数据库分红几个数据库，每一个数据库当一类数据，最后在拼接起来

分表：有两种分法：

1.水平拆分：一个表中有10亿条记录，将这10亿条记录分红每10万条记录为一个表

2.垂直拆分：将一个字段或者多个字段分红一个表

http://tieba.baidu.com/p/4558183228