前面咱们已经对MySQL数据库的安装、命令、备份、索引、事务以及存储引擎等各个方面有了初步的理解,而今天咱们将从宏观的角度来理解什么是MySQL数据库的主从复制与读写分离。mysql
在实际的生产环境中,若是对MySQL数据库的读与写都在一台服务器上进行操做,不管是在安全性,高可用性、仍是高并发性等诸多方面都是没法知足的;而这就须要对MySQL数据库进行主从复制与读写分离。linux
咱们使用一个架构图来引出MySQL数据库主从复制的原理以及其做用,请看下图:sql
服务器做用:shell
MySQL主服务器:负责数据的写入;数据库
MySQL从服务器:同步主服务器的数据并进行数据的轮循读取;vim
那么这三台具有相同服务的MySQL服务器就称为“MySQL群集”。咱们能够从上图中看出,这样的安排实现数据同步的基础上,实现数据的读写分离,从而在保证数据的可靠性的同时也大大减轻了主服务器的压力。后端
下面咱们对MySQL的主从复制和读写分离进行逐一介绍并给出配置实例。缓存
MySQL主从复制与读写分离有着紧密的联系,能够这么说,MySQL的读写分离的实现须要基于主从复制的基础之上。tomcat
咱们经过下图来理解MySQL主、从服务器是如何进行复制同步的过程的。安全
1)首先,MySQL主服务器在更新数据库或其余进行数据库相关操做时,会在二进制日志文件中记录这些改变(这咱们在前面的增量备份与恢复的文章中进行了讲述,log-bin的设置以及如何使用mysqladmin命令刷新该日志。)当写入日志完成后,主服务器会告知存储引擎提交事务;
2)MySQL从服务器会将主服务器的二进制日志文件(Binary log)复制到其中继日志(Relay log)中。中继日志一般存放在系统缓存中,所以中继日志的开销很小;
3)从服务器经过自身线程从中继日志中读取事件,更新自身的日志文件使其与主服务器中的数据一致。
ps:复制过程当中有一个很重要的限制,即在从服务器上复制是串行化的,这就代表主服务器上的并行更新操做不能在从服务器上并行操做。
所需设备(咱们在虚拟机上模拟进行配置)清单:
Centos7一台:做为主服务器——master,ip地址为192.168.68.133
Centos7两台:做为从服务器——slave一、slave2IP地址分别为192.168.68.12九、192.168.68.132
所需安装服务:
安装而且配置ntp服务、都安装了MySQL5.7版本的数据库
首先,咱们联想一下增量备份的恢复操做,咱们都是依赖于data目录下的二进制日志文件,经过两种方法实现的,其中一种就是根据时间节点进行备份恢复操做的。那么咱们须要进行MySQL主从服务器复制,就须要先同步全部MySQL服务器的系统时间。
1)服务器名称设置以区别
hostnamectl set-hostname master su
2)安装配置ntp服务
yum install ntp -y #修改ntp服务配置文件,添加下面的两句 vim /etc/ntp.conf server 127.127.68.0 #服务器本地网段,127.127表示的就是192.168 fudge 127.127.68.0 stratum 8 #时区设置(东八区) #保存退出 systemctl start ntpd #可使用netstat -nutp | grep ntpd 命令查看服务开启状态
3)关闭防火墙和SELinux功能
systemctl stop firewalld setenforce 0
1)服务器名称设置
hostnamectl set-hostname slave1(slave2) su
2)两台从服务器上安装ntp和ntpdate服务并开启服务
yum install ntp ntpdate -y systemctl start ntpd
3)两台从服务器上关闭防火墙和SELinux功能
systemctl stop firewalld setenforce 0
4)使用ntpdate命令进行时间同步
[root@slave1 ~]# /usr/sbin/ntpdate 192.168.68.133 9 Jan 15:35:13 ntpdate[67450]: the NTP socket is in use, exiting
1)修改mysql的主配置文件并重启mysql服务
vim /etc/my.cnf #配置以下: log-bin = master-bin #二进制日志文件 master-bin能够本身设置 server-id = 1 #服务器的id号,用于区别 log-slave-updates=true #开启从服务器更新日志功能(结合复制流程链接) systemctl restart mysqld.service
2)进入数据库进行权限设置(受权)与刷新
[root@lokott ~]# mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.17-log Source distribution Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> grant replication slave on *.* to 'myslave'@'192.168.68.%' identified by '123456'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges;#刷新系统权限表 Query OK, 0 rows affected (0.00 sec) mysql> show master status; +-------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +-------------------+----------+--------------+------------------+-------------------+ | master-bin.000001 | 603 | | | | +-------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
注意核心命令:受权命令的解释,还有以上位置Position 记录数据603
命令格式:
grant replication slave on *.* to 'myslave'@'192.168.68.%' identified by '123456'; #grant ——受权命令 replication slave 从服务器复制操做 *.*表示的是全部数据库全部表
其具体含义是:赋予192.168.68.0网段的主机(服务器)复制(同步)主服务器的全部数据库数据;
1)修改主配置文件并重启服务
vim /etc/my.cnf #配置以下 log-bin = mysql-bin server-id = 2 relay-log = relay-log-bin #中继日志 relay-log-index = slave-relay-log-bin.index #索引文件 #保存退出重启 systemctl restart mysqld.service
2)进入数据库配置同步
[root@slave1 ~]# mysql -uroot -p ...#省略部份内容 mysql> change master to master_host='192.168.68.133',master_user='myslave',master_password='123456',master_log_file='master-bin.000001',master_log_pos=603; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.68.133 Master_User: myslave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 603 Relay_Log_File: relay-log-bin.000002 Relay_Log_Pos: 321 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes ...#省略部份内容 上面的两行表示从服务器给出的IO线程和数据库SQL语句都在运行状态 Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
在主服务器上建立一个新的数据库,新建表和数据;在服务器上进行查看;
主服务器:
create database test; Query OK, 1 row affected (0.00 sec)
从服务上slave1查询:
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> exit Bye [root@slave1 ~]#
从服务器slave2上查询:
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> exit Bye [root@slave2 ~]#
该配置实现的是从服务器经过主服务器给予的权限进行复制操做,咱们须要经过实验配置理解上述的主从复制的原理以及其复制的过程。只有在理解掌握了如何进行MySQL数据库的主从复制,才能理解和进行MySQL读写分离的配置操做。
读写分离的原理,简单来讲就是实现上图中,在主服务器上写数据,使用从服务器轮循读取数据的功能。
咱们结合下图来理解读写分离的过程
基于代理层实现:代理通常位于客户端与服务器之间,代理服务器接到客户端请求经过判断后转发到后端数据库,有两个表明性程序。
1)MySQL_Proxy:经过自带的lua脚本进行SQL判断;
2)Amoeba:Java语言开发,阿里巴巴将其用于生产环境,不支持事务与存储过程;
环境:基于上面主从复制的流程进行后续陪配置
添加设备:Centos7两台:其中一台做为Amoeba代理服务器,另外一台做为客户端测试服务器。
所需软件包:在Amoeba代理服务器上须要安装jdk和amoeba相关环境及应用(amoeba使用Java开发)
1)安装jdk环境和ameoba环境
[root@amoeba ~]# cd tar/ [root@amoeba tar]# ls amoeba-mysql-binary-2.2.0.tar.gz jdk-6u14-linux-x64.bin
cd tar/ ls cp jdk-6u14-linux-x64.bin /usr/local/ cd /usr/local/ chmod +x jdk-6u14-linux-x64.bin ./jdk-6u14-linux-x64.bin mv jdk1.6.0_14/ /usr/local/jdk1.6 vim /etc/profile #添加以下几行 export JAVA_HOME=/usr/local/jdk1.6 export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib export PATH=$JAVA_HOME/lib:$JAVA_HOME/jre/bin/:$PATH:$HOME/bin export AMOEBA_HOME=/usr/local/amoeba export PATH=$PATH:$AMOEBA_HOME/bin source /etc/profile mkdir /usr/local/amoeba #回到压缩包目录下解压amoeba tar zxf amoeba-mysql-binary-2.2.0.tar.gz -C /usr/local/amoeba cd /usr/local/
2)配置amoeba相关参数
配置路径在/usr/local/amoeba/conf中,配置文件为ameoba.xml,dbServers.xml。前者为amoeba主配置文件(还记得tomcat吗?),后者是数据库服务器的相关配置文件。
一、更改ameoba.xml
30 <property name="user">amoeba</property> 31 32 <property name="password">123456</property> #user是咱们在client端登陆amoeba代理服务器的身份名称,password是登陆密码,待会咱们将会在client端上登陆 15 <property name="defaultPool">master</property> 116 117 <property name="writePool">master</property> 118 <property name="readPool">slaves</property> 119 #defaultPool表示默认的服务器 writePool表示指定写服务器(组),readPool表示指定读服务器(组),slaves将在dbServers.xml文件中配置
二、更改dbServers.xml
26 <property name="user">test</property> 27 28 <!-- mysql password --> 29 <property name="password">123123</property> 45 <dbServer name="master" parent="abstractServer"> 46 <factoryConfig> 47 <!-- mysql ip --> 48 <property name="ipAddress">192.168.68.133</property> 49 </factoryConfig> 50 </dbServer> 51 52 <dbServer name="slave1" parent="abstractServer"> 53 <factoryConfig> 54 <!-- mysql ip --> 55 <property name="ipAddress">192.168.68.129</property> 56 </factoryConfig> 57 </dbServer> 58 59 <dbServer name="slave2" parent="abstractServer"> 60 <factoryConfig> 61 <!-- mysql ip --> 62 <property name="ipAddress">192.168.68.132</property> 63 </factoryConfig> 64 </dbServer> 65 66 <dbServer name="slaves" virtual="true"> 67 <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool"> 68 <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA--> 69 <property name="loadbalance">1</property> 70 71 <!-- Separated by commas,such as: server1,server2,server1 --> 72 <property name="poolNames">slave1,slave2</property> 73 </poolConfig> 74 </dbServer>
三、开启amoeba(能够在另外一个terminal查看端口netstat -natp | grep 8066)
[root@amoeba bin]# amoeba start & [1] 121608 [root@amoeba bin]# log4j:WARN log4j config load completed from file:/usr/local/amoeba/conf/log4j.xml 2020-01-10 08:20:03,413 INFO context.MysqlRuntimeContext - Amoeba for Mysql current versoin=5.1.45-mysql-amoeba-proxy-2.2.0 log4j:WARN ip access config load completed from file:/usr/local/amoeba/conf/access_list.conf 2020-01-10 08:20:03,566 INFO net.ServerableConnectionManager - Amoeba for Mysql listening on 0.0.0.0/0.0.0.0:8066. 2020-01-10 08:20:03,567 INFO net.ServerableConnectionManager - Amoeba Monitor Server listening on /127.0.0.1:39466.
四、客户机client端上安装mysql数据库,登陆查看
[root@localhost ~]# mysql -u amoeba -p123456 -h 192.168.68.144 -P8066 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2023306452 Server version: 5.1.45-mysql-amoeba-proxy-2.2.0 Source distribution Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> use test; Database changed mysql> show tables; Empty set (0.01 sec) #由于此时咱们尚未建立如何数据表和数据,因此在客户端上是查看不到任何数据的。
五、咱们在主服务器上建立一个表而且在从服务器上查看是否存在这个表,存在则主从复制正常
主服务器:
mysql> use test; Database changed mysql> create table info (id int(5) not null primary key auto_increment,name varchar(10) not null); Query OK, 0 rows affected (0.02 sec) mysql> desc info; +-------+-------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+----------------+ | id | int(5) | NO | PRI | NULL | auto_increment | | name | varchar(10) | NO | | NULL | | +-------+-------------+------+-----+---------+----------------+ 2 rows in set (0.01 sec) mysql> select * from info; Empty set (0.00 sec)
从服务器:
[root@slave1 ~]# mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 14 Server version: 5.7.17-log Source distribution Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | test | +--------------------+ 5 rows in set (0.01 sec) mysql> use test Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +----------------+ | Tables_in_test | +----------------+ | info | +----------------+ 1 row in set (0.00 sec)
六、在client端上查看这个表
mysql> show tables; Empty set (0.01 sec) mysql> show tables; +----------------+ | Tables_in_test | +----------------+ | info | +----------------+ 1 row in set (0.00 sec)
七、那么咱们如何验证读写分离呢?还记得咱们在从服务器上的start slave命令吗?咱们能够中止主从复制,使用stop slave便可
mysql> stop slave; Query OK, 0 rows affected (0.01 sec)
八、咱们在client端插入新的数据,分别在主从服务器上进行查看
client端:
mysql> insert into info (name) values('zhangsan'); Query OK, 1 row affected (0.01 sec) mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 2 | zhangsan | +----+----------+ 1 row in set (0.00 sec)
主服务器:
mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 2 | zhangsan | +----+----------+ 1 row in set (0.00 sec)
从服务器:
mysql> select * from info; Empty set (0.00 sec)
从以上的实验中,咱们能够得出结论,通过主从复制与写分离的配置,咱们能够将写数据的任务在主服务器上进行,主服务器写入完成后,由从服务器通过主从复制的过程进行(start slave)复制,由于在生产环境中咱们不能stop slave(中止主从复制)的!!!。可是为了验证读写分离原理,咱们只能先stop slave了。
下面验证读分离。
咱们在从服务器上分别写入一些数据(两台从服务器上写的不同)
slave1:
mysql> insert into info (name) values('lisi'); Query OK, 1 row affected (0.00 sec) mysql> select * from info; +----+------+ | id | name | +----+------+ | 1 | lisi | +----+------+ 1 row in set (0.00 sec)
slave2:
mysql> insert into info (name) values('wangwu'); Query OK, 1 row affected (0.00 sec) mysql> select * from info; +----+--------+ | id | name | +----+--------+ | 1 | wangwu | +----+--------+ 1 row in set (0.00 sec)
client 查询:(可能会有问题,由于是采用了auto_increment自增列)
所以咱们在从服务器上开启复制功能,在主从服务器上修改字段,而后在从服务器上关闭复制功能,而且从新写入不一样的数据,在client端查看。
两台从服务器:
mysql> start slave; Query OK, 0 rows affected (0.01 sec)
主服务器:
mysql> alter table info modify id int(5) not null; Query OK, 1 row affected (0.02 sec) Records: 1 Duplicates: 0 Warnings: 0 mysql> desc info; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(5) | NO | PRI | NULL | | | name | varchar(10) | NO | | NULL | | +-------+-------------+------+-----+---------+-------+ 2 rows in set (0.00 sec) mysql> delete from info; Query OK, 1 row affected (0.00 sec)
从服务器:
mysql> alter table info modify id int(5) not null; Query OK, 1 row affected (0.02 sec) Records: 1 Duplicates: 0 Warnings: 0 mysql> desc info; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(5) | NO | PRI | NULL | | | name | varchar(10) | NO | | NULL | | +-------+-------------+------+-----+---------+-------+ 2 rows in set (0.00 sec) mysql> delete from info; Query OK, 1 row affected (0.00 sec)
client :从新写入数据
mysql> insert into info (id,name) values(1,'zhangsan'); Query OK, 1 row affected (0.01 sec)
主服务器:
mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | +----+----------+ 1 row in set (0.00 sec) #此时须要从新查看状态记录position,此时以下: mysql> show master status; +-------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +-------------------+----------+--------------+------------------+-------------------+ | master-bin.000001 | 2955 | | | | +-------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
从服务器:
mysql> change master to master_host='192.168.68.133',master_user='myslave',master_password='123456',master_log_file='master-bin.000001',master_log_pos=2955; ERROR 3021 (HY000): This operation cannot be performed with a running slave io thread; run STOP SLAVE IO_THREAD FOR CHANNEL '' first. mysql> STOP SLAVE IO_THREAD FOR CHANNEL ''; Query OK, 0 rows affected (0.00 sec) mysql> stop slave; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> change master to master_host='192.168.68.133',master_user='myslave',master_password='123456',master_log_file='master-bin.000001',master_log_pos=2955; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.68.133 Master_User: myslave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 2955 Relay_Log_File: relay-log-bin.000002 Relay_Log_Pos: 321 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 2955 Relay_Log_Space: 526 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 5bb93767-328a-11ea-820a-000c290bd936 Master_Info_File: /usr/local/mysql/data/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
算了,咱们将三个服务器上的数据表的数据都删了从新进行主从复制吧;按照主从复制执行以后继续进行读分离的操做;
关闭从服务器主从复制:
mysql> stop slave; Query OK, 0 rows affected (0.00 sec)
从服务器上写入不一样数据:
slave1:
mysql> insert into info values(2,'lisi'); Query OK, 1 row affected (0.00 sec) mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 2 | lisi | +----+----------+ 2 rows in set (0.00 sec)
slave2:
mysql> insert into info values(3,'wangwu'); Query OK, 1 row affected (0.00 sec) mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 3 | wangwu | +----+----------+ 2 rows in set (0.00 sec)
client:
mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 2 | lisi | +----+----------+ 2 rows in set (0.00 sec) mysql> select * from info; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 3 | wangwu | +----+----------+ 2 rows in set (0.00 sec)
MySQL读分离则是在MySQL的从服务器上进行数据读取操做。其中两台作负载均衡,轮循读取,减轻压力,提升并发访问。
本文主要介绍了MySQL主从复制与读写分离原理,以及如何配置和验证主从复制与读写分离的实验。
经过实验咱们能够得出如下结论:在生产环境中,咱们不会将从服务器中止复制同步功能,而且不会在从服务上写数据,由于这样会致使某些问题,若是在从服务器上写了与主服务器冲突的数据,主服务器不会同步从的,主服务器上与从服务器所冲突的数据不会同步给从服务器。