目前较为常见的mysql读写分离分为两种:
一、
基于程序代码内部实现:在代码中对select操做分发到从库;其它操做由主库执行;这类方法也是目前生产环境应用最普遍,知名的如DISCUZ
X2。优势是性能较好,由于在程序代码中实现,不须要增长额外的设备做为硬件开支。缺点是须要开发人员来实现,运维人员无从下手。 java
二、 基于中间代理层实现:咱们都知道代理通常是位于客户端和服务器之间,代理服务器接到客户端请求后经过判断而后转发到后端数据库。在这有两个表明性程序 mysql
mysql-proxy:mysql-proxy为mysql开源项目,经过其自带的lua脚本进行sql判断,虽然是mysql官方产品,可是mysql官方并不建议将mysql-proxy用到生产环境。
amoeba:由陈思儒开发,做者曾就任于阿里巴巴,现就任于盛大。该程序由java语言进行开发,目前只据说阿里巴巴将其用于生产环境。另外,此项目严重缺乏维护和推广(做者有个官方博客,不少用户反馈的问题发现做者不理睬)
通过上述简单的比较,经过程序代码实现mysql读写分离天然是一个不错的选择。可是并非全部的应用都适合在程序代码中实现读写分离,像大型SNS、B2C这类应用能够在代码中实现,由于这样对程序代码自己改动较小;像一些大型复杂的java应用,这种类型的应用在代码中实现对代码改动就较大了。因此,像这种应用通常就会考虑使用代理层来实现。 sql
下面咱们看一下如何搭建mysql-proxy来实现mysql读写分离
环境拓扑以下:
关于mysql、mysql主从的搭建,在此再也不演示,以下的操做均在mysql-proxy(192.168.1.200)服务器进行
1、安装mysql-proxy
一、安装lua (mysql-proxy须要使用lua脚本进行数据转发)
#tar zxvf
lua-5.1.4.tar.gz
#cd lua-5.1.4
#vi Makefile,修改INSTALL_TOP=
/usr/local/lua
#make posix
#make install
二、安装libevent
#tar
zxvf libevent-2.0.8-rc.tar.gz
#cd libevent-2.0.8-rc
#./configure
--prefix=/usr/local/libevent
#make && make install
三、安装check
#tar zxvf check-0.9.8.tar.gz
#cd check-0.9.8
#./configure && make && make install
四、安装mysql客户端
#tar zxvf mysql-5.0.92.tar.gz
#cd mysql-5.0.92
#./configure
--without-server && make && make install
五、设置环境变量
(安装mysql-proxy所需变量)
#vi /etc/profile
export
LUA_CFLAGS="-I/usr/local/lua/include" LUA_LIBS="-L/usr/local/lua/lib -llua -ldl"
LDFLAGS="-L/usr/local/libevent/lib -lm"
export
CPPFLAGS="-I/usr/local/libevent/include"
export
CFLAGS="-I/usr/local/libevent/include"
# source /etc/profile
六、安装mysql-proxy
#tar zxvf mysql-proxy-0.6.0.tar.gz
#cd
mysql-proxy-0.6.0
# ./configure --prefix=/usr/local/mysql-proxy --with-mysql
--with-lua
#make && make install
七、启动mysql-proxy
本次对两台数据库实现了读写分离;mysql-master为可读可写,mysql-slave为只读
#/usr/local/mysql-proxy/sbin/mysql-proxy
--proxy-backend-addresses=192.168.1.201:3306
--proxy-read-only-backend-addresses=192.168.1.202:3306
--proxy-lua-script=/usr/local/mysql-proxy/share/mysql-proxy/rw-splitting.lua
&
注:若是正常状况下启动后终端不会有任何提示信息,mysql-proxy启动后会启动两个端口4040和4041,4040用于SQL转发,4041用于管理mysql-proxy。若有多个mysql-slave能够依次在后面添加
2、测试
一、链接测试
由于默认状况下mysql数据库不容许用户在远程链接
mysql>grant
all privileges on *.* to identified by '123456';
mysql>flush privileges;
客户端链接
#mysql -uroot -p123456 -h192.168.1.200 -P4040
二、读写分离测试
为了测试出mysql读写分离的真实性,在测试以前,须要开启两台mysql的log功能,而后在mysql-slave服务器中止复制
① 在两台mysql配置文件my.cnf中加入log=query.log,而后重启 数据库
②在mysql-slave上执行SQL语句stop slave 后端
③在两台mysql上执行#tail -f /usr/local/mysql/var/query.log 服务器
④在客户端上链接mysql(三个链接以上),而后执行create、select等SQL语句,观察两台mysql的日志有何变化
注:生产环境中除了进行程序调试外,其它不要开启mysql查询日志,由于查询日志记录了客户端的全部语句,频繁的IO操做将会致使mysql总体性能降低
总结:在上述环境中,mysql-proxy和mysql-master、mysql-slave三台服务器均存在单点故障。若是在可用性要求较高的场合,单点隐患是绝对不容许的。为了不mysql-proxy单点隐患有两种方法,一种方法是mysql-proxy配合keepalived作双机,另外一种方法是将mysql-proxy和应用服务安装到同一台服务器上;为了不mysql-master单点故障可使用DRBD+heartbear作双机;避免mysql-slave单点故障增长多台mysql-slave便可,由于mysql-proxy会自动屏蔽后端发生故障的mysql-slave。 app
附: mysql-proxy LUA 读写分离脚本代码: 运维
--[[
--
-- author : KDr2
-- version 0.01
-- SYNOPSIS:
---
1.维护了一个链接池
--- 2.读写分离,简单的将select开头的语句放到slave上执行
---
3.事务支持,全部事务放到master上执行,事务中不更改链接
--- 4.简单日志
--
--]] ide
--- config vars
local min_idle_connections = 4
local
max_idle_connections = 8
local log_level=1
local encoding="utf8"
---
end of config 性能
-- 事务标识,在事务内不归还链接
local
transaction_flags={}
setmetatable(transaction_flags,{__index=function()
return 0 end})
-- log system
log={
level={debug=1,info=2,warn=3,error=4},
funcs={"debug","info","warn","error"},
}
function log.log(level,m)
if level >= log_level then
local msg="[" .. os.date("%Y-%m-%d %X")
.."] ".. log.funcs[level] .. ": " .. tostring(m)
print(msg) -- TODO
write msg into a log file.
end
end
for i,v in ipairs(log.funcs)
do
log[v]=function(m) log.log(log.level[v],m) end
end
-- connect to server
function connect_server()
log.info(" starting
connect_server ... ")
local least_idle_conns_ndx = 0
local
least_idle_conns = 0
for i = 1, #proxy.backends do
local s
= proxy.backends[i]
local pool = s.pool
local cur_idle =
pool.users[""].cur_idle_connections
log.debug("[".. s.address .."].connected_clients = " ..
s.connected_clients)
log.debug("[".. s.address .."].idling_connections
= " .. cur_idle)
log.debug("[".. s.address .."].type = " ..
s.type)
log.debug("[".. s.address .."].state = " .. s.state)
if s.state ~= proxy.BACKEND_STATE_DOWN then
-- try to
connect to each backend once at least
if cur_idle == 0
then
proxy.connection.backend_ndx = i
log.info("server [".. proxy.backends[i].address .."] open new
connection")
return
end
-- try to open at
least min_idle_connections
if least_idle_conns_ndx == 0
or
( cur_idle < min_idle_connections and
cur_idle < least_idle_conns ) then
least_idle_conns_ndx =
i
least_idle_conns = cur_idle
end
end
end
if least_idle_conns_ndx > 0 then
proxy.connection.backend_ndx
= least_idle_conns_ndx
end
if proxy.connection.backend_ndx
> 0 then
local s =
proxy.backends[proxy.connection.backend_ndx]
local pool = s.pool
local cur_idle = pool.users[""].cur_idle_connections
if cur_idle >= min_idle_connections then
-- we have 4
idling connections in the pool, that's good enough
log.debug("using
pooled connection from: " .. proxy.connection.backend_ndx)
return
proxy.PROXY_IGNORE_RESULT
end
end
-- open a new connection
log.info("opening new connection on: " ..
proxy.backends[proxy.connection.backend_ndx].address)
end
---
-- auth.packet is the packet
function read_auth_result( auth )
if
auth.packet:byte() == proxy.MYSQLD_PACKET_OK then
-- 链接正常
proxy.connection.backend_ndx = 0
elseif auth.packet:byte() ==
proxy.MYSQLD_PACKET_EOF then
-- we received either a
-- *
MYSQLD_PACKET_ERR and the auth failed or
-- * MYSQLD_PACKET_EOF which
means a OLD PASSWORD (4.0) was sent
log.error("(read_auth_result) ...
not ok yet");
elseif auth.packet:byte() == proxy.MYSQLD_PACKET_ERR
then
log.error("auth failed!")
end
end
---
-- read/write splitting
function read_query( packet )
log.debug("[read_query]")
log.debug("authed backend = " ..
proxy.connection.backend_ndx)
log.debug("used db = " ..
proxy.connection.client.default_db)
if packet:byte() == proxy.COM_QUIT then
proxy.response =
{
type = proxy.MYSQLD_PACKET_OK,
}
return
proxy.PROXY_SEND_RESULT
end
if proxy.connection.backend_ndx == 0 then
local
is_read=(string.upper(packet:sub(2))):match("^SELECT")
local
target_type=proxy.BACKEND_TYPE_RW
if is_read then
target_type=proxy.BACKEND_TYPE_RO end
for i = 1, #proxy.backends
do
local s = proxy.backends[i]
local pool = s.pool
local cur_idle =
pool.users[proxy.connection.client.username].cur_idle_connections
if cur_idle > 0 and
s.state ~=
proxy.BACKEND_STATE_DOWN and
s.type == target_type
then
proxy.connection.backend_ndx = i
break
end
end
end
-- sync the client-side
default_db with the server-side default_db
if proxy.connection.server and
proxy.connection.client.default_db ~= proxy.connection.server.default_db
then
local server_db=proxy.connection.server.default_db
local
client_db=proxy.connection.client.default_db
local default_db=
(#client_db > 0) and client_db or server_db
if #default_db > 0
then
proxy.queries:append(2, string.char(proxy.COM_INIT_DB) ..
default_db)
proxy.queries:append(2, string.char(proxy.COM_QUERY) ..
"set names '" .. encoding .."'")
log.info("change database to " ..
default_db);
end
end
if proxy.connection.backend_ndx > 0
then
log.debug("Query[" .. packet:sub(2) .. "] Target is [" ..
proxy.backends[proxy.connection.backend_ndx].address .."]")
end
proxy.queries:append(1, packet)
return proxy.PROXY_SEND_QUERY
end
---
-- as long as we are in a transaction keep the connection
--
otherwise release it so another client can use it
function read_query_result(
inj )
local res = assert(inj.resultset)
local flags =
res.flags
if inj.id ~= 1 then
-- ignore the result of the USE
<default_db>
return proxy.PROXY_IGNORE_RESULT
end
is_in_transaction = flags.in_trans
if flags.in_trans then
transaction_flags[proxy.connection.server.thread_id] =
transaction_flags[proxy.connection.server.thread_id] + 1
elseif
inj.query:sub(2):lower():match("^%s*commit%s*$") or
inj.query:sub(2):lower():match("^%s*rollback%s*$") then
transaction_flags[proxy.connection.server.thread_id] =
transaction_flags[proxy.connection.server.thread_id] - 1
if
transaction_flags[proxy.connection.server.thread_id] < 0 then
transaction_flags[proxy.connection.server.thread_id] = 0 end
end
log.debug("transaction res : " ..
tostring(transaction_flags[proxy.connection.server.thread_id]));
if
transaction_flags[proxy.connection.server.thread_id]==0 or
transaction_flags[proxy.connection.server.thread_id] == nil then
--
isnot in a transaction, need to release the backend
proxy.connection.backend_ndx = 0
end
end
---
-- close the connections if we have enough connections in the
pool
--
-- @return nil - close connection
-- IGNORE_RESULT - store
connection in the pool
function disconnect_client()
log.debug("[disconnect_client]")
if proxy.connection.backend_ndx == 0
then
for i = 1, #proxy.backends do
local s =
proxy.backends[i]
local pool = s.pool
local cur_idle =
pool.users[proxy.connection.client.username].cur_idle_connections
if s.state ~= proxy.BACKEND_STATE_DOWN and
cur_idle
> max_idle_connections then
-- try to disconnect a
backend
proxy.connection.backend_ndx = i
log.info("[".. proxy.backends[i].address .."] closing connection, idling: " ..
cur_idle)
return
end
end
return
proxy.PROXY_IGNORE_RESULT
end
end