amoeba实现mysql读写分离

时间 2019-11-06

原文原文链接

1、Amoeba 是什么：具备负载均衡、高可用性、sql过滤、读写分离、可路由相关的query到目标数据库、可并发请求多台数据库合并结果。html

2、Amoeba的安装
amoeba:192.168.2.203
masterDB：192.168.2.204
slaveDB：192.168.2.205
以上系统全为centos6.8java

Amoeba框架是居于JDK1.5开发的，采用了JDK1.5的特性，因此还须要安装java环境，建议使用javaSE1.5以上的JDK版本mysql

一、安装java环境linux

先去官网下载：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html算法

[root@bogon src]# rpm -ivh jdk-8u111-linux-x64.rpm
Preparing...                ########################################### [100%]
   1:jdk1.8.0_111           ########################################### [100%]
Unpacking JAR files...
    tools.jar...
    plugin.jar...
    javaws.jar...
    deploy.jar...
    rt.jar...
    jsse.jar...
    charsets.jar...
    localedata.jar...

而后设置java环境变量sql

[root@bogon src]# vim /etc/profile

#set java environment
JAVA_HOME=/usr/java/jdk1.8.0_111
JRE_HOME=/usr/java/jdk1.8.0_111/jre
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
export JAVA_HOME JRE_HOME CLASS_PATH PATH
[root@bogon amoeba]# source /etc/profile

测试是否安装成功数据库

[root@bogon src]# java -version
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

二、安装Amoeba
能够从https://sourceforge.net/projects/amoeba/下载最新版本的Amoeba，我这里下载的是amoeba-mysql-3.0.5-RC-distribution.zip。Amoeba安装很是简单，直接解压便可使用，这里将Amoeba解压到/usr/local/amoeba目录下，这样就安装完成了vim

[root@bogon amoeba]# pwd
/usr/local/amoeba
  [root@bogon amoeba]# ll
  总用量 20
  drwxrwxrwx. 2 root root 4096 7月 5 2013 benchmark
  drwxrwxrwx. 2 root root 4096 7月 5 2013 bin
  drwxrwxrwx. 2 root root 4096 7月 5 2013 conf
  -rwxrwxrwx. 1 root root 728 7月 5 2013 jvm.properties
  drwxrwxrwx. 2 root root 4096 7月 5 2013 lib

三、配置Amoeba后端

Amoeba的配置文件在本环境下位于/usr/local/amoeba/conf目录下。配置文件比较多，可是仅仅使用读写分离功能，只需配置两个文件便可，分别是dbServers.xml和amoeba.xml，若是须要配置ip访问控制，还须要修改access_list.conf文件，下面首先介绍dbServers.xmlcentos

[root@bogon amoeba]# cat conf/dbServers.xml 
<?xml version="1.0" encoding="gbk"?>

<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd">
<amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/">

        <!-- 
            Each dbServer needs to be configured into a Pool,
            If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration:
             add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig
             such as 'multiPool' dbServer   
        -->

    <dbServer name="abstractServer" abstractive="true">
        <factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">
            <property name="connectionManager">${defaultManager}</property>
            <property name="sendBufferSize">64</property>
            <property name="receiveBufferSize">128</property>

            <!-- mysql port -->
            <property name="port">3306</property>　　#设置Amoeba要链接的mysql数据库的端口，默认是3306

            <!-- mysql schema -->
            <property name="schema">testdb</property>　　#设置缺省的数据库，当链接amoeba时，操做表必须显式的指定数据库名，即采用dbname.tablename的方式，不支持 use dbname指定缺省库，由于操做会调度到各个后端dbserver

            <!-- mysql user -->
            <property name="user">test1</property>　　#设置amoeba链接后端数据库服务器的帐号和密码，所以须要在全部后端数据库上建立该用户，并受权amoeba服务器可链接

            <property name="password">111111</property>
        </factoryConfig>

        <poolConfig class="com.meidusa.toolkit.common.poolable.PoolableObjectPool">
            <property name="maxActive">500</property>　　#最大链接数，默认500
            <property name="maxIdle">500</property>　　　　#最大空闲链接数
            <property name="minIdle">1</property>　　　　#最新空闲链接数
            <property name="minEvictableIdleTimeMillis">600000</property>
            <property name="timeBetweenEvictionRunsMillis">600000</property>
            <property name="testOnBorrow">true</property>
            <property name="testOnReturn">true</property>
            <property name="testWhileIdle">true</property>
        </poolConfig>
    </dbServer>

    <dbServer name="writedb"  parent="abstractServer">　　#设置一个后端可写的dbServer，这里定义为writedb，这个名字能够任意命名，后面还会用到
        <factoryConfig>
            <!-- mysql ip -->
            <property name="ipAddress">192.168.2.204</property> #设置后端可写dbserver
        </factoryConfig>
    </dbServer>

    <dbServer name="slave"  parent="abstractServer">　　#设置后端可读dbserver
        <factoryConfig>
            <!-- mysql ip -->
            <property name="ipAddress">192.168.2.205</property>
        </factoryConfig>
    </dbServer>

    <dbServer name="myslave" virtual="true">　　#设置定义一个虚拟的dbserver，实际上至关于一个dbserver组，这里将可读的数据库ip统一放到一个组中，将这个组的名字命名为myslave
        <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
            <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
            <property name="loadbalance">1</property>　　#选择调度算法，1表示复制均衡，2表示权重，3表示HA， 这里选择1

            <!-- Separated by commas,such as: server1,server2,server1 -->
            <property name="poolNames">slave</property>　　#myslave组成员
        </poolConfig>
    </dbServer>

</amoeba:dbServers>

另外一个配置文件amoeba.xml

[root@bogon amoeba]# cat conf/amoeba.xml 
<?xml version="1.0" encoding="gbk"?>

<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">
<amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">

    <proxy>

        <!-- service class must implements com.meidusa.amoeba.service.Service -->
        <service name="Amoeba for Mysql" class="com.meidusa.amoeba.mysql.server.MySQLService">
            <!-- port -->
            <property name="port">8066</property>　　　　#设置amoeba监听的端口，默认是8066

            <!-- bind ipAddress -->　　　　#下面配置监听的接口，若是不设置，默认监听因此的IP
            <!-- 
            <property name="ipAddress">127.0.0.1</property>
             -->

            <property name="connectionFactory">
                <bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
                    <property name="sendBufferSize">128</property>
                    <property name="receiveBufferSize">64</property>
                </bean>
            </property>

            <property name="authenticateProvider">
                <bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">

# 提供客户端链接amoeba时须要使用这里设定的帐号 (这里的帐号密码和amoeba链接后端数据库服务器的密码无关)
                    <property name="user">root</property>　　　　

                    <property name="password">123456</property>

                    <property name="filter">
                        <bean class="com.meidusa.toolkit.net.authenticate.server.IPAccessController">
                            <property name="ipFile">${amoeba.home}/conf/access_list.conf</property>
                        </bean>
                    </property>
                </bean>
            </property>

        </service>

        <runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">

            <!-- proxy server client process thread size -->
            <property name="executeThreadSize">128</property>

            <!-- per connection cache prepared statement size  -->
            <property name="statementCacheSize">500</property>

            <!-- default charset -->
            <property name="serverCharset">utf8</property>

            <!-- query timeout( default: 60 second , TimeUnit:second) -->
            <property name="queryTimeout">60</property>
        </runtime>

    </proxy>

    <!-- 
        Each ConnectionManager will start as thread
        manager responsible for the Connection IO read , Death Detection
    -->
    <connectionManagerList>
        <connectionManager name="defaultManager" class="com.meidusa.toolkit.net.MultiConnectionManagerWrapper">
            <property name="subManagerClassName">com.meidusa.toolkit.net.AuthingableConnectionManager</property>
        </connectionManager>
    </connectionManagerList>

        <!-- default using file loader -->
    <dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">
        <property name="configFile">${amoeba.home}/conf/dbServers.xml</property>
    </dbServerLoader>

    <queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">
        <property name="ruleLoader">
            <bean class="com.meidusa.amoeba.route.TableRuleFileLoader">
                <property name="ruleFile">${amoeba.home}/conf/rule.xml</property>
                <property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>
            </bean>
        </property>
        <property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>
        <property name="LRUMapSize">1500</property>
        <property name="defaultPool">writedb</property>　　#设置amoeba默认的池，这里设置为writedb

        <property name="writePool">writedb</property>　　#这两个选项默认是注销掉的，须要取消注释，这里用来指定前面定义好的俩个读写池
        <property name="readPool">myslave</property>　　 #

        <property name="needParse">true</property>
    </queryRouter>
</amoeba:configuration>

四、在masterdb上建立数据库testdb

mysql> create database testdb;
Query OK, 1 row affected (0.08 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mydb               |
| mysql              |
| performance_schema |
| test               |
| testdb             |
+--------------------+
6 rows in set (0.00 sec)

查看slavedb是否复制成功

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mydb               |
| mysql              |
| performance_schema |
| test               |
| testdb             |
+--------------------+
6 rows in set (0.00 sec)

分别在masterdb和slavedb上为amoedb受权

mysql> GRANT ALL ON testdb.* TO 'test1'@'192.168.2.203' IDENTIFIED BY '111111';
Query OK, 0 rows affected (0.05 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.02 sec)

启动amoeba

[root@bogon amoeba]# /usr/local/amoeba/bin/launcher
Error: JAVA_HOME environment variable is not set.
[root@bogon amoeba]# vim /etc/profile^C
[root@bogon amoeba]# source /etc/profile
[root@bogon amoeba]# /usr/local/amoeba/bin/launcher
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=16m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=96m; support was removed in 8.0

The stack size specified is too small, Specify at least 228k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

报错:

Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
从错误文字上看，应该是因为stack size过小，致使JVM启动失败，要如何修改呢？
其实Amoeba已经考虑到这个问题，并将JVM参数配置写在属性文件里。如今，让咱们经过该属性文件修改JVM参数。
修改jvm.properties文件JVM_OPTIONS参数。

[root@bogon amoeba]# vim /usr/local/amoeba/jvm.properties 
改为：JVM_OPTIONS="-server -Xms1024m -Xmx1024m -Xss256k -XX:PermSize=16m -XX:MaxPermSize=96m"
原为：JVM_OPTIONS="-server -Xms256m -Xmx1024m -Xss196k -XX:PermSize=16m -XX:MaxPermSize=96m"

再次启动

[root@bogon ~]# /usr/local/amoeba/bin/launcher
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
    at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:127)
    at org.codehaus.classworlds.Launcher.main(Launcher.java:110)
Caused by: com.meidusa.toolkit.common.bean.util.InitialisationException: default pool required!,defaultPool=writedb invalid
    at com.meidusa.amoeba.route.AbstractQueryRouter.init(AbstractQueryRouter.java:469)
    at com.meidusa.amoeba.context.ProxyRuntimeContext.initAllInitialisableBeans(ProxyRuntimeContext.java:337)
    ... 11 more
 2016-10-24 18:46:37 [INFO] Project Name=Amoeba-MySQL, PID=1577 , System shutdown ....
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=16m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=96m; support was removed in 8.0
 2016-10-24 18:50:19 [INFO] Project Name=Amoeba-MySQL, PID=1602 , starting...
log4j:WARN log4j config load completed from file:/usr/local/amoeba/conf/log4j.xml
2016-10-24 18:50:21,668 INFO  context.MysqlRuntimeContext - Amoeba for Mysql current versoin=5.1.45-mysql-amoeba-proxy-3.0.4-BETA
log4j:WARN ip access config load completed from file:/usr/local/amoeba/conf/access_list.conf
2016-10-24 18:50:22,852 INFO  net.ServerableConnectionManager - Server listening on 0.0.0.0/0.0.0.0:8066.

查看端口

[root@bogon ~]# netstat -unlpt | grep java
tcp        0      0 :::8066                     :::*                        LISTEN      1602/java

由此可知Amoeba启动正常