ProxySQL官档翻译__20_Mirroring
20_Mirroring
备注：文章编写时间201904-201905期间，后续官方在github的更新没有被写入mysql
~
~
镜像[Mirroring]
注意：
1）它包含的规则能够随时改变；
2）它不支持预备(prepare)语句；git
1、mysql_query_rules的扩展[Extensions to mysql_query_rules]
修改了 mysql_query_rules 表，添加了2个列：
1） mirror_flagOUT
2） mirror_hostgroupgithub
所以，mysql_query_rules 表的新的定义变为：正则表达式
Admin> show create table mysql_query_rules\G
*************************** 1. row ***************************
table: mysql_query_rules
Create Table: CREATE TABLE mysql_query_rules (
rule_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 0,
username VARCHAR,
schemaname VARCHAR,
flagIN INT CHECK (flagIN >= 0) NOT NULL DEFAULT 0,
client_addr VARCHAR,
proxy_addr VARCHAR,
proxy_port INT CHECK (proxy_port >= 0 AND proxy_port <= 65535), digest VARCHAR,
match_digest VARCHAR,
match_pattern VARCHAR,
negate_match_pattern INT CHECK (negate_match_pattern IN (0,1)) NOT NULL DEFAULT 0,
re_modifiers VARCHAR DEFAULT 'CASELESS',
flagOUT INT CHECK (flagOUT >= 0),
replace_pattern VARCHAR CHECK(CASE WHEN replace_pattern IS NULL THEN 1 WHEN replace_pattern IS NOT NULL AND match_pattern IS NOT NULL THEN 1 ELSE 0 END),
destination_hostgroup INT DEFAULT NULL,
cache_ttl INT CHECK(cache_ttl > 0),
cache_empty_result INT CHECK (cache_empty_result IN (0,1)) DEFAULT NULL,
cache_timeout INT CHECK(cache_timeout >= 0),
reconnect INT CHECK (reconnect IN (0,1)) DEFAULT NULL,
timeout INT UNSIGNED CHECK (timeout >= 0),
retries INT CHECK (retries>=0 AND retries <=1000),
delay INT UNSIGNED CHECK (delay >=0),
next_query_flagIN INT UNSIGNED,
mirror_flagOUT INT UNSIGNED,
mirror_hostgroup INT UNSIGNED,
error_msg VARCHAR,
OK_msg VARCHAR,
sticky_conn INT CHECK (sticky_conn IN (0,1)),
multiplex INT CHECK (multiplex IN (0,1,2)),
gtid_from_hostgroup INT UNSIGNED,
log INT CHECK (log IN (0,1)),
apply INT CHECK(apply IN (0,1)) NOT NULL DEFAULT 0,
comment VARCHAR)
1 row in set (0.00 sec)
2、功能实现概述[Implementation overview]
当为匹配查询的规则设置了 mirror_flagOUT 或 mirror_hostgroup 时，将自动启用镜像实时查询功能。sql
请注意：
若是在规则中设置了 mirror_flagOUT 或 mirror_hostgroup 时又设置了replace_pattern：也就是若是原始查询的文本内容(SQL语句)已被重写(即查询规则设置了replace_pattern对原始SQL进行重写/替换)，则会为最终执行的查询(被修改后的语句)启用镜像查询功能，而会为原始SQL启用镜像查询：即若是查询(SQL语句)在根据digest、match_digest或match_pattern匹配到了查询规则，却发现该查询规则设置了replace_pattern对匹配到的查询SQL进行重写，那么，这时镜像逻辑将应用于被重写后的查询(SQL)。虽然被镜像的查询(即原始SQL语句)能够被从新编写或修改，但只要设置了mirror_flagOUT 或 mirror_hostgroup就会启用镜像查询功能。详情在后面。服务器
若是源查询(SQL)与多个查询规则匹配，则可能会屡次更改 mirror_flagOUT 或 mirror_hostgroup 。app
镜像逻辑以下：
1）若是在处理源查询时设置了 mirror_flagOUT 或 mirror_hostgroup (查询规则设置的)，则会建立一个新的mysql会话。
2）新的mysql会话将得到原始mysql会话的全部相同属性：相同的凭据，库名，默认主机组等（注意：charset当前未被复制）
3）若是在原始会话中设置了 mirror_hostgroup ，则新会话将其默认主机组更改成 mirror_hostgroup 。
4）若是未设置 mirror_flagOUT ，则新会话将针对定义的 mirror_hostgroup 执行原始查询。
5）若是在原始会话中设置了 mirror_flagOUT ，则新的mysql会话将尝试根据原始会话的mirror_flagOUT的值在 mysql_query_rules 中查找FlagIN值与之相等的查询规则(即查找FlagIN=mirror_flagOUT的规则)； 而后将镜像查询请求发送到这个规则中进行处理：这样就能够修改查询，如重写查询，或再次更改 hostgroup 。
(参考后面的<7、高级示例：使用镜像测试查询重写>内容)dom
3、案例一：原始查询与镜像查询选择同一主机组[Mirror selects to same hostgroup]
在这个很是简单的示例中，咱们将把全部SELECT语句发送到hostgroup10，包括原始语句和镜像语句。ide
一、设置查询规则
1）设置查询规则
将原始查询与镜像查询指向同一主机组：性能
Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+------------------+-------+ | 5 | 1 | NULL | NULL | NULL | 1 | +---------+--------+---------------+-----------------------+------------------+-------+ 1 row in set (0.00 sec) Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply) VALUES (6,1,'^SELECT',10,10,1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+------------------+-------+ | 5 | 1 | NULL | NULL | NULL | 1 | | 6 | 1 | ^SELECT | 10 | 10 | 1 | +---------+--------+---------------+-----------------------+------------------+-------+ 2 rows in set (0.00 sec)
2）加载配置到RUNTIME层
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM runtime_mysql_query_rules ; +---------+--------+---------------+-----------------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+------------------+-------+ | 5 | 1 | NULL | NULL | NULL | 1 | | 6 | 1 | ^SELECT | 10 | 10 | 1 | +---------+--------+---------------+-----------------------+------------------+-------+ 2 rows in set (0.00 sec)
二、链接ProxySQL执行查询
从mysql会话中咱们将运行一些查询：
1）使用业务帐号连入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空并查看当前以执行SQL的查询概要统计：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 10 | 1 | sbtest | show databases | | 10 | 1 | sbtest | show tables | +-----------+------------+------------+--------------------------------+ 2 rows in set (0.00 sec)
3）业务帐号执行查询
mysql> SELECT id FROM sbtest1 LIMIT 3; +------+ | id | +------+ | 1891 | | 1511 | | 8032 | +------+ 3 rows in set (0.00 sec)
4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 10 | 2 | sbtest | SELECT id FROM sbtest1 LIMIT ? | +-----------+------------+------------+--------------------------------+ 1 row in set (0.00 sec)
咱们能够看到 SELECT 语句被执行了两次！！
5）做为附加测试，咱们从新运行相同的查询：
mysql> SELECT id FROM sbtest1 LIMIT 3; +------+ | id | +------+ | 1891 | | 1511 | | 8032 | +------+ 3 rows in set (0.00 sec
6）ProxySQL管理端，查看当前以执行SQL的查询概要统计：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 10 | 4 | sbtest | SELECT id FROM sbtest1 LIMIT ? | +-----------+------------+------------+--------------------------------+ 1 row in set (0.01 sec)
count_star是咱们执行查询次数的两倍，由于它是镜像的。值得注意的是，ProxySQL会收集原始查询和镜像查询的指标。
4、案例二：原始查询与镜像查询选择到不一样的主机组[Mirror SELECT to different hostgroup]
在此示例中，咱们将从新配置proxysql以将全部SELECT语句发送到hostgroup10，但要在hostgroup20上执行镜像查询：
一、设置查询规则
1）设置查询规则
Admin> DELETE FROM mysql_query_rules; Query OK, 2 rows affected (0.00 sec) Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply) VALUES (5,1,'^SELECT',10,20,1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+------------------+-------+ | 5 | 1 | ^SELECT | 10 | 20 | 1 | +---------+--------+---------------+-----------------------+------------------+-------+ 1 row in set (0.00 sec)
2）加载配置到RUNTIME层
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.00 sec)
二、链接ProxySQL执行查询
1）使用业务帐号连入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空stats_mysql_query_digest以得到新的统计信息：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+----------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+----------------+ | 10 | 1 | sbtest | show databases | | 10 | 1 | sbtest | show tabels | | 10 | 2 | sbtest | show tables | +-----------+------------+------------+----------------+ 3 rows in set (0.00 sec) Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; Empty set (0.00 sec)
3）业务帐号执行查询
从mysql客户端咱们如今能够运行一些查询（为简单起见，咱们运行相同）：
mysql> SELECT id FROM sbtest1 LIMIT 3; +------+ | id | +------+ | 1891 | | 1511 | | 8032 | +------+ 3 rows in set (0.00 sec)
4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 20 | 1 | sbtest | SELECT id FROM sbtest1 LIMIT ? | | 10 | 1 | sbtest | SELECT id FROM sbtest1 LIMIT ? | +-----------+------------+------------+--------------------------------+ 2 rows in set (0.00 sec)
能够看到ProxySQL向hostgroup10和hostgroup20发送了相同的相同查询！
5、案例三：重写源查询和镜像[rewrite both source query and mirror]
在这个例子中，咱们将重写原始查询，而后镜像它：为简单起见，咱们将对表 sbtest[0-9]+ 的操做重写为对表 sbtest3 的操做：
一、设置查询规则
1）设置查询规则
Admin> DELETE FROM mysql_query_rules; Query OK, 1 row affected (0.00 sec) Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,replace_pattern,mirror_hostgroup,apply) VALUES (5,1,'sbtest[0-9]+',10,'sbtest3',20,1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,replace_pattern,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+-----------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | replace_pattern | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+-----------------+------------------+-------+ | 5 | 1 | sbtest[0-9]+ | 10 | sbtest3 | 20 | 1 | +---------+--------+---------------+-----------------------+-----------------+------------------+-------+ 1 row in set (0.00 sec)
2）加载配置到RUNTIME层
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.00 sec)
二、链接ProxySQL执行查询
1）使用业务帐号连入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空stats_mysql_query_digest以得到新的统计信息：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+----------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+----------------+ | 10 | 1 | sbtest | show databases | | 10 | 2 | sbtest | show tables | +-----------+------------+------------+----------------+ 2 rows in set (0.00 sec) Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; Empty set (0.01 sec)
3）业务帐号执行查询
从mysql客户端咱们能够运行一般的查询：
mysql> SELECT id FROM sbtest1 LIMIT 3; +------+ | id | +------+ | 1149 | | 9825 | | 5704 | +------+ 3 rows in set (0.00 sec)
正如预期的那样，此次查询输出的结果与前一个不一样，由于如今重写了原始查询。表名是是查询了sbtest1其实被改写到了sbtest3;经过下面内容能够验证。
4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：
Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest; +-----------+------------+------------+--------------------------------+ | hostgroup | count_star | schemaname | digest_text | +-----------+------------+------------+--------------------------------+ | 20 | 1 | sbtest | SELECT id FROM sbtest3 LIMIT ? | | 10 | 1 | sbtest | SELECT id FROM sbtest3 LIMIT ? | +-----------+------------+------------+--------------------------------+ 2 rows in set (0.01 sec)
正如所料，修改后的查询在两个主机组上执行，并且原始SQL中的表名sbtest1被重写为了sbtest3。
6、仅重写镜像查询[rewrite mirror query only]
在此示例中，咱们将只重写镜像查询。
这很是有用，例如，咱们想要了解重写查询的性能，或者新索引是否会提升性能。
在此示例中，咱们将以使用和不使用索引来比较相同查询的性能。固然，咱们也会将查询发送到相同的主机组。
一、设置查询规则
1）设置查询规则
建立如下规则（rule_id = 5）：
1）匹配 FROM sbtest1 ;
2）设置destination_hostgroup = 20 ;
3）设置mirror_flagOUT=100 ;
4）不设置mirror_hostgroup ;
Admin> DELETE FROM mysql_query_rules; Query OK, 1 row affected (0.00 sec) Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,apply) VALUES (5,1,'FROM sbtest1 ',20,100,1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+---------------+-----------------------+----------------+------------------+-------+ | rule_id | active | match_pattern | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply | +---------+--------+---------------+-----------------------+----------------+------------------+-------+ | 5 | 1 | FROM sbtest1 | 20 | 100 | NULL | 1 | +---------+--------+---------------+-----------------------+----------------+------------------+-------+ 1 row in set (0.00 sec)
因为设置了 mirror_flagOUT ，所以将建立一个新会话来运行相同的查询。可是，由于未设置 mirror_hostgroup ，因此会根据当前用户在 mysql_users 中设置的默认主机组，将查询将被发送到其默认主机组上。相反的，若是咱们但愿将镜像查询发送到与原始主机组相同的主机组。咱们能够在rule_id = 5的规则中设置mirror_hostgroup，或者建立
一个新规则。这里，选择后者，咱们将建立一个新规则来匹配重写查询：
Admin> INSERT INTO mysql_query_rules (rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,apply) VALUES (10,1,100,'FROM sbtest1 ',20,'FROM sbtest1 IGNORE INDEX(k_1) ',1); Query OK, 1 row affected (0.00 sec) Admin> SELECT rule_id,active,flagIN,match_pattern,replace_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM mysql_query_rules ; +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ | rule_id | active | flagIN | match_pattern | replace_pattern | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply | +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ | 5 | 1 | 0 | FROM sbtest1 | NULL | 20 | 100 | NULL | 1 | | 10 | 1 | 100 | FROM sbtest1 | FROM sbtest1 IGNORE INDEX(k_1) | 20 | NULL | NULL | 1 | +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ 2 rows in set (0.00 sec)
或查看规则的所有信息：
Admin> SELECT * FROM mysql_query_rules \G
*************************** 1. row ***************************
rule_id: 5
active: 1
username: NULL
schemaname: NULL
flagIN: 0
client_addr: NULL
proxy_addr: NULL
proxy_port: NULL
digest: NULL
match_digest: NULL
match_pattern: FROM sbtest1
negate_match_pattern: 0
re_modifiers: CASELESS
flagOUT: NULL
replace_pattern: NULL
destination_hostgroup: 20
cache_ttl: NULL
cache_empty_result: NULL
cache_timeout: NULL
reconnect: NULL
timeout: NULL
retries: NULL
delay: NULL
next_query_flagIN: NULL
mirror_flagOUT: 100
mirror_hostgroup: NULL
error_msg: NULL
OK_msg: NULL
sticky_conn: NULL
multiplex: NULL
gtid_from_hostgroup: NULL
log: NULL
apply: 1
comment: NULL
*************************** 2. row ***************************
rule_id: 10
active: 1
username: NULL
schemaname: NULL
flagIN: 100
client_addr: NULL
proxy_addr: NULL
proxy_port: NULL
digest: NULL
match_digest: NULL
match_pattern: FROM sbtest1
negate_match_pattern: 0
re_modifiers: CASELESS
flagOUT: NULL
replace_pattern: FROM sbtest1 IGNORE INDEX(k_1)
destination_hostgroup: 20
cache_ttl: NULL
cache_empty_result: NULL
cache_timeout: NULL
reconnect: NULL
timeout: NULL
retries: NULL
delay: NULL
next_query_flagIN: NULL
mirror_flagOUT: NULL
mirror_hostgroup: NULL
error_msg: NULL
OK_msg: NULL
sticky_conn: NULL
multiplex: NULL
gtid_from_hostgroup: NULL
log: NULL
apply: 1
comment: NULL
2 rows in set (0.00 sec)
须要注意的是，在rule_id = 10的规则中，镜像查询将匹配该规则[由于mirror_flagOUT(5号规则)=flagIN(100=10号规则)]，此时咱们须要设置 destination_hostgroup 而不是 mirror_hostgroup ：只应该为原始查询设置mirror_hostgroup ，以便当即能够将镜像查询请求发送到指定位置，而无需 mysql_query_rules 中的其余额外规则介入。
2）加载配置到RUNTIME层
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.00 sec) Admin> SELECT rule_id,active,flagIN,match_pattern,replace_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM runtime_mysql_query_rules ; +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ | rule_id | active | flagIN | match_pattern | replace_pattern | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply | +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ | 5 | 1 | 0 | FROM sbtest1 | NULL | 20 | 100 | NULL | 1 | | 10 | 1 | 100 | FROM sbtest1 | FROM sbtest1 IGNORE INDEX(k_1) | 20 | NULL | NULL | 1 | +---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+ 2 rows in set (0.00 sec)
配置已生效！
二、业务帐号执行查询
1）使用业务帐号连入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空stats_mysql_query_digest以得到新的统计信息：
Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset; +----------+ | COUNT(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec)
3）业务帐号执行查询
从mysql客户端咱们能够运行一般的查询：
mysql> SELECT id FROM sbtest1 ORDER BY k DESC LIMIT 3; +------+ | id | +------+ | 5076 | | 5121 | | 8573 | +------+ 3 rows in set (0.00 sec) mysql> SELECT id,k FROM sbtest1 ORDER BY k DESC LIMIT 3; +------+------+ | id | k | +------+------+ | 5076 | 7672 | | 5121 | 7655 | | 8573 | 7565 | +------+------+ 3 rows in set (0.01 sec)
4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：
Admin> SELECT hostgroup,count_star,sum_time,digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC; +-----------+------------+----------+--------------------------------------------------------------------+ | hostgroup | count_star | sum_time | digest_text | +-----------+------------+----------+--------------------------------------------------------------------+ | 20 | 1 | 3997 | SELECT id,k FROM sbtest1 IGNORE INDEX(k_1) ORDER BY k DESC LIMIT ? | | 20 | 1 | 3997 | SELECT id FROM sbtest1 IGNORE INDEX(k_1) ORDER BY k DESC LIMIT ? | | 20 | 1 | 615 | SELECT id FROM sbtest1 ORDER BY k DESC LIMIT ? | | 20 | 1 | 342 | SELECT id,k FROM sbtest1 ORDER BY k DESC LIMIT ? | +-----------+------------+----------+--------------------------------------------------------------------+ 4 rows in set (0.01 sec)
表stats_mysql_query_digest 的结果代表：
1）原始查询被镜像了；
2）原始查询未被重写(而是镜像)；
3）镜像查询被重写了(带了规则里的IGNORE语句)；
4）镜像查询速度要慢得多，由于忽略了索引；
7、高级示例：使用镜像测试查询重写[Advanced example: use mirroring to test query rewrite]
在处理镜像时，我被问到一个彻底不一样的问题，与查询重写有关：如何知道给定的正则表达式是否与给定的查询匹配，并验证重写模式是否正确？
更具体地说，问题是要了解重写是否正确而不影响实时流量。虽然镜像最初并非为此设计的，但它能够回答这个问题。
在这个例子中，咱们将编写一个规则来匹配全部SELECT，"镜像"它们，并尝试重写它们。
一、设置查询规则
1）设置查询规则
Admin> DELETE FROM mysql_query_rules;
Query OK, 2 rows affected (0.00 sec)
Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,apply) VALUES (5,1,'^SELECT ',20,100,1);
Query OK, 1 row affected (0.00 sec)
Admin> INSERT INTO mysql_query_rules (rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,apply) VALUES
(10,1,100,'^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$',20,'SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c',1);
Query OK, 1 row affected (0.00 sec)
查看设置结果：
Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM mysql_query_rules \G
*************************** 1. row ***************************
rule_id: 5
active: 1
flagIN: 0
match_pattern: ^SELECT
destination_hostgroup: 20
replace_pattern: NULL
mirror_flagOUT: 100
apply: 1
*************************** 2. row ***************************
rule_id: 10
active: 1
flagIN: 100
match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
mirror_flagOUT: NULL
apply: 1
2 rows in set (0.00 sec)
2）加载配置到RUNTIME层
Admin> LOAD MYSQL QUERY RULES TO RUNTIME ;
Query OK, 0 rows affected (0.00 sec)
Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM runtime_mysql_query_rules \G
*************************** 1. row ***************************
rule_id: 5
active: 1
flagIN: 0
match_pattern: ^SELECT
destination_hostgroup: 20
replace_pattern: NULL
mirror_flagOUT: 100
apply: 1
*************************** 2. row ***************************
rule_id: 10
active: 1
flagIN: 100
match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
mirror_flagOUT: NULL
apply: 1
2 rows in set (0.00 sec)
上面的正则表达式很是复杂，这就是为何镜像查询功能有用之处，它不是直接重写实时流量。
二、业务帐号执行查询
1）使用业务帐号连入ProxySQL
# mysql -h 188.188.0.71 -P 6033 -umsandbox -p mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | sbtest | +--------------------+ 2 rows in set (0.00 sec) mysql> use sbtest; mysql> show tables; +------------------+ | Tables_in_sbtest | +------------------+ | sbtest1 | | sbtest2 | | sbtest3 | | sbtest4 | | sbtest5 | +------------------+ 5 rows in set (0.00 sec)
2）ProxySQL管理端清空stats_mysql_query_digest以得到新的统计信息：
Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset; +----------+ | COUNT(*) | +----------+ | 10 | +----------+ 1 row in set (0.00 sec)
3）业务帐号执行查询
从mysql客户端咱们能够运行一般的查询：
mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c; +-------------------------------------------------------------------------------------------------------------------------+ | c | +-------------------------------------------------------------------------------------------------------------------------+ | 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 | | 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 | | 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 | +-------------------------------------------------------------------------------------------------------------------------+ 3 rows in set (0.00 sec)
查询已成功运行。 如上所述，咱们没有修改原始流量。
那 stats_mysql_query_digest 中是又怎么样？
4）ProxySQL管理端，查看当前以执行SQL的查询概要统计：
Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text; +-----------+------------+----------+----------------------------------------------------------------------+ | hostgroup | count_star | sum_time | digest_text | +-----------+------------+----------+----------------------------------------------------------------------+ | 20 | 2 | 1493 | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c | +-----------+------------+----------+----------------------------------------------------------------------+ 1 row in set (0.00 sec)
能够看到，原始查询执行了两次，所以某些内容没法正常运行。咱们能够注意到两个查询都被发送到hostgroup20：咱们应该相信rule_id = 10是匹配的，可是没有重写查询。
让咱们验证它是匹配的：
Admin> SELECT * from stats_mysql_query_rules; +---------+------+ | rule_id | hits | +---------+------+ | 5 | 2 | -->其中有一次为show tables触发的。 | 10 | 1 | +---------+------+ 2 rows in set (0.00 sec)
根据 stats_mysql_query_rules 中的规则命中信息能够看到，rule_id = 10的规则是匹配的。
那为何不重写查询？
5）问题修复
在从新审查一次rule_id=10的replace_pattern内容：
SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
发现，+号前面不应有转义的。更新查询规则10：
Admin> UPDATE mysql_query_rules SET replace_pattern='SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c' WHERE rule_id=10;
Query OK, 1 row affected (0.00 sec)
Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM runtime_mysql_query_rules WHERE rule_id=10 \G
*************************** 1. row ***************************
rule_id: 10
active: 1
flagIN: 100
match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c
mirror_flagOUT: NULL
apply: 1
1 row in set (0.00 sec)
修改已经生效！！
6）验证结果
ProxySQL管理端清空stats_mysql_query_digest以得到新的统计信息：
Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset; +----------+ | COUNT(*) | +----------+ | 5 | +----------+ 1 row in set (0.00 sec)
在客户端再次执行原SQL：
mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c; +-------------------------------------------------------------------------------------------------------------------------+ | c | +-------------------------------------------------------------------------------------------------------------------------+ | 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 | | 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 | | 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 | +-------------------------------------------------------------------------------------------------------------------------+ 3 rows in set (0.00 sec)
如今让咱们验证查询是否被正确重写：
Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text; +-----------+------------+----------+----------------------------------------------------------------------+ | hostgroup | count_star | sum_time | digest_text | +-----------+------------+----------+----------------------------------------------------------------------+ | 20 | 1 | 414 | SELECT DISTINCT c FROM sbtest1 WHERE id = ? + ? ORDER BY c | | 20 | 1 | 661 | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c | +-----------+------------+----------+----------------------------------------------------------------------+ 2 rows in set (0.00 sec)
到此能够看到，查询被正确重写，而且执行了！
8、高级示例：使用镜像和防火墙来测试查询重写[Advanced example: use mirroring and firewall to test query rewrite]
前面的示例/练习有点超前：那么，是否能够在不执行查询的状况下重写查询？答案是：能够的！
为此，咱们将为镜像查询设置 error_msg ：这样ProxySQL将处理镜像查询，但会过滤它而不将其发送到任何mysql服务器。
如最初所讲的那样，能够修改镜像查询，而且防火墙是修改镜像查询的示例。
例如：
接着前面的案例，继续操做。
1）修改查询规则
为规则10添加error_msg信息。
Admin> UPDATE mysql_query_rules SET error_msg="random error, blah blah" WHERE rule_id=10; Query OK, 1 row affected (0.00 sec)
2）加载配置到RUNTIME层
Admin> LOAD MYSQL QUERY RULES TO RUNTIME; Query OK, 0 rows affected (0.01 sec)
查看修改结果：
Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply,error_msg FROM runtime_mysql_query_rules WHERE rule_id=10 \G
*************************** 1. row ***************************
rule_id: 10
active: 1
flagIN: 100
match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c
mirror_flagOUT: NULL
apply: 1
error_msg: random error, blah blah
1 row in set (0.00 sec)
能够看到，修改已生效！！
3）ProxySQL管理端清空并查看当前以执行SQL的查询概要统计：
Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset; +----------+ | COUNT(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec)
4）业务帐号执行查询
在mysql客户端从新运行查询：
mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c; +-------------------------------------------------------------------------------------------------------------------------+ | c | +-------------------------------------------------------------------------------------------------------------------------+ | 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 | | 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 | | 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 | +-------------------------------------------------------------------------------------------------------------------------+ 3 rows in set (0.00 sec)
5）ProxySQL管理端，查看当前以执行SQL的查询概要统计：
Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text; +-----------+------------+----------+----------------------------------------------------------------------+ | hostgroup | count_star | sum_time | digest_text | +-----------+------------+----------+----------------------------------------------------------------------+ | 10 | 1 | 0 | SELECT DISTINCT c FROM sbtest1 WHERE id = ? + ? ORDER BY c | | 20 | 1 | 730 | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c | +-----------+------------+----------+----------------------------------------------------------------------+ 2 rows in set (0.00 sec) Admin> SELECT * from stats_mysql_query_rules; +---------+------+ | rule_id | hits | +---------+------+ | 5 | 1 | | 10 | 1 | +---------+------+ 2 rows in set (0.01 sec)
Great!!咱们已看到查询已被重写，但实际上并未在任何地方发送：
1）sum_time = 0 ，由于响应是当即的;
2）hostgroup=10 ，在规则10中，设置了destination_hostgroup=20，但这里却显示了业务帐号默认的主机组10；若是业务帐号未设置默认主机组，则显示为0；这种显示和设置不一致也表面没有发送到任何地方(不信能够打开默认主机组MySQL的general_log去验证).
完毕！
相关文章
- 1. ProxySQL官档翻译__24_Packaging
- 2. ProxySQL官档翻译__19_Monitor_Module
- 3. ProxySQL官档翻译__23_Error_codes
- 4. ProxySQL官档翻译__17_Multiplexing
- 5. ProxySQL官档翻译__18_Query_Cache
- 6. ProxySQL官档翻译__02_ProxySQL配置概述
- 7. ProxySQL官档翻译__22_Bug反馈
- 8. ProxySQL官档翻译__01_ProxySQL安装
- 9. ProxySQL官档翻译__21_Memory_leak_detection(内存泄漏检测)
- 10. ProxySQL官档翻译__09_ProxySQL配置之系统库_00_Admin_Schemas
- 更多相关文章...
- • WSDL 文档 - WSDL 教程
- • XSL-FO 文档 - XSL-FO 教程
- • 使用阿里云OSS+CDN部署前端页面与加速静态资源
- • ☆基于Java Instrument的Agent实现
相关标签/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
欢迎关注本站公众号,获取更多信息
相关文章
- 1. ProxySQL官档翻译__24_Packaging
- 2. ProxySQL官档翻译__19_Monitor_Module
- 3. ProxySQL官档翻译__23_Error_codes
- 4. ProxySQL官档翻译__17_Multiplexing
- 5. ProxySQL官档翻译__18_Query_Cache
- 6. ProxySQL官档翻译__02_ProxySQL配置概述
- 7. ProxySQL官档翻译__22_Bug反馈
- 8. ProxySQL官档翻译__01_ProxySQL安装
- 9. ProxySQL官档翻译__21_Memory_leak_detection(内存泄漏检测)
- 10. ProxySQL官档翻译__09_ProxySQL配置之系统库_00_Admin_Schemas