【原创】源码安装 Atlas-1.0.3 遇到的问题以及解决办法

时间 2019-11-10

标签原创源码安装 atlas 1.0.3 遇到问题以及解决办法繁體版

原文原文链接

今天 360 团队在 GitHub 上发布了 Atlas-2.0.0 版本。可喜可贺，但愿 360 的开发团队可以一直保持下去。借此时机，将以前对 Atlas-1.0.3 的一些理解和使用，以博客形式总结一下。本文主要讲一下在源码安装和刚开始运行 Atlas-1.0.3 时可能遇到的问题。

=====

按照安装说明，Atlas-1.0.3 的安装有以下依赖项：
glib(2.32.x)、libevent(1.4以上)、Lua(5.1.x)、OpenSSL(0.9.8以上)

基本执行命令以下：
./bootstrap.sh #可能须要修改其中的路径
make
sudo make install

依赖项的安装不作详细说明，假设已经所有安装好了。

=====

1. 执行 bootstrap.sh
下面是源码包中提供的 bootstrap.sh 脚本的内容：

[root@Betty Atlas-1.0.3]# vi bootstrap.sh 

#!/bin/sh 
base=$(cd "$(dirname "$0")"; pwd)
cd $base
PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure --with-mysql=/usr --prefix=/usr/local/mysql-proxy CFLAGS="-DHAVE_LUA_H -O2" LDFLAGS="-lm -ldl -lcrypto"
LUA_CFLAGS="-I/usr/local/include/" LUA_LIBS="-L/usr/local/lib -llua"

运行该脚本会报出以下的错误信息： python

[root@Betty Atlas-1.0.3]# ./bootstrap.sh
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking for a sed that does not truncate output... /bin/sed
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for gawk... (cached) gawk
checking for flex... flex
checking lex output file root... lex.yy
checking lex library... -lfl
checking whether yytext is a pointer... yes
checking how to run the C preprocessor... gcc -E
checking whether ln -s works... no, using cp -p
checking for a sed that does not truncate output... (cached) /bin/sed
checking whether make sets $(MAKE)... (cached) yes
checking whether gcc and cc understand -c and -o together... yes
checking for function prototypes... yes
checking for string.h... (cached) yes
checking whether byte ordering is bigendian... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking the maximum length of command line arguments... 98304
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for ar... ar
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking shared library path variable... LD_LIBRARY_PATH
checking lua module suffix... so
checking plugin suffix... so
checking for ANSI C header files... (cached) yes
checking for sys/wait.h that is POSIX.1 compatible... yes
checking arpa/inet.h usability... yes
checking arpa/inet.h presence... yes
checking for arpa/inet.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking sys/filio.h usability... no
checking sys/filio.h presence... no
checking for sys/filio.h... no
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking sys/un.h usability... yes
checking sys/un.h presence... yes
checking for sys/un.h... yes
checking sys/uio.h usability... yes
checking sys/uio.h presence... yes
checking for sys/uio.h... yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking sys/resource.h usability... yes
checking sys/resource.h presence... yes
checking for sys/resource.h... yes
checking pwd.h usability... yes
checking pwd.h presence... yes
checking for pwd.h... yes
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking libproc.h usability... no
checking libproc.h presence... no
checking for libproc.h... no
checking valgrind/valgrind.h usability... no
checking valgrind/valgrind.h presence... no
checking for valgrind/valgrind.h... no
checking syslog.h usability... yes
checking syslog.h presence... yes
checking for syslog.h... yes
checking for sys/time.h... (cached) yes
checking time.h usability... yes
checking time.h presence... yes
checking for time.h... yes
checking asm/msr.h usability... no
checking asm/msr.h presence... yes
configure: WARNING: asm/msr.h: present but cannot be compiled
configure: WARNING: asm/msr.h: check for missing prerequisite headers?
configure: WARNING: asm/msr.h: see the Autoconf documentation
configure: WARNING: asm/msr.h: section "Present But Cannot Be Compiled"
configure: WARNING: asm/msr.h: proceeding with the preprocessor's result
configure: WARNING: asm/msr.h: in the future, the compiler will take precedence
configure: WARNING: ## ------------------------------------------------------ ##
configure: WARNING: ## Report this to mysql-proxy-discuss@lists.launchpad.net ##
configure: WARNING: ## ------------------------------------------------------ ##
checking for asm/msr.h... yes
checking sys/timeb.h usability... yes
checking sys/timeb.h presence... yes
checking for sys/timeb.h... yes
checking sys/times.h usability... yes
checking sys/times.h presence... yes
checking for sys/times.h... yes
checking ia64intrin.h usability... no
checking ia64intrin.h presence... no
checking for ia64intrin.h... no
checking for rdtscll... no
checking for ftime... yes
checking for times... yes
checking for clock_gettime... no
checking for get_hrtime... no
checking for read_real_time... no
checking for gettimeofday... yes
checking for time... yes
checking for wait4... yes
checking cc is Sun CC... no
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking whether char is unsigned... no
checking return type of signal handlers... void
checking whether lstat dereferences a symlink specified with a trailing slash... no
checking whether stat accepts an empty string... no
checking for strftime... yes
checking for library containing socket... none required
checking for library containing gethostbyname... none required
checking for library containing hstrerror... none required
checking for socklen_t... yes
checking for ulong... yes
checking for ulong_t... no
checking for MySQL support... yes
checking for MySQL includes at... -I/usr/include/mysql
checking errmsg.h usability... no
checking errmsg.h presence... no
checking for errmsg.h... no
checking mysql.h usability... no
checking mysql.h presence... no
checking for mysql.h... no
configure: error: mysql.h is required, please install the mysql header package
[root@Betty Atlas-1.0.3]#

失败缘由：找不到 mysql.h 。
解决办法：在 bootstrap.sh 文件中添加 CPPFLAGS="-I/usr/local/mysql/include/" 。 mysql

修改后 bootstrap.sh 文件以下：

[root@Betty Atlas-1.0.3]# vi bootstrap.sh  

#!/bin/sh 
base=$(cd "$(dirname "$0")"; pwd)
cd $base
PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure --with-mysql=/usr --prefix=/usr/local/mysql-proxy CFLAGS="-DHAVE_LUA_H -O2" CPPFLAGS="-I/usr/local/mysql/include/" LDFLAGS="-lm -ldl -lcrypto" LUA_CFLAGS="-I/usr/local/include/" LUA_LIBS="-L/usr/local/lib -llua"

从新执行 bootstrap.sh 成功（即执行常规操做中的 configure ）。 linux

2. 执行 make

[root@Betty Atlas-1.0.3]# make
...
libtool: link: gcc -shared  .libs/libproxy_la-proxy-plugin.o   -Wl,-rpath -Wl,/root/workstation/mnt/Atlas-1.0.3/src/.libs -Wl,-rpath -Wl,/usr/local/lib -Wl,-rpath -Wl,/usr/local/mysql-proxy/lib -Wl,-rpath -Wl,/usr/local/lib -L/root/workstation/mnt/Atlas-1.0.3/src/.libs -L/usr/local/lib -L/usr/lib64/mysql -lmysqlclient -lz -lcrypt -L/usr/lib64 -lssl ../../src/.libs/libmysql-proxy.so /root/workstation/mnt/Atlas-1.0.3/src/.libs/libmysql-chassis.so ../../src/.libs/libsql-tokenizer.so /usr/local/lib/libevent.so -lnsl -lresolv -llua /usr/local/lib/libgmodule-2.0.so /usr/local/lib/libgthread-2.0.so /root/workstation/mnt/Atlas-1.0.3/src/.libs/libmysql-chassis-timing.so /root/workstation/mnt/Atlas-1.0.3/src/.libs/libmysql-chassis-glibext.so -lm -ldl -lcrypto /usr/local/lib/libglib-2.0.so -lpthread -lrt  -Wl,--export-dynamic -pthread   -pthread -Wl,-soname -Wl,libproxy.so -o .libs/libproxy.so
/usr/bin/ld: cannot find -lmysqlclient
collect2: ld returned 1 exit status
make[3]: *** [libproxy.la] Error 1
make[3]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3/plugins/proxy'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3/plugins'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3'
make: *** [all] Error 2
[root@Betty Atlas-1.0.3]#

失败缘由：找不到 mysqlclient 库。
解决办法：在 bootstrap.sh 文件中修改 LDFLAGS 的内容，添加 "-L/usr/local/mysql/lib/" 。 git

修改后 bootstrap.sh 文件以下：

[root@Betty Atlas-1.0.3]# vi bootstrap.sh 

#!/bin/sh 
base=$(cd "$(dirname "$0")"; pwd)
cd $base
PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure --with-mysql=/usr --prefix=/usr/local/mysql-proxy CFLAGS="-DHAVE_LUA_H -O2" CPPFLAGS="-I/usr/local/mysql
/include/" LDFLAGS="-L/usr/local/mysql/lib/ -lm -ldl -lcrypto" LUA_CFLAGS="-I/usr/local/include/" LUA_LIBS="-L/usr/local/lib -llua"

从新 bootstrap.sh && make 成功。 github

3. 执行 make install

[root@Betty Atlas-1.0.3]# make install
Making install in src
make[1]: Entering directory `/root/workstation/mnt/Atlas-1.0.3/src'
make  install-am installdirs installscript
make[2]: Entering directory `/root/workstation/mnt/Atlas-1.0.3/src'
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I/usr/include/mysql  -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include   -I/usr/local/include/ -pthread -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include   -pthread -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include   -I../lib/ -I/usr/local/mysql/include/  -DHAVE_LUA_H -O2 -MT libsql_tokenizer_la-sql-tokenizer-keywords.lo -MD -MP -MF .deps/libsql_tokenizer_la-sql-tokenizer-keywords.Tpo -c -o libsql_tokenizer_la-sql-tokenizer-keywords.lo `test -f '../lib/sql-tokenizer-keywords.c' || echo './'`../lib/sql-tokenizer-keywords.c
...
test -z "/usr/local/mysql-proxy/lib/pkgconfig" || /bin/mkdir -p "/usr/local/mysql-proxy/lib/pkgconfig"
 /usr/bin/install -c -m 644 mysql-proxy.pc mysql-chassis.pc '/usr/local/mysql-proxy/lib/pkgconfig'
make[2]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3'
make[1]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3'
[root@Betty Atlas-1.0.3]#

执行成功。 sql

此时你会发现，可执行程序 mysql-proxy 和 shell 脚本 mysql-proxyd 并无安装到系统的标准可执行文件所在目录。能够经过在环境变量 PATH 中添加的方式进行解决。

4. 运行可执行程序 mysql-proxy
尝试执行 mysql-proxy 看有什么输出（如下给出的 log 信息中存在本人添加的调试打印，故与原代码略有差别）。

[root@Betty ~]# mysql-proxy
2013-11-25 14:41:59: (debug) chassis-stats.c:35: created new global chassis stats at 0x41a3f40
2013-11-25 14:41:59: (debug) chassis-timings.c:124: created new global chassis timer info at 0x41a4dd0
2013-11-25 14:41:59: (debug) mysql-proxy-cli.c:329: [MoDB] print_version = 0    default_file = (NULL)
2013-11-25 14:41:59: (debug) mysql-proxy-cli.c:432: [MoDB] base_dir = /usr/local/mysql-proxy
2013-11-25 14:41:59: (critical) chassis-frontend.c:122: Failed to get log directory, please set by --log-path
2013-11-25 14:41:59: (message) Initiating shutdown, requested from mysql-proxy-cli.c:443
2013-11-25 14:41:59: (message) shutting down normally, exit code is: 1
[root@Betty ~]#

出现一条 critical 日志，大体意思是“没法找到保存日志文件的路径位置，须要经过 --log-path 进行指定”。而执行 mysql-proxyd 脚本获得输出以下： shell

[root@Betty ~]# mysql-proxyd 
Usage: /usr/local/mysql-proxy/bin/mysql-proxyd instance {start|stop|restart|status}

综上，想要运行起来 Atlas-1.0.3 ，至少要了解如何设置日志目录，以及 instance 所表明的含义。查看 GitHub 上的信息，发现能够经过自定义配置文件解决这两个问题。个人测试配置文件以下：数据库

[mysql-proxy]

plugins = admin, proxy

admin-username = moooofly
admin-password = moooofly
admin-lua-script = /usr/local/mysql-proxy/lib/mysql-proxy/lua/admin.lua

proxy-backend-addresses = 127.0.0.1:3306
#proxy-read-only-backend-addresses = 127.0.0.1:3305@1

daemon = true
keepalive = true

event-threads = 4

log-level = debug
log-path = /usr/local/mysql-proxy/log

instance = modb

proxy-address = 0.0.0.0:1234
admin-address = 0.0.0.0:2345

min-idle-connections = 128

#tables = person.mt.id.3
#pwds = user1:+jKsgB3YAG8=, user2:GS+tr4TPgqc=

charset = utf8

#client-ips = 127.0.0.1, 192.168.1
#lvs-ips = 192.168.1.1

上述配置中，不少选项目前不须要启用，因此都注释掉了。惟一须要说明的是，Atlas-1.0.3 对配置文件的名字有以下约束：要求配置文件名字和配置文件中的 instance 的名字要保持一致。例如个人配置文件名字为 modb.cnf ，因此 instance 的名字为 modb 。 bootstrap

从新使用 mysql-proxyd 脚本启动服务程序，输出信息以下：

[root@Betty ~]# mysql-proxyd modb status
MySQL-Proxy of modb is NOT running
[root@Betty ~]# mysql-proxyd modb start
OK: MySQL-Proxy of modb is started
[root@Betty ~]# mysql-proxyd modb status
MySQL-Proxy of modb is running (7186)
MySQL-Proxy of modb is running (7187)
[root@Betty ~]# ps aux|grep mysql-proxy
root      7186  0.0  0.0  57960  1028 ?        S    14:54   0:00 /usr/local/mysql-proxy/bin/mysql-proxy --defaults-file=/usr/local/mysql-proxy/conf/modb.cnf
root      7187  0.0  0.0 111524  2356 ?        Sl   14:54   0:00 /usr/local/mysql-proxy/bin/mysql-proxy --defaults-file=/usr/local/mysql-proxy/conf/modb.cnf
root      7207  0.0  0.0  61180   752 pts/4    S+   14:54   0:00 grep mysql-proxy
[root@Betty ~]#

能够看出，服务器程序已经正确启动。此时在 log 目录下会多出以下文件：服务器

[root@Betty log]# pwd
/usr/local/mysql-proxy/log
[root@Betty log]# 
[root@Betty log]# ll
总计 8
-rw-r----- 1 root root 1233 11-25 14:54 modb.log
-rw------- 1 root root    4 11-25 14:54 modb.pid
-rw-rw-rw- 1 root root    0 11-25 14:54 sql_modb.log
[root@Betty log]#

其中 modb.log 是服务器的通用日志；sql_modb.log 是经由 Atlas 进行处理、转发的 sql 日志。
启动 Atlas 时 modb.log 中的日志内容：

[root@Betty log]# tail -f modb.log 
2013-11-25 14:54:06: (message) mysql-proxy 0.8.2 started - instance: modb
2013-11-25 14:54:06: (message) max open file-descriptors = 1024
2013-11-25 14:54:06: (debug) mysql-proxy-cli.c:649: [MoDB] enter mainloop!!!
2013-11-25 14:54:06: (message) proxy listening on port 0.0.0.0:1234
2013-11-25 14:54:06: (message) added read/write backend: 127.0.0.1:3306
2013-11-25 14:54:06: (message) proxy-plugin.c:2929: [MoDB] current charset is set to utf8
2013-11-25 14:54:06: (message) proxy-plugin.c:2932: [MoDB] min_idle_connections is set to 32
2013-11-25 14:54:06: (message) chassis-event-thread.c:319: starting 3 worker threads(not include main thread)
2013-11-25 14:54:06: (message) chassis-unix-daemon.c:136: [angel] we try to keep PID=7187 alive
2013-11-25 14:54:06: (debug) chassis-unix-daemon.c:155: waiting for 7187

经过 MySQL 客户端经由 Atlas 访问 MySQL 数据库时，sql_modb.log 中的日志内容：

[root@Betty log]# tail -f sql_modb.log 


[11/25/2013 14:58:54] C:172.16.80.111 S:127.0.0.1 OK 0.155 "SET NAMES utf8"
[11/25/2013 14:58:54] C:172.16.80.111 S:127.0.0.1 OK 0.298 "SELECT @@character_set_database, @@collation_database"
[11/25/2013 14:58:54] C:172.16.80.111 S:127.0.0.1 OK 0.471 "SHOW FULL TABLES WHERE Table_type != 'VIEW'"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 0.124 "SET NAMES utf8"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 19.297 "SHOW TABLE STATUS LIKE 'test'"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 0.308 "SHOW CREATE TABLE `test`"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 24.835 "SELECT TABLE_NAME, PARTITION_NAME, SUBPARTITION_NAME, PARTITION_METHOD, SUBPARTITION_METHOD, PARTITION_EXPRESSION, SUBPARTITION_EXPRESSION, PARTITION_DESCRIPTION, PARTITION_COMMENT, NODEGROUP, TABLESPACE_NAME FROM information_schema.PARTITIONS WHERE TABLE_SCHEMA LIKE 'python' AND NOT ISNULL(PARTITION_NAME) AND TABLE_NAME LIKE 'test' ORDER BY TABLE_NAME, PARTITION_NAME, PARTITION_ORDINAL_POSITION, SUBPARTITION_ORDINAL_POSITION"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 0.279 "SHOW CREATE TABLE `test`"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 1.221 "SELECT * FROM `test` LIMIT 0, 1000"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 0.707 "SHOW COLUMNS FROM `python`.`test`"

至此，一个源码安装的、可用的、具备基本配置的 Atlas-1.0.3 完成了。