下载的包都存放在~/caibh/fdfs-package
目录下。html
下载libfastcommon
、fastdfs
、fastdfs-nginx-module
:nginx
cd ~/caibh/fdfs-package # -S print server response # -O write documents to file wget https://github.com/happyfish100/libfastcommon/archive/V1.0.39.tar.gz -SO libfastcommon.tar.gz wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz -SO fastdfs.tar.gz wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz -SO fastdfs-nginx-module.tar.gz # 解压 tar -xf xxx.tar.gz
libfastcommon
cd ~/caibh/fdfs-package cd libfastcommon-1.0.39 ./make.sh ./make.sh install
fastdfs
cd ~/caibh/fdfs-package cd fastdfs-5.11 ./make.sh ./make.sh install
安装好后,程序是在/usr/bin
目录下:git
$ which fdfs_trackerd /usr/bin/fdfs_trackerd
而配置文件是在/etc/fdfs
目录下:github
$ cd ~/caibh/fdfs-package/fastdfs-5.11 $ ls /etc/fdfs client.conf.sample storage_ids.conf.sample tracker.conf.sample storage.conf.sample
可是这些配置文件是不全的,并且都是模板,因此须要从fastdfs
包中拷贝过来,并修改配置:web
$ cd ~/caibh/fdfs-package/fastdfs-5.11/conf $ ls anti-steal.jpg client.conf http.conf mime.types storage.conf storage_ids.conf tracker.conf $ sudo cp ~/caibh/fdfs-package/fastdfs-5.11/conf/* /etc/fdfs
sudo vi /etc/fdfs/tracker.conf
:ubuntu
# the tracker server port port=22122 # the base path to store data and log files base_path=/home/caibh/fdfs # HTTP port on this tracker server http.server_port=9270
sudo vi /etc/fdfs/storage.conf
:bash
# storage所属的组 group_name=group1 # the storage server port port=23000 # the base path to store data and log files base_path=/home/caibh/fdfs # store_path#, based 0, if store_path0 not exists, it's value is base_path # the paths must be exist store_path0=/home/caibh/fdfs #store_path1=/home/caibh/fdfs2 # tracker服务器,虽然是同一台机器上,可是不能写127.0.0.1。这项配置能够出现一次或屡次 tracker_server=191.8.1.77:22122 # the port of the web server on this storage server http.server_port=8888
sudo vi /etc/fdfs/client.conf
:服务器
# the base path to store log files base_path=/home/caibh/fdfs/client # tracker_server can ocur more than once, and tracker_server format is # "host:port", host can be hostname or ip address tracker_server=191.8.1.77:22122 #HTTP settings http.tracker_server_port=9270
sudo vi /etc/fdfs/mod_fastdfs.conf
:app
# the base path to store log files base_path=/tmp # FastDFS tracker_server can ocur more than once, and tracker_server format is # "host:port", host can be hostname or ip address # valid only when load_fdfs_parameters_from_tracker is true tracker_server=191.8.1.77:22122 # the port of the local storage server # the default value is 23000 storage_server_port=23000 # the group name of the local storage server group_name=group1 # store_path#, based 0, if store_path0 not exists, it's value is base_path # the paths must be exist # must same as storage.conf store_path0=/home/caibh/fdfs #store_path1=/home/yuqing/fastdfs1
配置过程当中有几点要注意:负载均衡
~/fdfs/client
、~/fdfs/data
、~/fdfs/logs
确保各类配置文件之间引用的端口一直。好比:
mod_fastdfs.conf
文件中tracker_server
的端口应该跟tracker.conf
中port
一致;mod_fastdfs.conf
文件中storage_server_port
的端口应该跟跟storage.conf
中port
一致;其余配置或文件虽然不用修改,可是fastdfs-nginx-module
模块会用到:
anti-steal.jpg
http.conf
mime.types
tracker
和storage
:# 启动 fdfs_trackerd /etc/fdfs/tracker.conf start fdfs_storaged /etc/fdfs/storage.conf start # 查看日志 tail -n10 ~/fdfs/logs/trackerd.log tail -n10 ~/fdfs/logs/storaged.log # 若是日志显示有错误信息,须要根据信息来查找错误缘由
fdfs_test
测试上传$ fdfs_test /etc/fdfs/client.conf upload ~/caibh/test_images/XinXiJuZhiWang.jpg
fastdfs-nginx-module
思路:安装过程比较复杂,因为当前系统已经有一个nginx在运行了,因此要先确认当前系统nginx的版本,而后到官网下载对应版本的nginx源码,而后查看系统nginx的编译选项,用如出一辙的编译选项外加
fastdfs-nginx-module
的源码,从新编译一个nginx程序。用新的nginx替换系统的nginx。注意:
这里的安装场景是tracker和storage都装在同一个机器上。
查看已安装并在运行中的nginx的版本:
$ nginx -v nginx version: nginx/1.10.3 (Ubuntu)
到官网下载对应版本的nginx源码,并解压,更名为nginx-src
,而后进入nginx-src
目录
查看当前运行的ngxin原来的编译选项:
$ cd ~/caibh/nginx-src $ nginx -V nginx version: nginx/1.10.3 (Ubuntu) built with OpenSSL 1.0.2g 1 Mar 2016 TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-stream --with-stream_ssl_module --with-mail --with-mail_ssl_module --with-threads
为了方便编辑,复制上面信息,在nginx-src
目录下建立一个custom_configure.sh
文件,把复制的内容拷贝进去,并在最后加上fastdfs-nginx-module
的源码:
$ cd ~/caibh/nginx-src $ cat custom_configure.sh ./configure \ --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' \ --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' \ --prefix=/usr/share/nginx \ --conf-path=/etc/nginx/nginx.conf \ --http-log-path=/var/log/nginx/access.log \ --error-log-path=/var/log/nginx/error.log \ --lock-path=/var/lock/nginx.lock \ --pid-path=/run/nginx.pid \ --http-client-body-temp-path=/var/lib/nginx/body \ --http-fastcgi-temp-path=/var/lib/nginx/fastcgi \ --http-proxy-temp-path=/var/lib/nginx/proxy \ --http-scgi-temp-path=/var/lib/nginx/scgi \ --http-uwsgi-temp-path=/var/lib/nginx/uwsgi \ --with-debug \ --with-pcre-jit \ --with-ipv6 \ --with-http_ssl_module \ --with-http_stub_status_module \ --with-http_realip_module \ --with-http_auth_request_module \ --with-http_addition_module \ --with-http_dav_module \ --with-http_geoip_module \ --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_image_filter_module \ --with-http_v2_module \ --with-http_sub_module \ --with-http_xslt_module \ --with-stream \ --with-stream_ssl_module \ --with-mail \ --with-mail_ssl_module \ --with-threads \ --add-module=~/caibh/fdfs-package/fastdfs-nginx-module-1.20/src
执行custom_configure.sh
:
# ~/caibh/nginx-src # 执行编译 $ sudo ./custom_configure.sh # 报错 ./configure: error: the HTTP XSLT module requires the libxml2/libxslt libraries. You can either do not enable the module or install the libraries. # 解决 sudo apt-get install libxml2 libxml2-dev libxslt-dev # 再次执行 $ sudo ./custom_configure.sh # 报错 ./configure: error: the HTTP image filter module requires the GD library. You can either do not enable the module or install the libraries. #解决 sudo apt-get install libgd-dev # 再次执行 $ sudo ./custom_configure.sh # 报错 ./configure: error: the GeoIP module requires the GeoIP library. You can either do not enable the module or install the library. # 搜索 $ sudo apt-cache search geoip geoclue-ubuntu-geoip - Provide positioning for GeoClue via Ubuntu GeoIP services geoip-bin - IP lookup command line tools that use the GeoIP library geoip-database - IP lookup command line tools that use the GeoIP library (country database) geoip-dbg - debugging symbols for the GeoIP packages libgeoip-dev - Development files for the GeoIP library ..... # 解决 $ sudo apt-install libgeoip-dev
执行后有用的信息,经过这些信息能够知道编译出来的nginx会装到哪里,配置文件会放在哪里,错误日志会放在哪里:
Configuration summary + using threads + using system PCRE library + using system OpenSSL library + md5: using OpenSSL library + sha1: using OpenSSL library + using system zlib library nginx path prefix: "/usr/share/nginx" nginx binary file: "/usr/share/nginx/sbin/nginx" nginx modules path: "/usr/share/nginx/modules" nginx configuration prefix: "/etc/nginx" nginx configuration file: "/etc/nginx/nginx.conf" nginx pid file: "/run/nginx.pid" nginx error log file: "/var/log/nginx/error.log" nginx http access log file: "/var/log/nginx/access.log" nginx http client request body temporary files: "/var/lib/nginx/body" nginx http proxy temporary files: "/var/lib/nginx/proxy" nginx http fastcgi temporary files: "/var/lib/nginx/fastcgi" nginx http uwsgi temporary files: "/var/lib/nginx/uwsgi" nginx http scgi temporary files: "/var/lib/nginx/scgi"
中止nginx,并备份原来的nginx执行文件,和配置文件:
$ which nginx /usr/sbin/nginx $ sudo cp /usr/sbin/nginx /usr/sbin/nginx_org_bak $ sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
继续在nginx-src
目录执行make
和make install
:
$ cd ~/caibh/nginx-src $ sudo make # 报错 Fatal error: can't create objs/addon/src/ngx_http_fastdfs_module.o: Permission denied In file included from /home/xxxx/caibh/fdfs-package/fastdfs-nginx-module-1.20/src/common.c:26:0, from /home/xxxx/caibh/fdfs-package/fastdfs-nginx-module-1.20/src/ngx_http_fastdfs_module.c:6: /usr/include/fastdfs/fdfs_define.h:15:27: fatal error: common_define.h: No such file or directory # 解决 # 修改fastdfs-nginx-module的src/config文件 $ cd /home/xxxx/caibh/fdfs-package/fastdfs-nginx-module-1.20/src $ sudo vi config # 修改一下两项: ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/" CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/" # 从新运行 nginx-src/custom_configure.sh $ pwd /home/xxxx/caibh/nginx-src $ sudo ./custom_configure.sh $ sudo make $ sudo make install # make install 会在/usr/share/nginx/sbin目录下生成一个名为nginx的可执行文件 $ sudo ls /usr/share/nginx html sbin
关闭系统原有的nginx,并备份好系统原来的nginx文件:
sudo nginx -s stop cd /usr/sbin sudo cp mv /usr/sbin/nginx /usr/sbin/nginx_bak
备份好后建立一个软连接,放在/usr/sbin
目录下:
# 检查本身编译出来的nginx可否运行 $ ./nginx -v nginx version: nginx/1.10.3 # 建立软连接 $ sudo ln -s /usr/share/nginx/sbin/nginx /usr/sbin/nginx
配置nginx,加入路由规则,映射到fastdfs的数据目录:
# config for fastdfs-nginx-module server { listen 8777; location /M00 { root ~/fdfs/data; ngx_fastdfs_module; } }
开放端口:
sudo iptables -A IN_public_allow -p tcp --dport 8777 -j ACCEPT sudo iptables -L -n
测试上传图片:
$ fdfs_test /etc/fdfs/client.conf upload ~/caibh/test_images/XinXiJuZhiWang.jpg This is FastDFS client test program v5.11 Copyright (C) 2008, Happy Fish / YuQing FastDFS may be copied only under the terms of the GNU General Public License V3, which may be found in the FastDFS source kit. Please visit the FastDFS Home Page http://www.csource.org/ for more detail. [2019-02-22 14:54:33] DEBUG - base_path=/home/xxxx/fdfs/client, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0 tracker_query_storage_store_list_without_group: server 1. group_name=, ip_addr=191.8.2.203, port=23000 group_name=group1, ip_addr=191.8.2.203, port=23000 storage_upload_by_filename group_name=group1, remote_filename=M00/00/00/vwgCy1xvnKmAIDXDAAEM3XOobI4945.jpg source ip address: 191.8.2.203 file timestamp=2019-02-22 14:54:33 file size=68829 file crc32=1940417678 example file url: http://191.8.2.203:9270/group1/M00/00/00/vwgCy1xvnKmAIDXDAAEM3XOobI4945.jpg storage_upload_slave_by_filename group_name=group1, remote_filename=M00/00/00/vwgCy1xvnKmAIDXDAAEM3XOobI4945_big.jpg source ip address: 191.8.2.203 file timestamp=2019-02-22 14:54:33 file size=68829 file crc32=1940417678 example file url: http://191.8.2.203:9270/group1/M00/00/00/vwgCy1xvnKmAIDXDAAEM3XOobI4945_big.jpg
访问:
http://191.8.2.203:8777/M00/00/00/vwgCy1xvnKmAIDXDAAEM3XOobI4945_big.jpg # 若是能看到图片则表示安装fastdfs-nginx-module成功
FastDFS是一款高性能的分布式文件系统。主要功能包括:文件存储,文件同步,文件访问(上传下载)。它能够解决高容量和负载均衡的问题。FastDFS适合用来作文件相关的网站,如图片分享、视频分享等。
FastDFS有两种角色:tracker
和storage
。tracker
负责文件访问的跟踪和负载均衡。storage
负责存储文件,它的功能包括:文件存储、文件同步、提供文件访问接口,storage
还负责管理文件的元数据,这些数据是以键值对的形式表示的,如:width=1024,键是“width”,值是“1024”。
tracker
和storage
能够是一个或多个服务器。在tracker
或storage
的集群中,任什么时候候均可以从中删除服务器,对线上服务没有任何影响。tracker
集群中的服务器是点对点的。
存储服务器以 卷/组 形式组织,以获取高容量。存储系统包括一个或多个卷,卷与卷之间的文件是相互独立的。整个存储系统的总容量等于全部卷的容量之和。一个文件卷包括一个或多个存储服务器,一个卷中的存储服务器之间的文件是同样的。一个文件卷中的服务器会互相备份,且这些服务器之间是负载均衡的。当添加一个存储服务器到一个卷的时候,卷中的文件会自动复制到新的服务器,复制完之后,系统会把新服务器切换到线上,已提供存储服务。
当存储容量不足时,你能够添加一个或多个卷来扩展存储容量。
一个文件的标识包括两部分:卷名和文件名
step 1. 下载安装 libfastcommon
地址:https://github.com/happyfish1...
step 2. 下载安装 FastDFS
解压:
tar xzf FastDFS_v5.08.tar.gz
step 3. 进入FastDFS的目录
cd FastDFS
step 4. 执行make:
./make.sh
step 5. 执行make install
./make.sh install
step 6. 修改tracker
和storage
的配置文件:
/etc/fdfs/tracker.conf
几个重要的配置:
# the tracker server port port=22122 # the base path to store data and log files base_path=/home/caibh/fdfs # which group to upload file # when store_lookup set to 1, must set store_group to the group name store_group=group2 # HTTP port on this tracker server http.server_port=9270
/etc/fdfs/storage.conf
几个重要的配置:
# the storage server port port=23000 # the base path to store data and log files base_path=/home/caibh/fdfs # store_path#, based 0, if store_path0 not exists, it's value is base_path # the paths must be exist store_path0=/home/caibh/fdfs #store_path1=/home/caibh/fdfs2 # tracker_server can ocur more than once, and tracker_server format is # "host:port", host can be hostname or ip address # 在同一台机器上配置tracker和storage时,这里若是是127.0.0.1会报错,因此要配置真正的ip地址。 tracker_server=191.8.1.77:22122 # the port of the web server on this storage server http.server_port=8888
step 7. run server programs
启动tracker
:
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
in Linux, you can start fdfs_trackerd as a service:
/sbin/service fdfs_trackerd start
启动storage
:
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
in Linux, you can start fdfs_storaged as a service:
/sbin/service fdfs_storaged start
step 8. 测试
修改客户端配置文件:
/etc/fdfs/client.conf
几个重要的配置:
# the base path to store log files base_path=/home/caibh/fdfs/client # tracker_server can ocur more than once, and tracker_server format is # "host:port", host can be hostname or ip address tracker_server=191.8.1.77:22122 #HTTP settings http.tracker_server_port=9270
run the client test program:
/usr/bin/fdfs_test <client_conf_filename> <operation> /usr/bin/fdfs_test1 <client_conf_filename> <operation>
例如上传一个图片文件:
caibh@pc:~$ fdfs_test /etc/fdfs/client.conf upload /develop/fastdfs/test_file/LiuLangDiQiu.jpg This is FastDFS client test program v5.11 Copyright (C) 2008, Happy Fish / YuQing FastDFS may be copied only under the terms of the GNU General Public License V3, which may be found in the FastDFS source kit. Please visit the FastDFS Home Page http://www.csource.org/ for more detail. [2019-02-19 11:32:30] DEBUG - base_path=/home/caibh/fdfs/client, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0 tracker_query_storage_store_list_without_group: server 1. group_name=, ip_addr=191.8.1.77, port=23000 group_name=group1, ip_addr=191.8.1.77, port=23000 storage_upload_by_filename group_name=group1, remote_filename=M00/00/00/vwgBTVxreM6AD-mcAAEE_qeEI2s285.jpg source ip address: 191.8.1.77 file timestamp=2019-02-19 11:32:30 file size=66814 file crc32=2810454891 example file url: http://191.8.1.77:9270/group1/M00/00/00/vwgBTVxreM6AD-mcAAEE_qeEI2s285.jpg storage_upload_slave_by_filename group_name=group1, remote_filename=M00/00/00/vwgBTVxreM6AD-mcAAEE_qeEI2s285_big.jpg source ip address: 191.8.1.77 file timestamp=2019-02-19 11:32:30 file size=66814 file crc32=2810454891 example file url: http://191.8.1.77:9270/group1/M00/00/00/vwgBTVxreM6AD-mcAAEE_qeEI2s285_big.jpg
step 9. 查看状态
run the monitor program:
/usr/bin/fdfs_monitor <client_conf_filename> # 如: caibh@pc:~$ fdfs_monitor /etc/fdfs/storage.conf [2019-02-19 10:25:09] DEBUG - base_path=/home/caibh/fdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0 server_count=1, server_index=0 tracker server is 191.8.1.77:22122 group count: 1 Group 1: group name = group1 disk total space = 111868 MB disk free space = 62228 MB trunk free space = 0 MB storage server count = 1 active server count = 1 storage server port = 23000 storage HTTP port = 8888 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0 Storage 1: id = 191.8.1.77 ip_addr = 191.8.1.77 (191-8-1-77.user.vivozap.com.br) ACTIVE http domain = version = 5.11 join time = 2019-02-19 10:22:53 up time = 2019-02-19 10:22:53 total storage = 111868 MB free storage = 62228 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888
tracker server config file sample please see conf/tracker.conf
storage server config file sample please see conf/storage.conf
client config file sample please see conf/client.conf
Item detail
item name | type | default | Must |
---|---|---|---|
base_path | string | Y | |
disabled | boolean | false | N |
bind_addr | string | N | |
network_timeout | int | 30(s) | N |
max_connections | int | 256 | N |
log_level | string | info | N |
run_by_group | string | N | |
run_by_user | string | N | |
allow_hosts | string | * | N |
sync_log_buff_interval | int | 10(s) | N |
thread_stack_size | string | 1M | N |
memo:
item name | type | default | Must |
---|---|---|---|
item name | type | default | Must |
port | int | 22000 | N |
store_lookup | int | 0 | N |
store_group | string | N | |
store_server | int | 0 | N |
store_path | int | 0 | N |
download_server | int | 0 | N |
reserved_storage_space | string | 1GB | N |
memo:
item name | type | default | Must |
---|---|---|---|
group_name | string | Y | |
tracker_server | string | Y | |
port | int | 23000 | N |
heart_beat_interval | int | 30(s) | N |
stat_report_interval | int | 300(s) | N |
sync_wait_msec | int | 100(ms) | N |
sync_interval | int | 0(ms) | N |
sync_start_time | string | 00:00 | N |
sync_end_time | string | 23:59 | N |
store_path_count | int | 1 | N |
store_path0 | string | base_path | N |
store_path# | string | N | |
subdir_count_per_path | int | 256 | N |
check_file_duplicate | boolean | 0 | N |
key_namespace | string | N | |
keep_alive | boolean | 0 | N |
sync_binlog_buff_interval | int | 60s | N |
memo:
step 1. first install the FastDFS storage server and client library, the FastDFS version should >= 5.11. download address: https://github.com/happyfish1...
step 2. install nginx server
FastDFS nginx module test passed with nginx 1.15.1,
my nginx installed in /usr/local/nginx
step 3. download FastDFS nginx module source package and unpack it, such as:
cd /home/yuqing git clone https://github.com/happyfish100/fastdfs
step 4. enter the nginx source dir, compile and install the module, such as:
cd nginx-1.15.1 ./configure --add-module=/home/yuqing/fastdfs-nginx-module/src make; make install
注意: before compile, you can changeFDFS_OUTPUT_CHUNK_SIZE
andFDFS_MOD_CONF_FILENAME
macro in the config file as:
CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
step 5. config the nginx config file such as nginx.conf, add the following lines:
location /M00 { root /home/yuqing/fastdfs/data; ngx_fastdfs_module; }
step 6. 建立软连接 ${fastdfs_base_path}/data/M00
连接到 ${fastdfs_base_path}/data
,命令以下:
ln -s /home/yuqing/fastdfs/data /home/yuqing/fastdfs/data/M00
step 7. change the config file /etc/fdfs/mod_fastdfs.conf, more detail please see it
step 8. 重启nginx:
/usr/local/nginx/sbin/nginx -s stop; /usr/local/nginx/sbin/nginx