nginx upload module的使用

时间 2019-11-16

原文原文链接

如今的网站，总会有一点与用户交互的功能，例如容许用户上传头像，上传照片，上传附件这类的。PHP写的程序，对于上传文件效率不是很高。
幸亏，nginx有一个名为upload的module能够解决这个问题。网络上已经有不少关于upload module的文章，可是大部分都是介绍编译安装这些的，对于一些细节叙述不是很清楚，因而本身写了这篇。
参考了不少其余人的文档，在此致谢，详细见参考文档部分。

1、upload module的工做原理
nginx upload module模块经过nginx服务器来接受用户上传的文件，它自动分析客户端的上传请求，将上传的文件保存到 upload_store 所指向的目录位置。
而后这些文件信息将被从原始的请求中剔除，从新组装好上传参数后转交到后端由 upload_pass 指定的location去处理，这样后端就能够任意处理上传的文件。
每个上传的 file 字段值将能够由upload_set_form_field 指定的值替换。
上传的文件能够经过$upload_tmp_path 变量访问到。
上传的文件通过处理之后，由 upload_cleanup 指定的条件控制删除清理。

以上表达可能不够直观，咱们举个例子，假如咱们使用以下一个页面做为上传页面:

[root@Nginx-Mod-DEV wwwroot]# cat index.html
<html>
<head>
<title>Test upload</title>
</head>
<body>
<h2>Select files to upload</h2>
<form name="upload" method="POST" enctype="multipart/form-data" action="/upload">
<input type="file" name="file1"><br>
<input type="file" name="file2"><br>
<input type="submit" name="submit" value="Upload">
<input type="hidden" name="test" value="value">
</form>
</body>
</html>

那么咱们选择两个文件,点击"Upload"按钮之后,nginx upload module会接收上传来的文件，将上传的文件保存到 upload_store 所指向的目录位置。
而后上传的文件信息将被从原始的请求中剔除，根据nginx.conf中的配置从新组装好上传参数，而后转到后端由 upload_pass 指定的位置去处理。
假如咱们上传得两个文件分别名为:Picture 1.png和Picture 2.png，那么upload module接收文件之后，会传递给后端相似下边这样的一组参数:

"file1.path"=>"/tmp/0000123458",
"file2.path"=>"/tmp/0000123459",
"file1.content_type"=>"image/png",
"file2.content_type"=>"image/png",
"file1.name"=>"Picture 1.png",
"file2.name"=>"Picture 2.png",

这样后端的PHP代码就可使用$_POST变量获取这些参数来处理上传进来的文件了。下边是一个简单的示例，只是显示出相应的变量的值，而后根据文件的MD5值，取第一个字符做为对应的一级目录名称，取最后一个字符做为第二级目录名称，而后将上传的文件移动到相应的目录中。

<?php
$temppath = $_POST["file1_path"];
$name = $_POST["file1_name"];
$md5 = $_POST["file1_md5"];
$f_dir = substr($md5,0,1);
$s_dir = substr($md5,-1);
$final_file_path = "/".$f_dir."/".$s_dir."/".$name;

echo $temppath."<br />";
echo $name."<br />";
echo $md5."<br />";
echo $f_dir."<br />";
echo $s_dir."<br />";
echo $final_file_path;
rename($temppath,$final_file_path);
?>

由于nginx upload module已经作完了最费时的mime解析工做，后端的PHP代码只须要简单的移动文件到合适的位置就能够了。由于upload module是使用C语言写的，比起PHP做解析工做的效率高多了，所以极大地提升了文件上传的效率。

2、upload module的配置参数简要说明

下边是一些配置参数的说明:

upload_pass 指明了须要后续处理的php地址
upload_cleanup 若是php出现400 404 499 500-505之类的错误，则删除上传的文件
upload_store 上传文件存放地址
upload_store_access 上传文件的访问权限，user:r是指用户可读
upload_limit_rate 上传限速，若是设置为0则表示不限制
upload_pass_form_field 从表单原样转到后端的参数，能够正则表达式表示
官方的例子是upload_pass_form_field "^submit$|^description$";
意思是把submit，description这两个字段也原样经过upload_pass传递到后端php处理。若是但愿把全部的表单字段都传给后端能够用upload_pass_form_field "^.*$";

upload_set_form_field可使用的几个变量
    $upload_field_name    表单的name值
    $upload_content_type    上传文件的类型
    $upload_file_name    客户端上传的原始文件名称
    $upload_tmp_path    文件上传后保存在服务端的位置

upload_aggregate_form_field 能够多使用的几个变量,文件接收完毕后生成的
    $upload_file_md5    文件的MD5校验值
    $upload_file_md5_uc    大写字母表示的MD5校验值
    $upload_file_sha1    文件的SHA1校验值
    $upload_file_sha1_uc    大写字母表示的SHA1校验值
    $upload_file_crc32    16进制表示的文件CRC32值
    $upload_file_size    文件大小

3、编译安装的步骤

这个很简单，也有不少人写了不错的文章，在此流水帐记录一下。

1.编译环境准备---这部分不须要，由于我要用到nginx-lua-module，因此列在这里了

[ lua 5.1 ]
wget http://www.lua.org/ftp/lua-5.1.5.tar.gz
tar xvzf lua-5.1.5.tar.gz
cd lua-5.1.5/src
make linux
cd ..
make install

此处默认安装位置是/usr/local,若是安装到其余位置须要:
export LUA_LIB=/path/to/lua/lib
export LUA_INC=/path/to/lua/include

另外,lua-ngix-module目前版本与lua 5.2不兼容,不要选择lua 5.2

[ pcre 8.30 ]
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.30.tar.gz
tar xvzf pcre-8.30.tar.gz

2.编译nginx

[ chaoslawful-lua-nginx-module ]---这部分不须要
https://github.com/chaoslawful/lua-nginx-module/downloads

[ nginx_upload_module-2.2.0 ]
wget http://www.grid.net.ru/nginx/download/nginx_upload_module-2.2.0.tar.gz
tar xvzf nginx_upload_module-2.2.0.tar.gz

[ nginx 1.0.15 ]
wget http://nginx.org/download/nginx-1.0.15.tar.gz
tar xvzf nginx-1.0.15.tar.gz
cd nginx-1.0.15
./configure --prefix=/usr/local/nginx --with-pcre=../pcre-8.30 --add-module=../nginx_upload_module-2.2.0 --add-module=../chaoslawful-lua-nginx-module-8d28785 --without-mail_pop3_module --without-mail_imap_module  --without-mail_smtp_module --with-http_ssl_module  --with-http_stub_status_module  --with-http_gzip_static_module
make && make install

4、一些常见的问题

疑问1.upload_set_form_field的变量名
http://serverfault.com/questions/152194/merging-variable-with-string-in-config-file

Question:
I the have following setup in my conf file
upload_set_form_field $upload_field_name.name "$upload_file_name";
But I want change chosen param name to:
upload_set_form_field ($upload_field_name+"[name]") "$upload_file_name";
So I can get "attachment[name]" but this doesn't work. I would be very happy if someone could help me with merging variables with string in nginx config file :).

Anwser：
Nginx does not have a concatenation character, rather it's based on valid and invalid characters, for instance in the directive:
try_files $uri $uri/ @fallback;
$uri is the variable and / is a string to append since / cannot be in a variable name.
Similarly you should try
$upload_field_name[name] "$upload_file_name";
If this doesn't work then try.
set $foo [name];
$upload_field_name$foo "$upload_file_name";
I cannot say if the upload module will even allow this, though. Minor syntax errors might also be present.

疑问2.upload模块不支持输入数组
http://newbdez33.blogspot.com/2009/05/nginx-upload-module-does-not-support.html

Nginx upload module does not support input array for PHP
Please refer below email,
    Jacky Zhang wrote:
    > Hi Valery,
    >
    > It's fine if I using nginx upload module to upload a signle file.
    > but It look incorrect that when I use a input like this:
    >
    >

    <input type="file" name="userfiles[]" />

    >
    > Do you have any suggestion?

    No, arrays won't work, you have to list every file input field individually.
    --
    Best regards,
    Valery Kholodkov

疑问3.如何限制上传文件的大小

http://wiki.nginx.org/HttpCoreModule#client_max_body_size

php.ini中要设置一下

file_uploads     on     是否容许经过HTTP上传文件的开关。默认为ON便是开
upload_tmp_dir     –     文件上传至服务器上存储临时文件的地方，若是没指定就会用系统默认的临时文件夹
upload_max_filesize     8m     望文生意，即容许上传文件大小的最大值。默认为2M
post_max_size     8m     指经过表单POST给PHP的所能接收的最大值，包括表单里的全部值。默认为8M

通常地，设置好上述四个参数后，在网络正常的状况下，上传<=8M的文件是不成问题
但若是要上传>8M的大致积文件，只设置上述四项还必定能行的通。除非你的网络真有100M/S的上传高速，不然你还得继续设置下面的参数。
max_execution_time     600     每一个PHP页面运行的最大时间值(秒)，默认30秒
max_input_time     600     每一个PHP页面接收数据所需的最大时间，默认60秒
memory_limit     8m     每一个PHP页面所吃掉的最大内存，默认8M

另外还要在nginx.conf中作设置。
upload_max_file_size <size>   这个是个软限制
client_max_body_size 这个是硬限制(默认1m)

附录: nginx.conf的配置

# Generic startup file.
user nginx;
worker_processes  4;
worker_rlimit_nofile 65536;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

# Keeps the logs free of messages about not being able to bind().
#daemon     off;

events {
        use epoll;
        worker_connections  10240;
}

http {
#       rewrite_log on;

        include mime.types;
        default_type       application/octet-stream;
        access_log         /var/log/nginx/access.log;
        sendfile           on;
#       tcp_nopush         on;
        keepalive_timeout  3;
#       tcp_nodelay        on;

        gzip on;
        gzip_http_version 1.1;
        gzip_comp_level 2;
        gzip_types    text/plain text/css
                      application/x-javascript text/xml
                      application/xml application/xml+rss
                      text/javascript;
        client_max_body_size 13m;
        index index.php index.html index.htm;

        server {
        listen 80;
        server_name image.demo.com;
        access_log /var/log/nginx/image.access.log;
        location / {
        index index.html;
        root  /images;
        }
        }

        server{
        listen 80;
        server_name upload.demo.com;
        root  /var/wwwroot;

        # 上传页面会把文件上传到这个location
        location /upload {
        # 文件上传之后转给后端的PHP代码去处理
        upload_pass /movefile.php;

        # 上传的文件临时存储位置
        # 此处注意，我把临时文件目录放到了tmpfs中，为了速度，但有丢失数据的风险！
        upload_store /dev/shm;

        # Allow uploaded files to be read only by user
        upload_store_access user:r;

        # Set specified fields in request body
        upload_set_form_field $upload_field_name.name "$upload_file_name";
        upload_set_form_field $upload_field_name.content_type "$upload_content_type";
        upload_set_form_field $upload_field_name.path "$upload_tmp_path";

        # Inform backend about hash and size of a file
        upload_aggregate_form_field "$upload_field_name.md5" "$upload_file_md5";
        upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";

        upload_pass_form_field "^submit$|^description$";

        upload_cleanup 400 404 499 500-505;
        }

#        location @test {
#        proxy_pass   http://127.0.0.1;
#        }

        location ~ \.php {
            fastcgi_pass  127.0.0.1:9000;
            fastcgi_index index.php;
            include fastcgi.conf;
        }
        }

}

参考文档:javascript

http://blog.martinfjordvald.com/2010/08/file-uploading-with-php-and-nginx/php

http://brainspl.at/articles/2008/07/20/nginx-upload-modulecss

http://www.phpabc.cn/nginxphp-fpmyou-hua-post-xing-neng.htmlhtml

http://deidara.blog.51cto.com/400447/389873java

http://anerg.com/read.php?55node

http://t.lava.cn/blog.php?id=23726linux

http://blog.joshsoftware.com/2010/10/20/uploading-multiple-files-with-nginx-upload-module-and-upload-progress-bar/nginx

http://matthewhutchinson.net/2010/1/6/nginx-upload-module-with-paperclip-on-railsgit

http://b.oldhu.com/2009/06/09/uploading-multiple-large-files-to-a-rails-application/github

http://www.tutorialchip.com/php-upload-class/