nginx+uwsgi+django+supervisor+mysql+redis

时间 2019-11-07

标签 nginx+uwsgi+django+supervisor+mysql+redis nginx uwsgi django supervisor mysql redis 栏目 Nginx 繁體版

原文原文链接

目录css

1. 概述 3html

2. 安装与配置 3node

2.1 django项目与应用建立 3python

2.2 uwsgi安装与配置 6mysql

2.3 supervisor安装与配置 8nginx

2.4 nginx安装与做为反向代理服务器的配置 10web

3. nginx+uwsgi优化 16sql

3.1 uwsgi进程线程数配置优化 16数据库

3.2 Django数据库链接参数CONN_MAX_AGE优化 20django

3.3 nginx中worker_rlimit_nofile与worker_connections优化 21

1. 概述

基于CentOS7.4的Nginx+UWsgi+Django+Supervisor是常见的web项目部署方式。为方便项目部署，了解各个软件配置，性能调优提供参考，以及高并发环境下一些异常修复与规避。

2. 安装与配置

PtCloud可根据环境的不一样进行灵活性的搭建和配置，但都须要使用PtCloud发行版光盘进行角色的选择，并至少选择一台物理机做为计算节点。根据根据安装的不一样状况作如下的安装说明。

2.1 Django项目与app建立

2.1.1 给python2.7安装django1.10的库

[root@bogon my_projects]# pip install django==1.10

2.1.2 建立django项目helloworld

[root@bogon my_projects]# django-admin.py startproject hellowold

2.1.3 建立第一个应用 application1

[root@bogon my_projects]# cd hellowold

[root@bogon hellowold]# django-admin.py startapp application1

2.1.4 修改setting.py中的ALLOWED_HOSTS

说明：ALLOWED_HOSTS后面所跟的属性值是一个字符串列表值，这个字符串列表值表示当下这个Django站点能够提供的host/domain(主机/域名)。*通配符去配置，另外当DEBUG设置为False的时候必须配置这个配置。不然会抛出异常。

将/home/my_projects/helloworld/helloworld/setting.py中的ALLOWED_HOSTS修改成如下内容

ALLOWED_HOSTS = ['*']

2.1.5 运行django项目

[root@bogon hellowold]# ./manage.py runserver 0.0.0.0:1001

在另外一个终端执行如下命令，进行测试

[root@bogon my_projects]# curl http://127.0.0.1:1001

<!DOCTYPE html>

html * { padding:0; margin:0; }

body * { padding:10px 20px; }

body * * { padding:0; }

body { font:small sans-serif; }

body>div { border-bottom:1px solid #ddd; }

h1 { font-weight:normal; }

h2 { margin-bottom:.8em; }

h2 span { font-size:80%; color:#666; font-weight:normal; }

h3 { margin:1em 0 .5em 0; }

h4 { margin:0 0 .5em 0; font-weight: normal; }

table { border:1px solid #ccc; border-collapse: collapse; width:100%; background:white; }

tbody td, tbody th { vertical-align:top; padding:2px 3px; }

thead th {

padding:1px 6px 1px 3px; background:#fefefe; text-align:left;

font-weight:normal; font-size:11px; border:1px solid #ddd;

}

tbody th { width:12em; text-align:right; color:#666; padding-right:.5em; }

#summary { background: #e0ebff; }

#summary h2 { font-weight: normal; color: #666; }

#explanation { background:#eee; }

#instructions { background:#f6f6f6; }

#summary table { border:none; background:transparent; }

</style>

</head>

<body>

<h1>It worked!</h1>

<h2>Congratulations on your first Django-powered page.</h2>

</div>

<p>

Of course, you haven't actually done any work yet. Next, start your first app by running <code>python manage.py startapp [app_label]</code>.

</p>

</div>

<p>

You're seeing this message because you have <code>DEBUG = True</code> in your Django settings file and you haven't configured any URLs. Get to work!

</p>

</div>

</body></html>

2.2 uwsgi安装与配置

2.2.1安装uwsgi

[root@bogon hellowold]# yum -y install uwsgi

2.2.2 django项目hellowold中uwsgi.ini配置

[root@bogon hellowold]# pwd

/home/my_projects/hellowld

[root@bogon hellowold]# cat uwsgi.ini

[uwsgi]

chdir = /home/my_projects/helloworld

module = helloworld.wsgi:application

reload-mercy = 10

user = root

uid = root

master = True

harakiri-verbose = true

post-buffering = 65536

buffer-size = 65536

Harakiri = 30

threads = 30 # 与进程的乘积不大于数据库容许的最大链接数

processes = 8 # 二倍于CPU核数

# socket = 127.0.0.1:7001 # nginx和 uwsgi间走的是 wsgi 协议，对应的nginx那边也须要配置uwsgi的一些参数

http = 127.0.0.1:7001 # uwsgi将本端口的流按照http协议解析

chmod-socket = 664

vacuum = true

2.2.3 用uwsgi启动django项目

若是步骤2.1.5启动的程序还未中止，请用ctrl+c先中止，而后执行如下命令

[root@bogon hellowold]# uwsgi --ini /home/my_projects/helloworld/uwsgi.ini

在另外一个终端中查看结果，执行

[root@bogon hellowold]# curl http://127.0.0.1:1001 |grep Congratulations

<h2>The install worked successfully! Congratulations!</h2>

2.3 supervisor安装与配置

2.3.1 supervisor安装

[root@bogon hellowold]# yum install supervisor

2.3.2 supervisor配置

将/etc/supervisor/supervisor.conf修改成如下内容

[unix_http_server]

file=/var/run/supervisor/supervisor.sock ; (the path to the socket file)

[supervisord]

logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)

logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)

logfile_backups=10 ; (num of main logfile rotation backups;default 10)

loglevel=info ; (log level;default info; others: debug,warn,trace)

pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)

nodaemon=false ; (start in foreground if true;default false)

minfds=1024 ; (min. avail startup file descriptors;default 1024)

minprocs=200 ; (min. avail process descriptors;default 200)

[rpcinterface:supervisor]

supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]

serverurl=unix:///var/run/supervisor/supervisor.sock ; use a unix:// URL for a unix socket

[include]

files = /etc/supervisor/conf.d/*.conf

在/etc/supervisor/supervisor.d目录下建立helloworld.conf内容以下

[program:helloworld]

directory = /home/my_projects/helloworld

command = uwsgi --ini /home/my_projects/helloworld/uwsgi.ini

user = root

autostart = true

autorestart = true

stopsignal = QUIT

redirect_stderr = true

loglevel = error

stdout_logfile = /var/log/uwsgi/client_uwsgi_out.log

stderr_logfile = /var/log/uwsgi/client_uwsgi_err.log

logfile_maxbytes = 10M

aogfile_maxbytes = 2M

若是步骤2.2.2启动的程序还在运行，请先ctrl+c中止，而后执行如下命令

[root@bogon hellowold]# supervisord -c /etc/supervisor/supervisord.conf # 启动supervisor服务

helloworld RUNNING pid 28899, uptime 0:00:03

其它supervisor相关命令

supervisorctl status # 查看supervisor管理的程序

supervisorctl stop all # 中止supervisor守护的全部进程

supervisorctl start all # 启动supervisor守护的全部进程

supervisorctl stop helloworld # 中止supervisor守护的helloworld进程(也能够是其它被守护的进程，该名称在/etc/supervisor/conf.d/***.conf文件中[program:后配置)。

supervisorctl start helloworld # 启动supervisor守护的helloworld进程

supervisorctl restart helloworld # 重启supervisor守护的helloworld进程

2.4 Nginx安装与做为反向代理服务器的配置

2.4.1安装nginx

[root@bogon my_projects]# yum -y install nginx

安装好nginx后，nginx也为本身建立好了用户。在/etc/nginx/nginx.conf中user后面就是nginx执行时所使用的用户。如：“user nginx;”

2.4.2配置nginx

除原有niginx.conf中的用户不修改外，其他修改成如下内容。

# user nginx; # nginx 用户为原文件中的nginx用户。

worker_processes auto;

pid /run/nginx.pid;

events {

worker_connections 768;

}

http {

sendfile on;

tcp_nopush on;

tcp_nodelay on;

keepalive_timeout 65;

types_hash_max_size 4096;

include /etc/nginx/mime.types;

default_type application/octet-stream;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE

ssl_prefer_server_ciphers on;

access_log /var/log/nginx/access.log;

error_log /var/log/nginx/error.log;

gzip on;

gzip_disable "msie6";

include /etc/nginx/conf.d/*.conf;

}

给helloworld项目设置反向代理规则

[root@bogon my_projects]# vim /etc/nginx/conf.d/helloworld.conf

server {

listen 80;

server_name 127.0.0.1; # ext-sandbox.51zhuan.com;

charset utf-8;

access_log /var/log/nginx/helloworld/nginx.access.log;

error_log /var/log/nginx/helloworld/nginx.error.log;

# Load configuration files for the default server block.

location / {

proxy_pass_header Server;

proxy_set_header Host $http_host;

proxy_redirect off;

proxy_set_header X-Real-IP $remote_addr;

proxy_pass http://127.0.0.1:1001;

}

error_page 404 /404.html;

location = /40x.html {

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

}

检查配置文件nginx.conf的正确性：

[root@bogon my_projects]# nginx -t

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conf test is successful

2.1.3启动 Nginx

Nginx 从新启动命令以下：

[root@bogon my_projects]# service nginx restart

访问站点

[root@bogon my_projects]# curl http://127.0.01

<!DOCTYPE html>

<html>

<head>

<title>Welcome to nginx!</title>

<style>

body {

width: 35em;

margin: 0 auto;

font-family: Tahoma, Verdana, Arial, sans-serif;

}

</style>

</head>

<body>

<h1>Welcome to nginALLOWED_HOSTS = ['*']

x!</h1>

<p>If you see this page, the nginx web server is successfully installed and

working. Further configuration is required.</p>

<p>For online documentation and support please refer to

<a href="http://nginx.org/">nginx.org</a>.<br/>

Commercial support is available at

<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>

</body>

</html>

1.2.4 Nginx 其余命令

如下包含了 Nginx 经常使用的几个命令：

nginx -s reload # 从新载入配置文件

nginx -s reopen # 重启 Nginx

nginx -s stop # 中止 Nginx

service nginx status # 使用service查看nginx的状态

service nginx start # 使用service启动nginx

service nginx stop # 使用service中止nginx

service nginx restart # 使用service重启nginx

3.nginx+uwsgi优化

3.1 http与socket路由方式

3.1.1 http路由方式

使用http路由方式，uwsgi与nginx之间也会以http协议进行通讯。在uwsgi内部会生成一个监听已配置端口的服务器，转发请求到由master进程管理的4个uWSGI worker组成的池中。

3.1.2 socket路由方式

nginx与uwsgi间使用wsgi协议进行socket通讯，uwsgi接受到nginx传输的数据后直接由workers进程对数据进行处理，而后返回给nginx，nginx再解析成http返回给客户端。

修改uwsgi启动方式为socket，将uwsgi.ini中的http = 127.0.0.1:1001修改成socket = 127.0.0.1:1001

[root@bogon my_projects]# cd /home/my_projects/helloworld

[root@bogon helloworld]# cat uwsgi.ini

[uwsgi]

chdir = /home/agu/temp/helloworld

module = helloworld.wsgi:application

reload-mercy = 10

user = root

uid = root

master = True

harakiri-verbose = true

post-buffering = 65536

buffer-size = 65536

harakiri = 30

threads = 30

processes = 8

socket = 127.0.0.1:1001

# http = 127.0.0.1:1001

chmod-socket = 664

vacuum = true

重启服务

[root@bogon helloworld]# supervisorctl restart helloworld

在使用socket方式时nginx须要与uwsgi适配，在/etc/nginx/nginx.conf文件http模块中加入如下两行

include uwsgi_params;

uwsgi_read_timeout 120;

建立/etc/nginx/uwsgi_params该文件内容以下

uwsgi_param QUERY_STRING $query_string;

uwsgi_param REQUEST_METHOD $request_method;

uwsgi_param CONTENT_TYPE $content_type;

uwsgi_param CONTENT_LENGTH $content_length;

uwsgi_param REQUEST_URI $request_uri;

uwsgi_param PATH_INFO $document_uri;

uwsgi_param DOCUMENT_ROOT $document_root;

uwsgi_param SERVER_PROTOCOL $server_protocol;

uwsgi_param REQUEST_SCHEME $scheme;

uwsgi_param HTTPS $https if_not_empty;

uwsgi_param REMOTE_ADDR $remote_addr;

uwsgi_param REMOTE_PORT $remote_port;

uwsgi_param SERVER_PORT $server_port;

uwsgi_param SERVER_NAME $server_name;

nginx重载配置文件或重启nginx

[root@bogon helloworld]# nginx -s reload

在另外一个终端中查看结果，执行

[root@bogon hellowold]# curl http://127.0.0.1:1001 |grep successfully

<h2>The install worked successfully! Congratulations!</h2>

3.1 uwsgi进程线程数配置优化

3.1.1 uwsgi进程线程数配置

配置文件路径/home/my_projects/helloworld/uwsgi.ini

修改processes进程数与threads进程数

3.1.2测试服务器配置与结果报表

3.1.2测试结论

当进程数为4线程数为60时平均吞吐量最高，错误率也相对较低。因此得出结论当进程数为CPU核数2倍时，效率最高。

3.2 Django数据库链接参数CONN_MAX_AGE优化

3.2.1 CONN_MAX_AGE参数配置

路径/home/my_projects/helloworld/helloworld/settings.py database参数修改成如下内容

DATABASES = {

"default": {

"ENGINE": "django.db.backends.mysql",

"NAME": "integral_wall_sandbox",

"USER": "root",

"PASSWORD": "ztwl",

"HOST": "127.0.0.1",

"PORT": "3306",

"CONN_MAX_AGE": 28800, # 8 hours

}

测试报表

测试结论

1.当django数据库链接中配置CONN_MAX_AGE参数时，只有views业务中从sql执行开始到本次请求结束的时间超过CONN_MAX_AGE时会当即断开与数据库的链接。若是执行时间少于CONN_MAX_AGE不会断开链接。

2.当django数据库链接中不配置CONN_MAX_AGE参数时，views业务中执行sql时会创建起与数据库的链接，数据库操做结束后当即断开链接。

3.配置了CONN_MAX_AGE参数省去了频繁链接与断开数据的时间，因此服务器吞吐量略大于不配置CONN_MAX_AGE时的吞吐量。

4.uwsgi实际启动进程数=1个主进程+uwsgi配置的进程数；主进程始终1个线程，子进程中的线程始终为uwsgi中配置的线程数。

3.3 nginx中worker_rlimit_nofile与worker_connections优化

3.3.1 worker_rlimit_nofile解释

本参数为nginx工做进程改变打开最多文件描述符数目的限制。用来在不重启主进程的状况下增长限制，小于等于unix系统打开文件最大进程数的配置

3.3.2 当worker_rlimit_nofile不配置或者配置量太小，在高并发下可能出现的异常

异常日志路径/var/log/nginx/helloworld/nginx.access.log

异常日志：socket() failed (24: Too many open files) while connecc ting to upstream

3.3.3 worker_connections含义

每个worker进程能并发处理（发起）的最大链接数。应小于等于worker_rlimit_nofile值。

3.3.4 当worker_connections配置量太小，在高并发下可能出现的异常

异常日志路径/var/log/nginx/helloworld/nginx.access.log

异常日志：socket() failed (24: Too many open files) while connecc ting to upstream

更多相关文章...