基于Varnish 3.0.4的网站静态加速

Varnish 的一些特色:
php

(1)、是基于内存缓存,重启后数据将消失;css

(2)、利用虚拟内存方式,I/O 性能好;html

(3)、支持设置 0~60 秒内的精确缓存时间;web

(4)、VCL 配置管理比较灵活;正则表达式

(5)、32位机器上缓存文件大小为最大2G;算法

(6)、具备强大的管理功能,例如 top、stat、admin、list 等;shell

(7)、状态机设计巧妙,结构清晰;vim

(8)、利用二叉堆管理缓存文件,达到积极删除目的。 后端

-----------------------------------------------------------------------------------------------------------------------------------------------缓存

Varnish 的 Storage 方式可分为两种:

1)、Malloc 经过 malloc 获取内存;

2)、Mmap file 建立大文件,经过二分法分段映射成 1G 之内的大块。


以 Mmap file 的缓存方式启动 I/O 也会造成瓶颈,缘由主要是 Varnish 缓存的数据先会刷到磁盘上,而后在一次性读到内存中,这在访问量大的时候同时也会对 I/O 形成很大的压力。Malloc 缓存方式虽然对 I/O 没有压力,因全部缓存数据都写到内存中。


Malloc 方式启动:

/usr/local/varnish/sbin/varnishd -u nobody -g nogroup -f /usr/local/varnish/etc/varnish.vcl -s malloc,4G -w 50,51200,120 -a 192.168.10:80 -T 127.0.0.1:8080


Mmap file 方式启动:

/usr/local/varnish/sbin/varnishd -u nobody -g nogroup -f /usr/local/varnish/etc/varnish.vcl -s file,/data/varnish/varnish_storage.bin,4G -w 50,51200,120 -a 192.168.10:80 -T 127.0.0.1:8080

-----------------------------------------------------------------------------------------------------------------------------------------------

Varnish 进程的工做模式:

Varnish 启动或有2个进程 master(management) 进程和 child(worker) 进程。master 读入存储配置命令,进行初始化,而后 fork,监控 child。child 则分配线程进行 cache 工做,child 还会作管理线程和生成不少 worker 线程。


child 进程主线程初始化过程当中,将存储大文件整个加载到内存中,若是该文件超出系统的虚拟内存,则会减小原来配置 mmap 大小,而后继续加载,这时候建立并初始化空闲存储结构体,放在存储管理的 struct 中,等待分配。


接着 Varnish 某个负责接受新 http 链接的线程开始等待用户,若是有新的 http 链接,可是这个线程只负责接收,而后唤醒等待线程池中的 work 线程,进行请求处理。


worker 线程读入 uri 后,将会查找已有的 object,命中直接返回,没有命中,则会从后端服务器中取出来,放到缓存中。若是缓存已满,会根据 LRU 算法,释放旧的 object。对于释放缓存,有一个超时线程会检测缓存中全部 object 的生命周期,若是缓存过时 (ttl),则删除,释放相应的存储内存。

-----------------------------------------------------------------------------------------------------------------------------------------------

Varnish 的优势:

(1)、Varnish 的稳定性很高,二者在完成相同负荷的工做时,Squid 服务器发生故障的概率要高于 Varnish,由于使用Squid 要常常重启;

(2)、Varnish 访问速度更快,Varnish 采用了Visual Page Cache技术,全部缓存数据都直接从内存读取,而 Squid 是从硬盘读取,于是 Varnish 在访问速度方面会更快;

(3)、Varnish 能够支持更多的并发链接,由于 Varnish 的 TCP 链接释放要比 Squid 快。于是在高并发链接状况下能够支持更多 TCP 链接;

(4)、Varnish 能够经过管理端口,使用正则表达式批量的清除部分缓存,而 Squid 是作不到的;

(5)、Squid 属因而单进程使用单核 CPU,但 Varnish 是经过 fork 形式打开多进程来作处理,因此是合理的使用全部核来处理相应的请求。


与传统的 Squid 相比,Varnish 也有缺点:

1)、Varnish 进程一旦 Hang、Crash 或者重启,缓存数据都会从内存中彻底释放,此时全部请求都会发送到后端服务器,在高并发状况下,会给后端服务器形成很大压力;

2)、在 Varnish 使用中,若是单个 url 的请求经过 HA/F5(负载均衡)、每次请求不一样的 Varnish 服务器中,被请求Varnish 服务器都会被穿透到后端,并且一样的请求会在多台服务器上缓存,也会形成 Varnish 的缓存的资源浪费,也会形成性能降低。


解决方案:

1)、综上所述在访问量很大的状况下推荐使用 Varnish 的内存缓存方式启动,并且后面须要跟多台 Squid 服务器。主要为了防止前面的 Varnish 服务被重启的状况下,前期确定会有不少的穿透,这样 Squid 能够担当第二层 CACHE,并且也弥补了 Varnish 缓存在内存中重启都会释放的问题;

2)、这样的问题能够在负载均衡上作 url 哈希,让单个 url 请求固定请求到一台 Varnish 服务器上,能够解决该问题。


注:以上内容是在借鉴别人总结的基础上,作了一些精简、修正,并加入了一些本身的总结。


-----------------------------------------------------------------------------------------------------------------------------------------------

系统环境:SUSE Linux Enterprise Server 10 SP1 (x86_64)

硬件配置:8G内存、8核CPU


一、编译安装

ftp://invisible-island.net/ncurses/ncurses-5.9.tar.gz

# tar xvzf ncurses-5.9.tar.gz

# ./configure --prefix=/usr/local

# make LAGS=-fPIC

# make LAGS=-fPIC install


http://downloads.sourceforge.net/project/pcre/pcre/8.32/pcre-8.32.tar.gz

# tar xvzf  pcre-8.32.tar.gz

# cd pcre-8.32

# ./configure --prefix=/usr/local

# make

# make install


ftp://ftp.gnu.org/gnu/readline/readline-6.2.tar.gz

# tar xvzf readline-6.2.tar.gz

# ./configure --prefix=/usr/local

# make

# make install


http://repo.varnish-cache.org/source/varnish-3.0.4.tar.gz

# tar xvzf varnish-3.0.4.tar.gz

# ./configure --prefix=/usr/local/varnish

# make

# make install

-----------------------------------------------------------------------------------------------------------------------------------------------

二、后端WEB服务健康检测

# vim /usr/local/varnish/etc/health_check.vcl

probe backend_healthcheck {
    .interval = 5s;
    .timeout = 3s;
    .window = 10;
    .threshold = 8;
         
    .request =
    "GET /index.html HTTP/1.1"
    "Host: mycheckweb.mytest.com"
    "Connection: close"
    "Accept-Encoding: foo/bar";
}

-----------------------------------------------------------------------------------------------------------------------------------------------

三、后端WEB服务定义

# vim /usr/local/varnish/etc/hosts/10.160.22.88_8080.conf

backend WEBSRV_10_160_22_88_8080 {
    .host = "10.160.22.88";
    .port = "8080";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
    .connect_timeout = 50s;
    .between_bytes_timeout = 30s;
    .first_byte_timeout = 30s;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
    .probe = backend_healthcheck;
}

# vim /usr/local/varnish/etc/hosts/10.173.146.35_8080.conf

backend WEBSRV_10_173_146_35_8080 {
    .host = "10.173.146.35";
    .port = "8080";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
    .connect_timeout = 50s;
    .between_bytes_timeout = 30s;
    .first_byte_timeout = 30s;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
    .probe = backend_healthcheck;
}

-----------------------------------------------------------------------------------------------------------------------------------------------

四、集群定义

# vim /usr/local/varnish/etc/cluster.vcl

include "/usr/local/varnish/etc/health_check.vcl";
include "/usr/local/varnish/etc/hosts/10.160.22.88_8080.conf";
include "/usr/local/varnish/etc/hosts/10.173.146.35_8080.conf";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
director CLUSTER_BACKEND_SERVER round-robin {
    { .backend = WEBSRV_10_160_22_88_8080; }
    { .backend = WEBSRV_10_173_146_35_8080; }
}

-----------------------------------------------------------------------------------------------------------------------------------------------

五、Varnish主配置文件

# vim /usr/local/varnish/etc/varnish.vcl

include "/usr/local/varnish/etc/cluster.vcl";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
acl allow_purge_cache {
    "127.0.0.1";
    "10.0.0.0"/8;
    "172.0.0.0"/8;
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_recv {
    if (req.request == "PURGE") {
        if (!client.ip ~ allow_purge_cache) {
            error 405 "Not Allowed.";
        }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
        return (lookup);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.host ~ "^(.*).mytest.com") {
        set req.backend = CLUSTER_BACKEND_SERVER;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 动态资源直接抛到后端服务器
    if (req.url ~ "\.(php|asp|aspx|jsp|do|ashx|shtml)($|\?)") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 静态资源须要去除cookie信息
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        unset req.http.cookie;
        return (lookup);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Cache-Control ~ "no-cache") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "DELETE") {
        return (pipe);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request != "GET" && req.request != "HEAD") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Authorization || req.http.Cookie) {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 静态资源压缩
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)$") {
            remove req.http.Accept-Encoding;
        } elseif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elseif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            remove req.http.Accept-Encoding;
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 防盗链设置
    if (req.http.referer ~ "http://.*") {
        if (!(req.http.referer ~ "http://.*\.qq\.com" ||
            req.http.referer ~ "http://.*\.baidu\.com" ||
            req.http.referer ~ "http://.*\.google\.com.*" ||
            req.http.referer ~ "http://.*\.sogou\.com" ||
            req.http.referer ~ "http://.*\.soso\.com" ||
            req.http.referer ~ "http://.*\.so\.com")) {
            set req.http.host = "www.mytest.com";
            set req.url = "/";
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (!req.backend.healthy) {
        unset req.http.Cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 跳过缓存大文件
    if (req.http.x-pipe && req.restarts > 0) {
        unset req.http.x-pipe;
        return (pipe);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 若backend是健康的,则仅grace 5s,若是backend不健康,则grace 1m,主要用于提升并发时的吞吐率
    if (req.backend.healthy) {
        set req.grace = 5s;
    } else {
        set req.grace = 1m;
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_pipe {
    return (pipe);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_pass {
    if (req.request == "PURGE") {
        error 502 "PURGE on a passed object";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_hash {
    hash_data(req.url);
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Accept-Encoding ~ "gzip") {
        hash_data("gzip");
    } elseif (req.http.Accept-Encoding ~ "deflate") {
        hash_data("deflate");
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    return (hash);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_hit {
    if (req.request == "PURGE") {
        purge;
        error 200 "Purged.";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_miss {
    if (req.request == "PURGE") {
        purge;
        error 404 "Purged.";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_fetch {
    ## 确保全部Cache中的内容在TTL过时后5分钟内不被删除,以应对高并发的场合
    set beresp.grace = 5m;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (beresp.http.Set-Cookie) {
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 若是返回头有Cache-Control,则删除Set-Cookie头
    if (beresp.http.Cache-Control && beresp.ttl > 0s) {
        set beresp.grace = 1m;
        unset beresp.http.Set-Cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 不缓存大于10MB的资源文件
    if (beresp.http.Content-Length ~ "[0-9]{8,}") {
        set req.http.x-pipe = "1";
        return (restart);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.url ~ "\.(php|asp|aspx|jsp|do|ashx|shtml)($|\?)") {
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        unset beresp.http.set-cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 若是返回头没有Cache-Control,则标记为hit_for_pass,强制后续请求回源
    if ((!beresp.http.Cache-Control && !beresp.http.Expires) ||
         beresp.http.Pragma ~ "no-cache" ||
         beresp.http.Cache-Control ~ "(no-cache|no-store|private)") {
        set beresp.ttl = 120s;
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") {
        set beresp.ttl = 120s;
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 对不一样类型静态资源进行缓存时间设置
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico)($|\?)") {
        set beresp.ttl = 15m;
    } elseif (req.request == "GET" && req.url ~ "\.(gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        set beresp.ttl = 30m;
    } else {
        set beresp.ttl = 10m;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    return (deliver);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT from " + req.http.host;
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS from " + req.http.host;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 去掉没必要要的头信息
    unset resp.http.X-Powered-By;
    unset resp.http.Server;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    unset resp.http.Via;
    unset resp.http.X-Varnish;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    unset resp.http.Age;
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_error {
    if (obj.status == 503 && req.restarts < 5) {
        set obj.http.X-Restarts = req.restarts;
        return (restart);
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_init {
    return (ok);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_fini {
    return (ok);
}

-----------------------------------------------------------------------------------------------------------------------------------------------

六、Varnish启动参数配置文件

# vim /usr/local/varnish/etc/varnish.conf

# Configuration file for varnish
#
# /etc/init.d/varnishd expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Maximum number of open files (for ulimit -n)
NFILES=131072
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Maximum number of threads (for ulimit -u)
NPROCS="unlimited"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Set this to 1 to make init script reload try to switch vcl without restart.
# To make this work, you need to set the following variables
# explicit: VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_ADDRESS,
# VARNISH_ADMIN_LISTEN_PORT, VARNISH_SECRET_FILE, or in short,
# use Alternative 3, Advanced configuration, below
RELOAD_VCL=1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Main configuration file. You probably want to change it :)
VARNISH_VCL_CONF=/usr/local/varnish/etc/varnish.vcl
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=0.0.0.0
VARNISH_LISTEN_PORT=80
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=8080
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Shared secret file for admin interface
#VARNISH_SECRET_FILE=/etc/varnish/secret
#
# # The minimum number of worker threads to start
VARNISH_MIN_THREADS=10
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # The Maximum number of worker threads to start
VARNISH_MAX_THREADS=5000
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Idle timeout for worker threads
VARNISH_THREAD_TIMEOUT=120
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Cache file location
VARNISH_STORAGE_FILE=/data/varnish/varnish_storage.bin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=3G
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Backend storage specification
#VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Default TTL used when the backend does not specify one
VARNISH_TTL=120
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Other optimization parameter
HTTP_RESP_HDR_LEN="http_resp_hdr_len=8192"
HTTP_MAX_HDR="http_max_hdr=256"
HTTP_REQ_HDR_LEN="http_req_hdr_len=8192"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
THREAD_POOLS="thread_pools=8"
THREAD_POOL_MIN="thread_pool_min=50"
THREAD_POOL_MAX="thread_pool_max=5120"
THREAD_POOL_TIMEOUT="thread_pool_timeout=10"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
LRU_INTERVAL="lru_interval=20"
LISTEN_DEPTH="listen_depth=1024"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # DAEMON_OPTS is used by the init script.  If you add or remove options, make
# # sure you update this section, too.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
             -f ${VARNISH_VCL_CONF} \
             -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
             -t ${VARNISH_TTL} \
             -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
             -u nobody -g nogroup \
             -s ${VARNISH_STORAGE} \
             -p ${HTTP_RESP_HDR_LEN} \
             -p ${HTTP_MAX_HDR} \
             -p ${HTTP_REQ_HDR_LEN} \
             -p ${THREAD_POOLS} \
             -p ${THREAD_POOL_MIN} \
             -p ${THREAD_POOL_MAX} \
             -p ${THREAD_POOL_TIMEOUT} \
             -p ${LRU_INTERVAL} \
             -p ${LISTEN_DEPTH}"

-----------------------------------------------------------------------------------------------------------------------------------------------

七、Varnish 配置重载脚本

# vim /usr/local/varnish/sbin/varnish_reload_vcl

#!/bin/bash
#
# reload vcl revisited
# A script that loads new vcl based on data from /usr/local/varnish/etc/varnish.conf
# Ingvar Hagelund <ingvar@redpill-linpro.com>
#
# This is free software, distributed under the standard 2 clause BSD license,
# see the LICENSE file in the Varnish documentation directory
#
# The following environment variables have to be set:
# RELOAD_VCL, VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_PORT
# The following are optional:
# VARNISH_SECRET_FILE, VARNISH_ADMIN_LISTEN_ADDRESS
#
# Requires GNU bash and GNU date
#
                                                                                                                                                                                                                                                                                                                                                                                                                                          
debug=false
                                                                                                                                                                                                                                                                                                                                                                                                                                          
missing() {
    echo "Missing configuration variable: $1"
    exit 2
}
                                                                                                                                                                                                                                                                                                                                                                                                                                          
print_debug() {
    echo "
Parsed configuration:
RELOAD_VCL=\"$RELOAD_VCL\"
VARNISH_VCL_CONF=\"$VARNISH_VCL_CONF\"
VARNISH_ADMIN_LISTEN_ADDRESS=\"$VARNISH_ADMIN_LISTEN_ADDRESS\"
VARNISH_ADMIN_LISTEN_PORT=\"$VARNISH_ADMIN_LISTEN_PORT\"
VARNISH_SECRET_FILE=\"$VARNISH_SECRET_FILE\"
"
}
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Read configuration
. /usr/local/varnish/etc/varnish.conf
                                                                                                                                                                                                                                                                                                                                                                                                                                          
$debug && print_debug
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Check configuration
if [ ! "$RELOAD_VCL" = "1" ]; then
    echo "Error: RELOAD_VCL is not set to 1"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_VCL_CONF" ]; then
    echo "Error: VARNISH_VCL_CONF is not set"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ ! -s "$VARNISH_VCL_CONF" ]; then
    echo "Eror: VCL config $VARNISH_VCL_CONF is unreadable or empty"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_ADMIN_LISTEN_ADDRESS" ]; then
    echo "Warning: VARNISH_ADMIN_LISTEN_ADDRESS is not set, using 127.0.0.1"
    VARNISH_ADMIN_LISTEN_ADDRESS="127.0.0.1"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_ADMIN_LISTEN_PORT" ]; then
    echo "Error: VARNISH_ADMIN_LISTEN_PORT is not set"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
#elif [ -z "$VARNISH_SECRET_FILE" ]; then
#   echo "Warning: VARNISH_SECRET_FILE is not set"
#   secret=""
                                                                                                                                                                                                                                                                                                                                                                                                                                          
#elif [ ! -s "$VARNISH_SECRET_FILE" ]; then
#   echo "Error: varnish secret file $VARNISH_SECRET_FILE is unreadable or empty"
#   exit 2
else
#   secret="-S $VARNISH_SECRET_FILE"
    echo
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Done parsing, set up command
#VARNISHADM="varnishadm $secret -T $VARNISH_ADMIN_LISTEN_ADDRESS:$VARNISH_ADMIN_LISTEN_PORT"
VARNISHADM="/usr/local/varnish/bin/varnishadm -T $VARNISH_ADMIN_LISTEN_ADDRESS:$VARNISH_ADMIN_LISTEN_PORT"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Now do the real work
new_config="reload_$(date +%FT%H:%M:%S)"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Check if we are able to connect at all
if $VARNISHADM vcl.list > /dev/null; then
    $debug && echo vcl.list succeeded
else
    echo "Unable to run $VARNISHADM vcl.list"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.list | awk ' { print $3 } ' | grep -q $new_config; then
    echo Trying to use new config $new_config, but that is already in use
    exit 2
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
current_config=$( $VARNISHADM vcl.list | awk ' /^active/ { print $3 } ' )
                                                                                                                                                                                                                                                                                                                                                                                                                                          
echo "Loading vcl from $VARNISH_VCL_CONF"
echo "Current running config name is $current_config"
echo "Using new config name $new_config"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.load $new_config $VARNISH_VCL_CONF; then
    $debug && echo "$VARNISHADM vcl.load succeded"
else
    echo "$VARNISHADM vcl.load failed"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.use $new_config; then
    $debug && echo "$VARNISHADM vcl.use succeded"
else
    echo "$VARNISHADM vcl.use failed"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
$VARNISHADM vcl.list
echo Done
                                                                                                                                                                                                                                                                                                                                                                                                                                          
exit 0

-----------------------------------------------------------------------------------------------------------------------------------------------

八、Varnish启动脚本

# vim /etc/init.d/varnishd

#!/bin/bash
#
# varnish Control the Varnish Cache
#
# chkconfig: - 90 10
# description: Varnish is a high-perfomance HTTP accelerator
# processname: varnishd
# config: /usr/local/varnish/etc/varnish.conf
# PIDFILE: /var/run/varnishd.pid
#
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
                                                                                                                                                                                                                                                                                                                                                                                                              
BINFILE="/usr/local/varnish/sbin/varnishd"
PROG="varnishd"
                                                                                                                                                                                                                                                                                                                                                                                                              
RETVAL=0
                                                                                                                                                                                                                                                                                                                                                                                                              
GFILE="/usr/local/varnish/etc/varnish.conf"
PIDFILE="/var/run/varnish.pid"
LOCKFILE="/var/lock/varnish.lock"
RELOAD_EXEC="/usr/local/varnish/sbin/varnish_reload_vcl"
                                                                                                                                                                                                                                                                                                                                                                                                              
[[ -e ${GFILE} ]] && . ${GFILE}
                                                                                                                                                                                                                                                                                                                                                                                                              
start() {
    IS_EXIST=`ps -A -oppid,pid,cmd | grep sbin/${PROG} | grep -v grep`
    [[ -n "${IS_EXIST}" ]] && echo "The process of ${PROG} has been running." && exit 1
                                                                                                                                                                                                                                                                                                                                                                                                                 
    [[ ! -x ${BINFILE} ]] && echo ${BINFILE} has no found && exit 5
    [[ ! -f ${GFILE} ]] && echo ${GFILE} has no found && exit 6
                                                                                                                                                                                                                                                                                                                                                                                                                 
    echo -n "Starting Varnish Cache......"
                                                                                                                                                                                                                                                                                                                                                                                                                 
    ulimit -n ${NFILES:-131072}
    ulimit -l ${MEMLOCK:-82000}
    ulimit -u ${NPROCS:-unlimited}
                                                                                                                                                                                                                                                                                                                                                                                                                 
    if [[ "${DAEMON_OPTS}X" == "X" ]]; then
        echo -n "Please setting DAEMON_OPTS options in ${GFILE}"
        RETVAL=6
    else
        VARNISH_CACHE_DIR=`dirname $VARNISH_STORAGE_FILE`
        RETVAL=`grep -w '^VARNISH_STORAGE' ${GFILE} | grep malloc`
        if [[ "${RETVAL}X" = "X" ]]; then
            mkdir -p ${VARNISH_CACHE_DIR} && chown -R nobody:nogroup ${VARNISH_CACHE_DIR}
        else
            [[ -e ${VARNISH_CACHE_DIR} ]] && rm -rf ${VARNISH_CACHE_DIR}
        fi
                                                                                                                                                                                                                                                                                                                                                                                                                     
        ${BINFILE} -P ${PIDFILE} $DAEMON_OPTS >/dev/null 2>&1
        RETVAL=$?
        echo
        [[ $RETVAL -eq 0 ]] && touch ${LOCKFILE}
    fi
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
stop() {
    echo -n "Stopping Varnish Cache......"
    /sbin/killproc -QUIT ${PROG}
    RETVAL=$?
    echo
    [[ $RETVAL -eq 0 ]] && rm -f ${LOCKFILE}
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
restart() {
    stop
    sleep 1
    start
}
                                                                                                                                                                                                                                                                                                                                                                                                              
reload() {
    [[ "$RELOAD_VCL" = "1" ]] && $RELOAD_EXEC || restart
                                                                                                                                                                                                                                                                                                                                                                                                                 
    RETVAL=$?
    echo
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
configtest() {
    if [[ -f ${GFILE} ]]; then
        ${BINFILE} -f ${GFILE} -C -n /tmp >/dev/null 2>&1
        RETVAL=$?
        [[ $? -eq 0 ]] && echo "The syntax is ok." || echo "The syntax is error."
    else
        RETVAL=$?
        echo "The config file: ${GFILE} is no exists."
    fi
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
case "$1" in
start|stop|restart|reload|configtest)
    $1
    ;;
                                                                                                                                                                                                                                                                                                                                                                                                                 
*)
    echo "Usage: $0 {start|stop|restart|reload|configtest}"
    exit 2
esac
                                                                                                                                                                                                                                                                                                                                                                                                              
exit $RETVAL

# chmod +x /etc/init.d/varnishd

# chkconfig --add varnishd

-----------------------------------------------------------------------------------------------------------------------------------------------

九、Varnish服务健康检查脚本

# vim /usr/local/varnish/sbin/check_varnish_health.sh

#!/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
                                                                                                                                                                                                                                                                                                                                                                                                       
## varnish启动参数配置文件
VARNISH_CONFIG="/usr/local/varnish/etc/varnish.conf"
                                                                                                                                                                                                                                                                                                                                                                                                       
IS_DEBUG=false
                                                                                                                                                                                                                                                                                                                                                                                                       
[[ -e ${VARNISH_CONFIG} ]] && . ${VARNISH_CONFIG}
                                                                                                                                                                                                                                                                                                                                                                                                       
${IS_DEBUG} && echo "监听地址:${VARNISH_LISTEN_ADDRESS}  监听端口:${VARNISH_LISTEN_PORT}"
LAN_IPADDR=`/sbin/ifconfig eth1 | awk -F ':' '/inet addr/{print $2}' | sed 's/[a-zA-Z ]//g'`
                                                                                                                                                                                                                                                                                                                                                                                                       
RETVAL=`nmap --system-dns -sT -p ${VARNISH_LISTEN_PORT} ${LAN_IPADDR} | grep open`
[[ -z ${RETVAL} ]] && /sbin/service varnishd restart >/dev/null 2>&1

# crontab -e

*/5 * * * * /usr/local/varnish/sbin/check_varnish_health.sh >/dev/null 2>&1

相关文章
相关标签/搜索