centos实现三个节点高可用

centos实现三个节点高可用

使用的资源为keepalived和nginx

高可用主机IP地址
192.168.136.131
192.168.136.133
192.168.136.134

nginx负载均衡配置
192.168.136.131
192.168.136.133
192.168.136.134
三台主机的80端口对应

192.168.136.131:8080
192.168.136.131:8083
192.168.136.134:8080
192.168.136.134:8084
192.168.136.133:8080
192.168.136.133:8081

nginx的Web服务器
192.168.136.131的8080、8083
192.168.136.134的8080、8084
192.168.136.133的8080、8081

一、每一个节点安装nginx和安装keepalived

1.1 安装nginx 1.16.1 ,参考
http://www.javashuo.com/article/p-xvbdwzqu-er.html
1.2 安装keepalived

yum -y install keepalived
启动keepalived

systemctl start keepalived
systemctl enable keepalived

二、修改keepalived配置

2.1 修改master节点的/etc/keepalived/keepalived.conf配置文件信息

! Configuration File for keepalived
vrrp_script check_nginx_alive {
    script "/usr/bin/check_nginx_alive.sh"
    interval 3
    weight -10
}

global_defs {
   router_id lbs_nginx
}

vrrp_instance VI_1 {
    state MASTER
    interface ens32
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.136.125
    }
    track_script {
        check_nginx_alive
    }

}


virtual_server 192.168.136.125 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    persistence_timeout 50
    protocol TCP
}

2.2 增长master节点的 执行脚本 /usr/bin/check_nginx_alive.sh,不管主节点仍是备节点都须要增长该文件

/usr/bin/check_nginx_alive.sh

#!/bin/sh
 
PATH=/bin:/sbin:/usr/bin:/usr/sbin
 
A=`ps -C nginx --no-header |wc -l`
 
if [ $A -eq 0 ]
   then
     echo 'nginx server is died'
     killall keepalived
fi

2.3 修改backup节点的/etc/keepalived/keepalived.conf配置文件信息
注意，不一样的备节点的优先级不同， priority 20

! Configuration File for keepalived
vrrp_script check_nginx_alive {
    script "/usr/bin/check_nginx_alive.sh"
    interval 3
    weight -10
}

global_defs {
   router_id lbs_nginx
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens32
    virtual_router_id 51
    priority 20
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.136.125
    }
    track_script {
        check_nginx_alive
    }

}


virtual_server 192.168.136.125 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    persistence_timeout 50
    protocol TCP
}

三、 增长nginx的配置，

3.1 修改nginx的配置文件以下

default.conf 8080端口
default_8084.conf 8084端口
default_lbs.conf 负载均衡配置

server {
    listen       8080;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}

3.2 负载均衡端口配置,负载均很名字和要一致，这里是nginx_lbs

upstream nginx_lbs { 
        server 192.168.136.131:8080 weight=1;
        server 192.168.136.131:8083 weight=1;
        server 192.168.136.134:8080 weight=1;
        server 192.168.136.134:8084 weight=1;
        server 192.168.136.133:8080 weight=2;
        server 192.168.136.133:8081 weight=2;
  }


server {
    listen       80;
    server_name  localhost;

    client_max_body_size 50m;
    client_body_buffer_size 256k;
    location / {
       proxy_pass        http://nginx_lbs;
       proxy_set_header X-Forwarded-For $remote_addr;
       proxy_set_header Host $host;
    }
}

四、启动keepalived

4.1 确认nginx的keepalived是否开放
通过查看能够看到131和134的 nginx.conf中的 keepalive_timeout 65;，至关于打开了keepalive

4.2 启动keepalived

启动Master 192.168.136.131和Backup 192.168.136.134 和 Backup 192.168.136.133的 Keepalived节点
systemctl start keepalived

五、查看keepaliced是否已经放开

ip addr 看到地址中是否有浮动地址

http://192.168.136.125能够正常访问了，能够将每一个网站下的index.html文件增长相应标识

六、异常状况查看

6.1 每台主机只能访问本主机全部网站和其余主机个别网站，后来发现缘由为，防火墙未打开
修改iptables后正常，
命令为：

vim /etc/sysconfig/iptables 
        添加8080和8083端口 
            -A INPUT -m state –state NEW -m tcp -p tcp –dport 8080 -j ACCEPT 
            -A INPUT -m state –state NEW -m tcp -p tcp –dport 8083 -j ACCEPT
        启动IPTABLES
            systemctl enable iptables.service
            systemctl start iptables.service
        #重启防火墙使配置文件生效 
            systemctl restart iptables.service

6.2 怎么确认目前的通讯信息正常
查看网卡接收的信息
安装tcpdump
yum install -y tcpdump
查看端口vrrp信息
tcpdump -i ens32 vrrp -n
正常状况下，只有一个IP会出如今信息中

6.3 查看keepalived的运行日志
tail -f /var/log/messages

6.4 tcpdump命令看到多个IP的信息，而且主备keepalived服务器上的虚拟IP都增长上了
通过核实缘由keepalived须要单独的端口进行通讯，目前使用112
经过iptables增长后正常
-A INPUT -p 112 -j ACCEPT

6.5 /etc/keepalived/keepalived.conf 设置中不能增长real_server这个参数，增长real_server后报错，估计目前使用广播和组播进行确认，不须要设置具体的IP地址

估计是哪一个版本开始就不用设置了或者换了参数了