nginx + lvs （dr)

环境：

      两台   centos7.1 部署 nginx 

            ( web1   ip 172.17.12.108 hostname:primordial

               web2  ip  172.17.12.111 hostname:copy )

      一台   centos 7.1 部署 lvs

( lvs   ip   172.17.12.110  hostname:lvs )

       vip  172.17.12.121

primordial /copy:

yum install nginx -y

[root@primordial ~]#echo "web1" >/usr/share/nginx/html/index.html 

[root@copy ~]# echo "web2" >/usr/share/nginx/html/index.html

systemctl restart nginx

[root@copy ~]# netstat -ntlp |grep 80 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 2838/nginx: master

在浏览器中测试访问nginx服务器IP，显示以下：

若不能正常访问，请检查 selinux是否为 disabled；

                                      iptables规则是否限制

                                      firewall是否关闭

                                      80端口是否被占用

lvs:

[root@lvs ~]# ifconfig eno16777736:0 172.17.12.121 #配置vip

[root@lvs ~]# ifconfig

eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 172.17.12.110 netmask 255.255.255.0 broadcast 172.17.12.255

inet6 fe80::20c:29ff:fee9:31c7 prefixlen 64 scopeid 0x20<link>

ether 00:0c:29:e9:31:c7 txqueuelen 1000 (Ethernet)

RX packets 5624 bytes 934642 (912.7 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 1333 bytes 127248 (124.2 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

 

eno16777736:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 172.17.12.121 netmask 255.255.0.0 broadcast 172.17.255.255

ether 00:0c:29:e9:31:c7 txqueuelen 1000 (Ethernet)

 

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536

inet 127.0.0.1 netmask 255.0.0.0

inet6 ::1 prefixlen 128 scopeid 0x10<host>

loop txqueuelen 0 (Local Loopback)

RX packets 12 bytes 1260 (1.2 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 12 bytes 1260 (1.2 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

 

上述配置不生效可尝试括号中方法：

（ ifconfig eno16777736:0 down

ifconfig eno16777736:0 $vip broadcast $vip netmask 255.255.255.255 up

route add -host $vip dev eno16777736:0 ）

yum install ipvasdm -y

[root@lvs ~]# ipvsadm -A -t 172.17.12.121:80 -s rr


[root@lvs ~]# ipvsadm -a -t 172.17.12.121:80 -r 172.17.12.108 -g


[root@lvs ~]# ipvsadm -a -t 172.17.12.121:80 -r 172.17.12.111 -g


[root@lvs ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.17.12.121:80 rr -> 172.17.12.108:80 Route 1 0 0 -> 172.17.12.111:80 Route 1 0 0




echo "1" >/proc/sys/net/ipv4/ip_forward #开启端口间转发功能

sysctl -p  #生效

primordial /copy:

[root@primordial ~]# ifconfig lo:0 172.17.12.121 netmask 255.255.255.255 broadcast 172.17.12.121

[root@primordial ~]# ifconfig

eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 172.17.12.108 netmask 255.255.255.0 broadcast 172.17.12.255

inet6 fe80::20c:29ff:feb0:5eaa prefixlen 64 scopeid 0x20<link>

ether 00:0c:29:b0:5e:aa txqueuelen 1000 (Ethernet)

RX packets 20343 bytes 8995179 (8.5 MiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 6642 bytes 488054 (476.6 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

 

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536

inet 127.0.0.1 netmask 255.0.0.0

inet6 ::1 prefixlen 128 scopeid 0x10<host>

loop txqueuelen 0 (Local Loopback)

RX packets 26 bytes 2684 (2.6 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 26 bytes 2684 (2.6 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

 

lo:0: flags=73<UP,LOOPBACK,RUNNING> mtu 65536

inet 172.17.12.121 netmask 255.255.255.255

loop txqueuelen 0 (Local Loopback)

echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce

echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore

echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

sysctl -p

添加路由：

route add default gw 172.17.12.121

route add -host 172.17.12.121 dev lo:0

后记：

   后面因为场地问题实验环境不变，vip换成192.168.37.131，web1 ip换成192.168.37.128，web2 ip换成192.168.37.129，lvs ip换成192.168.37.130

实验中出现的问题：

1.当配置没有问题，浏览器始终访问至同一台web时。可清理Windows本地ARP缓存，在cmd界面输入：arp -d 或在浏览器强制刷新 ：Ctrl+F5

2.当在lvs服务器上curl  vip能够实现下图中轮询的调度而浏览器访问始终在同一台web服务器上的状况，缘由是lvs配置vip的虚拟网卡down掉了

从新生效后lvs本机不能curl vip，浏览器可实现负载轮询

3.对于ifconfig配置的ip是临死生效不永久的问题，可采用将配置信息写成脚本，放在开机启动目录下

4.ip_forward 开启路由转发功能在lvs服务器上配置

5.新建集群时，ipvsadm -A -t $vip -s scheduler [-p ]

      只要后面跟上-p选项，即表明开启长链接