TCP四层代理透传客户端真实IP

缘起

最近公司准备自建邮箱，大概十年前折腾过自建邮箱的事情，感受坑不少，此次体会了下确实不少，特别是如今管局对邮箱端口管控愈来愈严格的状况下，不少以前能行得通的方案须要修改下。
因为SMTP、POP三、IMAP等协议须要作四层代理且还须要能穿透客户端真实IP，硬件网络层代理软件又不能使用的状况下只能考虑软代理，下面是记录的两种解决方案。

方案

Nginx解决方案

1.注意nginx编译时须要加上stream模块及stream_realip_module模块;一个用来四层负载，一个用来获取客户端真实IP
2.开启透传功能proxy_protocol on，用于将链接信息从请求链接的源传递到请求链接到的目标

具体配置以下

# /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.16.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx  --with-stream --with-stream_realip_module

# cat /usr/local/nginx/conf/nginx.conf

user  nginx;
worker_processes  auto;

events {
    worker_connections  10240;
    }
include tcp/*.conf;

# cat /usr/local/nginx/conf/tcp/mail.conf

stream {
    log_format proxy '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received '
                 '$session_time "$upstream_addr" '
                 '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';

    access_log /usr/local/nginx/logs/mail.log proxy;

    upstream mail_http {
        server 192.168.188.88:80 ;
    }
    server {
        listen 80;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_pass mail_web;
    }

    upstream mail_https {
        server 192.168.188.88:443 ;
    }
    server {
        listen 443;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_pass mail_https;
    }

    upstream mail_smtp {
        server 192.168.188.88:25 ;
    }
    server {
        listen 25 ;
        proxy_connect_timeout 10s;
        proxy_timeout 300s;
        proxy_protocol on ;
        proxy_pass mail_smtp;
    }

    upstream mail_smtps {
        server 192.168.188.88:465 ;
    }
    server {
        listen 465;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_smtps;
    }

    upstream mail_pop3 {
        server 192.168.188.88:110 ;
    }
    server {
        listen 110;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_pop3;
    }

    upstream mail_pop3s {
        server 192.168.188.88:995 ;
    }
    server {
        listen 995;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_pop3s;
    }

    upstream mail_imap {
        server 192.168.188.88:143 ;
    }
    server {
        listen 143;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_imap;
    }

    upstream mail_imaps {
        server 192.168.188.88:993 ;
    }
    server {
        listen 993;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_imaps;
    }

}

Haproxy解决方案

1.开通tcp 模式 mode tcp
2.开启透传 send-proxy
注意：基本上haproxy参数需结合haproxy -c测试，一条条的来过才能肯定最终哪一个参数起做用

# haproxy -v
HA-Proxy version 1.5.18 2016/05/10
Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>

# cat /etc/haproxy/haproxy.cf
listen stats
 mode http
 bind 0.0.0.0:9999
 stats enable
 log global
 stats uri     /haproxy-status
 stats auth    haadmin:yourpassword

listen WEB
    bind 80
    mode http
    server web01 192.168.188.88:80  check 

listen WEBSSL
    bind 443
    mode tcp
    server webssl01 192.168.188.88:443  check 

listen MAIL
    bind 25
    mode tcp
    server mail01 192.168.188.88:25  check send-proxy

listen MAILS
    bind 465
    mode tcp
    server mailssl01 192.168.188.88:465  check send-proxy

listen POP3
    bind 110
    mode tcp
    server pop301 192.168.188.88:110  check send-proxy

listen POP3S
    bind 995 
    mode tcp
    server pop3ssl01 192.168.188.88:995  check send-proxy

listen IMAP
    bind 143
    mode tcp
    server imap01 192.168.188.88:143  check send-proxy

listen IMAPS
    bind 993
    mode tcp
    server imapssl01 192.168.188.88:993  check send-proxy

尾声

固然了，这只是自建邮件服务器其中一个小小的问题，基本上一步一个问题，因此如今自建邮箱愈来愈少了，基本上都被云邮箱取代了，就连Postfix 2012年的时候都中止更新维护了，估计再后来懂这门手艺的人会愈来愈少了。