SpringBoot + Dubbo的项目如何优雅停机

什么是优雅停机？

在web服务（Http协议）上线的时候，会经过kill命令杀死进程,这个时候在已经accept的请求还在线程池里面，咱们要保证这部分请求正常处理而且返回数据以后再停机.

dubbo服务（Tcp协议）也是一样的道理.

优雅停机包括：线程池的优雅关闭，数据库链接池的关闭，数据源的关闭，kafka链接的关闭....

没有优雅停机的现象

1.客户端拿不到数据

2.数据源报已经关闭

Failed to obtain JDBC Connection; nested exception is com.alibaba.druid.pool.DataSourceClosedException: dataSource already closed

JVM的钩子

Runtime.addShutDownHook

咱们的服务是基于SpringBoot+Dubbo+业务线程池来实现的.

dubbo框架提供了destroyAll来实现自身的优雅关闭，Spring容器对Bean的优雅关闭是经过@PreDestory来实现的，咱们本身建立的ThreadPool也能够经过Runtime.addShutDownHook来实现优雅关闭。

可是这里有一个关键的问题是这三者是并行执行的，dubbo在进入优雅停机状态中的时候已经中止接收新的业务请求,然而已经接收的请求须要继续处理，可是有可能此时Spring的优雅关闭已经执行完成，致使在处理请求的时候出现异常（好比DataSource已经close了）。

那如何保证Spring容器等待dubbo优雅关闭执行完成之后再执行bean的@PreDestory方法（销毁bean）呢？下面有请Spring的ApplicationListener！

 

Spring应用生命周期

Spring能够经过继承下面这个接口来实现对应用声明周期的回调

public interface ApplicationListener<E extends ApplicationEvent> extends EventListener {
    void onApplicationEvent(E var1);
}

ContextClosedEvent是在全部bean执行PreDestory以前发出的事件广播.咱们在这个事件回调中执行Dubbo的优雅关闭，就不会出现数据源已经关闭的异常.

SpringBoot + Dubbo 服务的优雅关闭

思路很简单，咱们只要监听Spring容器的声明周期,在容器启动的时候把Dubbo注册到JVM的shutdown hook删除，而后在ContextClosedEvent中执行Dubbo的优雅关闭.

@Configuration
@Slf4j
public class DefaultSpringDubboConfigurations {

    @Bean
    DubboShutdownListener dubboShutdownListener() {
        return new DubboShutdownListener();
    }

    public static class DubboShutdownListener implements ApplicationListener, PriorityOrdered {

        @Override
        public void onApplicationEvent(ApplicationEvent event) {
            if (event instanceof ApplicationStartedEvent) {
                Runtime.getRuntime().removeShutdownHook(DubboShutdownHook.getDubboShutdownHook());
                log.info("dubbo default shutdown hook removed,will be managed by spring");
            } else if (event instanceof ContextClosedEvent) {
                log.info("start destroy dubbo on spring close event");
                DubboShutdownHook.getDubboShutdownHook().destroyAll();
                log.info("dubbo destroy finished");
            }
        }

        @Override
        public int getOrder() {
            return 0;
        }
    }
}

 

业务线程池如何优雅关闭？

错误的作法：

Runtime.getRuntime().addShutdownHook(new Thread(){
           // 线程池虽然关闭，可是队列中的任务任然继续执行，因此用 shutdown()方式关闭线程池时须要考虑是不是你想要的效果
            //若是但愿当即中止，抛弃队列中的任务，可使用shutdownNow()
            threadPoolExecutor.shutdown();
        });

上面的代码忽视了多个钩子函数是并发执行的问题,线程池的业务逻辑可能须要数据源连接、redis连接等，可是这个时候有可能数据源已经关闭了。

正确的作法：

@PostConstruct
    public void afterPropertiesSet() throws Exception {
        DEAL_EVENT_THREAD_POOL = ThreadPoolUtils.newExecutor(CORE_POOL_SIZE, MAXIMUM_POOL_SIZE, KEEP_ALIVE_TIME, TimeUnit.SECONDS,
                QUEUE_MAX_SIZE, "DealEventLogTask-service");
    }

    @PreDestroy
    public void destroy() throws Exception {
        ThreadPoolUtils.stop(DEAL_EVENT_THREAD_POOL);
    }

SpringBoot内嵌Tomcat的优雅停机

@Configuration
public class DefaultTomcatFactoryConfigurations {

    @Bean
    TomcatServletWebServerFactory tomcatServletWebServerFactory(@Value("${server.port}") int port,
            @Qualifier("gracefulShutdownListener") GracefulShutdownListener gracefulShutdownListener) {

        TomcatServletWebServerFactory factory = new TomcatServletWebServerFactory();
        factory.setPort(port);
        factory.setProtocol("org.apache.coyote.http11.Http11Nio2Protocol");

        List<LifecycleListener> contextLifecycleListeners = Lists.newArrayList(new VersionLoggerListener(),
                new JreMemoryLeakPreventionListener(), new ThreadLocalLeakPreventionListener());
        factory.setContextLifecycleListeners(contextLifecycleListeners);

        factory.addConnectorCustomizers(gracefulShutdownListener);

        //  RFC 6265 对cookie domain有较多限制，其中一条是不能以 "." 开头，与com.kuaikan.common.utils.web.CookieUtils有冲突
        factory.addContextCustomizers(context -> context.setCookieProcessor(new LegacyCookieProcessor()));

        return factory;
    }

    @Bean
    GracefulShutdownListener gracefulShutdownListener(
            @Value("${server.shutdown-wait-seconds:30}") int shutdownWaitSeconds) {
        GracefulShutdownListener gracefulShutdownListener = new GracefulShutdownListener();
        gracefulShutdownListener.setShutdownWaitSeconds(shutdownWaitSeconds);
        return gracefulShutdownListener;
    }

    @Slf4j
    public static class GracefulShutdownListener
            implements TomcatConnectorCustomizer, ApplicationListener<ContextClosedEvent>, PriorityOrdered {

        private Connector connector;
        private int shutdownWaitSeconds;

        public void setShutdownWaitSeconds(int shutdownWaitSeconds) {
            this.shutdownWaitSeconds = shutdownWaitSeconds;
        }

        @Override
        public void customize(Connector connector) {
            this.connector = connector;
        }

        @Override
        public void onApplicationEvent(ContextClosedEvent contextClosedEvent) {
            this.connector.pause();
            Executor executor = this.connector.getProtocolHandler().getExecutor();
            if (executor instanceof ThreadPoolExecutor) {
                log.info("start to shutdown tomcat server,executor:{}", executor);
                try {
                    ThreadPoolExecutor threadPoolExecutor = (ThreadPoolExecutor) executor;
                    threadPoolExecutor.shutdown();
                    if (!threadPoolExecutor.awaitTermination(shutdownWaitSeconds, TimeUnit.SECONDS)) {
                        log.warn(
                                "tomcat thread pool did not shutdown gracefully within {} seconds. Proceeding with forceful shutdown",
                                shutdownWaitSeconds);
                    }
                } catch (Exception e) {
                    log.error("stop tomcat graceful exception", e);
                    Thread.currentThread().interrupt();
                }
                log.info("tomcat server stopped");
            }
        }

        @Override
        public int getOrder() {
            return Ordered.HIGHEST_PRECEDENCE;
        }
    }
}