场景:一个上传接口,须要上传几十M的文件,文件中包含10几W的数据,而后对10+W的数据进行同步批量插入,每次批量插入1W。最后返回结果。java
项目上线一段时间后,上传接口出现问题,数据库用的MySQL5.7.21,报了以下错误:mysql
2018-07-16 01:30:03.497 ERROR com.alibaba.druid.pool.DruidDataSource Line:1594 - discard connection com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet successfully received from the server was 1,480 milliseconds ago. The last packet sent successfully to the server was 1,480 milliseconds ago. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3603) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3492) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4043) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155) at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1379) at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:498) at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:63) at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79) at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63) at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:324) at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156) at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109) at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:137) at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61) at com.sun.proxy.$Proxy63.query(Unknown Source) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141) at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:434) at com.sun.proxy.$Proxy25.selectList(Unknown Source) at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:231) at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:137) at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:75) at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53) at com.sun.proxy.$Proxy51.selectReportListByReportDate(Unknown Source) at com.rxwx.service.report.impl.PrcmtReportServiceImpl.generateProcurementReport(PrcmtReportServiceImpl.java:97) at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$8c585594.generateProcurementReport(<generated>) at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$9a830f32.generateProcurementReport(<generated>) at com.rxwx.task.sup.GenarateReportJob.selfGenReportTask1(GenarateReportJob.java:37) at com.rxwx.task.sup.GenarateReportJob$$FastClassBySpringCGLIB$$64b31449.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost. at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3052) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3503) ... 58 common frames omitted
首先百度了一发,按照网上的解决方案,修改了my.cnf中的超时时间:git
wait_timeout=31536000 interactive_timeout=31536000
将原来默认的8小时改成1年。这里单位是秒。github
修改了德鲁伊的:testOnBorrow = true。spring
跑了一天,MMP,凌晨又出现这个错误了。sql
一个电话把睡梦中的我吵醒,扰我周末扰我梦,没得办法,重启下服务,重启下MySQL,先上传吧,睡觉。数据库
周一到公司谷歌了一发,修改my.cnf的net_read_timeout和net_write_timeout 参数,将这两个参数调大,改成6000。因为项目中的数据存在无效数据,因此删除了10天以前的无效数据,减小了表中的数量。明天看效果。apache
show global variables like "%timeout%";服务器
connect_timeout网络
链接超时 mysql链接共有6次握手,3次TCP协议这个跟connect_timeout参数没有关系,另外3次跟connect_timeout参数有关系,该参数主要是为了防止网络不佳时应用重连致使链接数涨太快,通常默认便可。
delayed_insert_timeout
这是为MyISAM INSERT DELAY设计的超时参数,在INSERT DELAY停止前等待INSERT语句的时间
interactive_timeout
服务器关闭交互式链接前等待活动的秒数。交互式客户端定义为在mysql_real_connect()中使用CLIENT_INTERACTIVE选项的客户端。参数默认值:28800秒(8小时)
lock_wait_timeout
锁等待超时时间
net_read_timeout / net_write_timeout
这个参数只对TCP/IP连接有效,分别是数据库等待接收客户端发送网络包和发送网络包给客户端的超时时间,这是在Activity状态下的线程才有效的参数
slave_net_timeout
这是Slave判断主机是否挂掉的超时设置,在设定时间内依然没有获取到Master的回应就人为Master挂掉了
wait_timeout
服务器关闭非交互链接以前等待活动的秒数。
<font color = "red">效果很明显。今天没有出现问题。