JVM调优实战

对项目 i..net 网站进行了接口的压测，压测过程当中发现各微服务的 heap 内存设置太小（-Xms64m –Xmx128m），致使压测过程当中服务假死（频繁fullgc致使服务假死）和发生OOM。

一、xx接口的压测状况以下，150并发进行压测，发现响应时间在小于1s一段时间后，响应时间陡升到3s。

二、 TPS 也是从420 陡降到 60 左右。

三、trace 调用链跟踪发现 xxxxsevice中的 xxxxWorktime方法调用耗时3S

四、继续跟踪该方法的调用链，发现调用 attend-xxxx项目中的接口耗时3S

五、查看后台日志发现：atten-xxxx 服务发生了 OOM；经过telnet命令链接xxxx的端口，发现没法链接端口，服务处于假死状态。

六、查看该服务的heap使用状况，基本上被耗尽

七、jstat 发现 发生了 频繁的 FullGC

八、压测过程当中发如今进行资源争夺时，有时还把mongo 给拖死：

ERROR c.y.a.sign.exception.BaseControllerException - attend-xxxx统一异常捕获...
org.springframework.dao.DataAcce***esourceFailureException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is
Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=REPLICA_SET, servers=[{address=xx.xx.0.40:47020, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}, {address=xx.xx.0.40:47020, type=REPLICA_SET_ARBITER, roundTripTime=0.6 ms, state=CONNECTED}, {address=xx.xx.0.40:47020, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]
at com.mongodb.connection.BaseCluster.createTimeoutException(BaseCluster.java:377)
at com.mongodb.connection.BaseCluster.selectServer(BaseCluster.java:104)
at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:75)
at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:71)
at com.mongodb.binding.ClusterBinding.getWriteConnectionSource(ClusterBinding.java:68)
at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:411)
at com.mongodb.operation.BaseWriteOperation.execute(BaseWriteOperation.java:133)
at com.mongodb.operation.BaseWriteOperation.execute(BaseWriteOperation.java:60)
at com.mongodb.Mongo.execute(Mongo.java:845)
at com.mongodb.Mongo$2.execute(Mongo.java:828)
at com.mongodb.DBCollection.executeWriteOperation(DBCollection.java:342)
at com.mongodb.DBCollection.insert(DBCollection.java:337)
at com.mongodb.DBCollection.insert(DBCollection.java:328)
at com.mongodb.DBCollection.insert(DBCollection.java:298)
at com.mongodb.DBCollection.insert(DBCollection.java:264)
at com.mongodb.DBCollection.insert(DBCollection.java:186)
at com.mongodb.DBCollection.save(DBCollection.java:407)
at com.mongodb.DBCollection.save(DBCollection.java:382)

九、数据库在压测过程当中被压死：

综上所述，调优建议以下：

一、 微服务的heap 设置过小了，须要扩容内存，建议设置为（-Xms512m –Xmx1024m）

二、 数据库服务服务器建议单独部署物理机，且数据库内存建议设置64G以上；建议采用多主从或分片部署。