最近接手的是个python脚本,经过crontab按期运行,用facebook提供的api抓取数据,处理后再插入到数据库.css
许多人反映这个服务不稳定,抓取的数据不全.html
我在测试的过程当中发现,这个脚本开发的时间已通过了好久了,facebook提供的接口数据已经有所调整. 因此会有数据python
格式不对的问题;还有就是在跑了个小时后,会抛出 ‘MySQL server has gone away’ 的异常..mysql
网上找到的相似案例以下:git
http://stackoverflow.com/questions/8689649/solving-mysql-server-has-gone-away-errorsgithub
http://stackoverflow.com/questions/7942154/mysql-error-2006-mysql-server-has-gone-awaysql
我观察到操做数据库的类库是torndb,已通过去这么久,会不会这个bug在官方也有反应并修复了呢.数据库
果不其然..api
1:
2: def _ensure_connected(self):
3: # Mysql by default closes client connections that are idle for
4: # 8 hours, but the client library does not report this fact until
5: # you try to perform a query and it fails. Protect against this
6: # case by preemptively closing and reopening the connection
7: # if it has been idle for too long (7 hours by default).
8: if (self._db is None or
9: (time.time() - self._last_use_time > self.max_idle_time)):
10: self.reconnect()
11: self._last_use_time = time.time()