centos6u3 安装 celery 总结

耗时大概6小时.

执行 pip install celery 以后, 在 mac 上 celery 能够正常运行, 在 centos 6u3 上报错以下:

Traceback (most recent call last):
  File "/home/work/collie/venv/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/__main__.py", line 13, in main
    from celery.bin.celery import main as _main
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/bin/__init__.py", line 2, in <module>
    from .base import Option
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/bin/base.py", line 17, in <module>
    from celery import VERSION_BANNER, Celery, maybe_patch_concurrency
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/local.py", line 509, in __getattr__
    module = __import__(self._object_origins[name], None, None, [name])
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/app/__init__.py", line 5, in <module>
    from celery import _state
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/_state.py", line 15, in <module>
    from celery.utils.threads import LocalStack
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/utils/__init__.py", line 10, in <module>
    from .nodenames import worker_direct, nodename, nodesplit
  File "/home/work/collie/venv/lib/python2.7/site-packages/celery/utils/nodenames.py", line 7, in <module>
    from kombu.entity import Exchange, Queue
  File "/home/work/collie/venv/lib/python2.7/site-packages/kombu/entity.py", line 6, in <module>
    from .abstract import MaybeChannelBound, Object
  File "/home/work/collie/venv/lib/python2.7/site-packages/kombu/abstract.py", line 6, in <module>
    from .connection import maybe_channel
  File "/home/work/collie/venv/lib/python2.7/site-packages/kombu/connection.py", line 15, in <module>
    from kombu import exceptions
  File "/home/work/collie/venv/lib/python2.7/site-packages/kombu/exceptions.py", line 6, in <module>
    from amqp import ChannelError, ConnectionError, ResourceError
  File "/home/work/collie/venv/lib/python2.7/site-packages/amqp/__init__.py", line 47, in <module>
    from .connection import Connection  # noqa
  File "/home/work/collie/venv/lib/python2.7/site-packages/amqp/connection.py", line 29, in <module>
    from . import spec
ImportError: cannot import name spec

遇到这个问题时, 心态很差, 没有深刻的分析问题, 而是寻求在生产环境服务器上复现与 mac 同样的环境, 以求问题能够本身解决.

最开始觉得是 rabbitMQ 没有安装致使, 转而寻求安装 rabbitMQ, 生产环境 10KB 的下载速度, 浪费了不少时间.

而后怀疑是 virtualenv 的问题, 而后在另外一台服务器上采用绝对路径安装了 python 2.7.12, pip 以后发现问题依旧.

进一步怀疑是 pip 下载的文件有损, 开始手动下载相关包文件安装, 问题依旧.

最后回到报错的信息, 模块加载失败, 进入到 amqp 包文件目录, 解释器下加载 spec, 没有报警.

执行 from .connection import Connection, 报错信息以下:

>>> from connection import Connection
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "connection.py", line 26, in <module>
    from vine import ensure_promise
  File "/usr/local/lib/python2.7/site-packages/vine/__init__.py", line 8, in <module>
    from .abstract import Thenable
  File "/usr/local/lib/python2.7/site-packages/vine/abstract.py", line 8, in <module>
    from .five import with_metaclass
  File "/usr/local/lib/python2.7/site-packages/vine/five.py", line 67, in <module>
    import platform
  File "platform.py", line 16, in <module>
    int, platform.release().split('-')[0].split('.')))
AttributeError: 'module' object has no attribute 'release'

报错 platform 没有 release 方法, 这个是不对的, platform 是有这个方法的.

发现缘由, amqp 2.1.2 版本以后, 增长了 platform.py 的同名文件

# platform.py
"""Platform compatibility."""
from __future__ import absolute_import, unicode_literals

import sys
import platform

# Jython does not have this attribute
try:
    from socket import SOL_TCP
except ImportError:  # pragma: no cover

看起来, 当 vine 包中 five.py 执行到 platform.release 时, 解释器混淆了两个 platform, 致使报错.

不过问题到这里并不能结束, 虽然问题能够在 centos 6, 非 /usr/bin/ 路径安装的 Python 2.7.11 下复现, 为何 mac 下就没有这个问题.

mac 下的默认 Python 也不是系统自带版本, brew install 与手动安装原理没有区别, 考虑到 amqp 2.1.2 发布时间已经一段时间, 而且在 celery 的 github 上也没有 issue 报出, 还须要想一下, 为何恰恰在这些机器上出现了这个问题.

顺便, 在寻找 amqp 的历史版本时, 发现无从下手, pypi 页面只提供最新版, 最后在手动安装的过程当中, 发现原来 pypi 上是有存储的.

地址 https://pypi.python.org/simple/amqp/

最后, 直面问题的态度, virtualenv 的原理是否彻底理解了, import 相对路径绝对路径是否清楚, 异常信息是否定真看过, 都是些很基础的事情, 这个须要增强. 项目上线以后须要再总结一下.