scrapy是目前准备要学习的爬虫框架,其在ubuntu14.04下的安装过程以下:python
ubuntu14.04下默认安装了2.7的python以及setuptools,若未安装,可经过下面指令安装:shell
sudo apt-get install python sudo apt-get install python-setuptools
而后安装Twisted:ubuntu
sudo apt-get install python-twisted
而后是Scrapy:api
sudo apt-get install python-scrapy
安装完成后,若是直接键入scrapy启动的话会报相似以下错误:框架
File "/usr/local/bin/scrapy", line 5, in <module> from pkg_resources import load_entry_point ...... pkg_resources.ContextualVersionConflict: (pyasn1 0.1.7 (/usr/lib/python2.7/dist-packages), Requirement.parse('pyasn1>=0.1.8'), set(['pyasn1-modules']))
按提示是个版本依赖的错误。scrapy
此时先安装pip:ide
sudo apt-get install python-pip
升级pip:学习
sudo pip install --upgrade pip
而后对提示的pyasn1升级:fetch
sudo pip install --upgrade pyasn1
再启动scrapy就能够成功了:ui
Scrapy 1.0.3 - no active project Usage: scrapy <command> [options] [args] Available commands: bench Run quick benchmark test commands fetch Fetch a URL using the Scrapy downloader runspider Run a self-contained spider (without creating a project) settings Get settings values shell Interactive scraping console startproject Create new project version Print Scrapy version view Open URL in browser, as seen by Scrapy [ more ] More commands available when run from project directory Use "scrapy <command> -h" to see more info about a command