win10下安装Pyspider

一、安装python

https://www.python.org/downloads/release/python-350/

二、安装对应python版本的pycurl

https://bintray.com/pycurl/pycurl/pycurl/view#files/

三、设置windows的环境变量path

必须设置两个，一个用于python自己，一个用于pip安装

注：此步骤可能须要重启电脑

四、 安装 pyspider

在windows终端模式下执行以下

pip install pyspider复制代码

五、 运行pyspider

直接键入命令

pyspider复制代码

执行[pip install pyspider]报错处理

Command "c:\users\xxx\appdata\local\programs\python\python35\python.exe -c "import setuptools, tokenize;__file__='C:\\Users\\xxx\\AppData\\Local\\Temp\\pip-build-t45zj444\\MarkupSafe\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\xxx\AppData\Local\Temp\pip-png7wz6e-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\xxx\AppData\Local\Temp\pip-build-t45zj444\MarkupSafe
You are using pip version 7.1.2, however version 19.0.3 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.复制代码

执行指令更新pip

python -m pip install --upgrade pip复制代码

执行[pyspider]报错处理

C:\Users\xxx>pyspider
c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\pyspider\libs\utils.py:196: FutureWarning: timeout is not supported on your platform.
  warnings.warn("timeout is not supported on your platform.", FutureWarning)
[W 190409 23:22:56 run:413] phantomjs not found, continue running without it.
[I 190409 23:22:59 result_worker:49] result_worker starting...
[I 190409 23:22:59 processor:211] processor starting...
[I 190409 23:22:59 scheduler:647] scheduler starting...
[I 190409 23:22:59 scheduler:586] in 5m: new:0,success:0,retry:0,failed:0
[I 190409 23:23:02 tornado_fetcher:638] fetcher starting...
[I 190409 23:23:02 scheduler:782] scheduler.xmlrpc listening on 127.0.0.1:23333
[I 190409 23:23:03 app:84] webui exiting...
[I 190409 23:23:03 tornado_fetcher:671] fetcher exiting...
[I 190409 23:23:03 scheduler:663] scheduler exiting...
[I 190409 23:23:04 result_worker:66] result_worker exiting...
[I 190409 23:23:04 processor:229] processor exiting...
Traceback (most recent call last):
  File "C:\Users\xxx\AppData\Local\Programs\Python\Python35\Scripts\pyspider-script.py", line 9, in <module>
    load_entry_point('pyspider==0.3.10', 'console_scripts', 'pyspider')()
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\pyspider\run.py", line 754, in main
    cli()
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\core.py", line 717, in main
    rv = self.invoke(ctx)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\core.py", line 1114, in invoke
    return Command.invoke(self, ctx)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\pyspider\run.py", line 165, in cli
    ctx.invoke(all)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\pyspider\run.py", line 497, in all
    ctx.invoke(webui, **webui_config)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\click\decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\pyspider\run.py", line 384, in webui
    app.run(host=host, port=port)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\pyspider\webui\app.py", line 59, in run
    from .webdav import dav_app
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\pyspider\webui\webdav.py", line 216, in <module>
    dav_app = WsgiDAVApp(config)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\wsgidav\wsgidav_app.py", line 135, in __init__
    _check_config(config)
  File "c:\users\xxx\appdata\local\programs\python\python35\lib\site-packages\wsgidav\wsgidav_app.py", line 119, in _check_config
    raise ValueError("Invalid configuration:\n - " + "\n - ".join(errors))
ValueError: Invalid configuration:
  - Deprecated option 'domaincontroller': use 'http_authenticator.domain_controller' instead.复制代码

wsgidav发布的3.x版本，相对于2.x(例如2.4.1)更改了一些用法

执行指令卸载并从新安装低版本 wsgidav

python -m pip uninstall wsgidav
python -m pip install wsgidav==2.4.1
复制代码