scrapy框架异常之no more duplicates will be shown (see DUPEFILTER_DEBUG to show all duplicates)

今天在用scrapy爬虫时,报了下面的错误:python 2019-01-17 16:47:18 [scrapy.dupefilters] DEBUG: Filtered duplicate request: <GET https://newhouse.fang.com/house/s/b95/> - no more duplicates will be shown (see DUPEFILTER
相关文章
相关标签/搜索