webmagic
最新版是0.7.3版本,在爬取只支持
TLS1.2
的
https
站点会报错,修改HttpClientGenerator中的buildSSLConnectionSocketFactory方法,
return new SSLConnectionSocketFactory(createIgnoreVerifySSL(), new String[]{"SSLv3", "TLSv1", "TLSv1.1", "TLSv1.2"},
null,
new DefaultHostnameVerifier())