1、requests使用了urllib3库,默认的http connection是keep-alive的,设置关闭
2、只用session进行操作。即只创建一个连接,并设置最大连接数或者重试次数。
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
s = requests.session()
retry = Retry(connect=3, backoff_factor=0.5)
adapter = HTTPAdapter(max_retries=retry)
s.mount('http://', adapter)
s.mount('https://', adapter)
s.keep_alive = False
resp = s.get(url,headers=self.headers,timeout=5)
亲测加入以上代码,Max retries exceeded with url问题已经解决
补充:
error1:
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x00000000038F2B00>: Failed to establish a new connection: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败。',))
解决办法:
session.keep_alive=False
error2:
python hostname doesn't match either of facebookXXXXX
解决办法:
import ssl ssl.match_hostname = lambda cert, hostname: True