最近我在爬拉钩的
https://www.lagou.com/gongsi/0-2-0-0这个页面点进去进入公司的页面 爬虫地址信息 有时候报验证码,求指教
headers = {
"User-Agent": "Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.2; U; de-DE) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/234.40.1 Safari/534.6 TouchPad/1.0",
"X-Requested-With": "XMLHttpRequest", "Referer": "
https://www.lagou.com/gongsi/0-1-0-0"
}
cookies_str = "user_trace_token=20180909010719-4eb82332-59f2-4979-b7ba-4a96de35eb40; _ga=GA1.2.1153938840.1536426437; LGUID=20180909010720-a5755fe0-b389-11e8-8ccd-525400f775ce; index_location_city=%E5%8C%97%E4%........“
res = requests.get(companyLink, headers=header, cookies=get_cookies(cookies_str )
偶尔会弹出这个链接
https://passport.lagou.com/login/login.html?msg=validation&uStatus=2&clientIp=47.105.103.225