爬取網(wǎng)址：http://www.meijutt.com/alltop_hit.html
爬取內容：排行榜名稱
爬取目標：存儲到本地文件中

涉及，瀏覽器偽裝，代理服務器，異常處理

代碼如下：

from bs4 import BeautifulSoup #調用庫 from urllib import request,error try:url = 'http://www.meijutt.com/alltop_hit.html'headers = ("User-Agent","Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393")proxy_ip = '119.28.194.66' #代理服務器ipproxy_ip_address = request.ProxyHandler({"http:": proxy_ip})openner = request.build_opener(proxy_ip_address, request.HTTPHandler) #使用代理ipopenner.addheaders = [headers] #瀏覽器偽裝request.install_opener(openner)data = openner.open(url).read().decode("gb2312") #獲取網(wǎng)頁源碼并解碼file = open(r"E:\crawl\meiju.txt", "a")i = 1soup = BeautifulSoup(data,"html.parser")nodes = soup.find_all("a",target="_blank")for node in nodes:file.write(str(i)+".")file.write(node.text.replace("\n","")+"\n") #將文件寫入指定的文件中i+=1 except error.URLError as e:print("再檢查下程序吧！")

總結

以上是生活随笔為你收集整理的python爬虫之美剧排行榜爬取(加浏览器伪装，代理，异常处理)的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內容還不錯，歡迎將生活随笔推薦給好友。