Home > other >  Python, crawl web data did not show an error, but could not crawl the site content is the cause of w
Python, crawl web data did not show an error, but could not crawl the site content is the cause of w

Time:12-04

The import requests
The import urllib. Request
The from bs4 import BeautifulSoup

The header={
"The user-agent: Mozilla/5.0 (Windows NT 10.0; Win64. X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36 "
}
Text=requests. Get (" https://www.endata.com.cn/BoxOffice/", headers=headers). The text
Main_page=BeautifulSoup (text, "HTML parser")

Table=main_page. Find (" div ", attrs={" id ":" TopList "})
F=open (" movie box office. CSV, "mode='a')
TRS=table. Find_all (' tr)
For the tr in TRS:
LST=tr. Find_all (" td ")
If len (LST)!=0:
For td in LST:
F.w rite (td) text) strip ())
F.w rite () ", "
F.w rite (" \ n ")