Home > other >  Urgent urgent, crawler bosses to help solve the problem of a crawler result is empty
Urgent urgent, crawler bosses to help solve the problem of a crawler result is empty

Time:04-12

 import bs4 
The import requests
The import re
The import XLWT
Import a datetime

Date=datetime. Datetime. Now (). The strftime (' % Y - % m - % d ') # call file timestamp, facilitate data update
Url='https://www.aliexpress.com/wholesale' # url
Content={' SearchText ':' Nike ', 'page' : '1', 'ie' : 'utf8', 'g', 'y'} # dictionary passed the url parameter

# initialization data containers
Title=[]

Content (' page ')=2 # for page Numbers here, according to the specific Settings page parameter
Resp=requests. Get (url, params=payload)

Print (resp. Url) # print access url
Resp. Encoding="utf-8" # set encoding
Soup=bs4. BeautifulSoup (resp. The text, ". The HTML parser ")
# print (resp. Text)
Print (soup. Prettify ())
# title
All_title=soup. Find_all (' a ', running the class_=re.com (" the item - the title "))
Print (all_title)
Print (" title ", all_title)
For j in all_title:
Soup_title=bs4. BeautifulSoup (STR (j), ". The HTML parser ")
Print (soup_title)
The title. Append (soup_title. A [' title '])

Separate the resp. The text as an HTML file to BeautifulSoup can climb to the content, but the use of the form of web pages to climb to the content of the empty [], namely to ask who is this why
  • Related