I'm trying to fetch product info on different pages from a site. I wrote a function to loop through each product and collect the information. when it reaches some page the pattern changed, and the function could not fetch the info but ran into an error. how can I make the code return a nan value if the condition set is not met?
Here is my function:
def get_quality_rating_info(links):
Quality_rating = []
ratings = []
while len(Quality_rating) < len(links):
for link in links:
print("Progress: Size of data collected {}... {}/{}".format(str(len(Quality_rating)),str(len(Quality_rating)),str(len(links))))
product_url = requests.get(link)
new_soup = BeautifulSoup(product_url.text, "html.parser")
for i in range(4):
quality = new_soup.findAll("div", {"class":"value"})[i]
ratings.append(str(quality.text.strip()))
Quality_rating.append(ratings[0])
###Changes the list to null after appending ratings
rating = []
print("Successfully collected Quality rating data!")
return Quality_rating
one of the links before the page style changed: https://www.jdpower.com/detail/2020/dodge/charger/flagstaff-az/2c3cdxhg6lh245286
one of the links after it changed: https://www.jdpower.com/detail/2020/ram/1500/bakersfield-ca/1c6rrebg4ln267192
Here is the error info error info
CodePudding user response:
Wrap the entire block that might cause exception in try...except statement, and in except block return what you desire, e.g. None. https://docs.python.org/3/tutorial/errors.html