I am teaching myself watching youtube videos. I am working through example project and I am able to mirror what is shown on youtube. However when I go to next stage and include a loop like done in the video I get this msg - AttributeError: 'NoneType' object has no attribute 'text' If I comment out and only do one value(urL) it returns the appropriate value:
The sample code is as follows
import requests
from bs4 import BeautifulSoup
baseurl = 'https://www.thewhiskeyexchange.com/'
headers = {
'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36'}
productlinks = []
for x in range(1,3):
r = requests.get(f'https://www.thewhiskyexchange.com/c/35/japanese-whisky?pg={x}')
soup = BeautifulSoup(r.content, 'lxml')
productlist = soup.find_all('div', class_='product-grid')
for item in productlist:
for link in item.find_all('a', href=True):
productlinks.append(baseurl link['href'])
#testlink = 'https://www.thewhiskyexchange.com/p/25880/hanyu-1985-the-joker-monochrome-label'
print(productlinks)
for link in productlinks:
r = requests.get(link, headers=headers)
soup = BeautifulSoup(r.content, 'lxml')
name = soup.find('h1',class_='product-main__name').text.strip()
price = soup.find('p', class_='product-action__price').text.strip()
try:
rating = soup.find('div',class_='review-overview').text.strip()
except:
rating = 'no rating'
whisky = {
'name': name,
'price': price,
'rating': rating
}
print(whisky)
Not sure what I am doing wrong am I getting some sort of denial. Hope someone can help so I can continue on my learning journey.
CodePudding user response:
Your problem is this line:
name = soup.find('h1',class_='product-main__name').text.strip()
There is no h1 tag named 'product-main__name' and the function is returning none.
It appears the problem is that you mispelled the website in this line:
baseurl = 'https://www.thewhiskeyexchange.com/'
This is causing the wrong website to be requested and therefore not having the desired tag.