I know there are similar questions to this one that are answered which I already tried applying and didn't fix my problem.
My problem is that on this website: http://books.toscrape.com/catalogue/page-1.html there are 20 prices and when I try to scrape the prices, I only get the first price but not other 19.
Here's the code
from bs4 import BeautifulSoup
import requests
URL = 'http://books.toscrape.com/catalogue/page-1.html'
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
results = soup.find_all("div", class_ = "col-sm-8 col-md-9")
for i in results :
prices = i.find("p", class_ = "price_color")
print(prices.text.strip())
print()
CodePudding user response:
this code should work!
import requests
from bs4 import BeautifulSoup
URL = 'http://books.toscrape.com/catalogue/page-1.html'
page = requests.get(URL)
soup = BeautifulSoup(page.content, 'html.parser')
list_of_books = soup.select(
# using chrom selector
'#default > div > div > div > div > section > div:nth-child(2) > ol > li'
)
for book in list_of_books:
price = book.find('p', {'class': 'price_color'})
print(price.text.strip())
i just used chorme selector this is a screenshot of it
you are using the find
and find_all
in the wrong places.