I'm completely new to programming and I'm trying to write my first program, amazon scrapper. I wonder how can I get getText()
from find_all command. Here is my code:
import requests
from bs4 import BeautifulSoup
import lxml
url = "https://www.amazon.com/s?k=msi laptop&crid=2QIX2XTRWVVU7&sprefix=msi laptop,aps,281&ref=nb_sb_noss_1"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36",
"Accept-Language": "en",
}
r = requests.get(url, headers=headers)
soup = BeautifulSoup(r.text, "lxml")
names = soup.find_all("span", attrs={"class": 'a-size-medium a-color-base a-text-normal'})
names_lenght = len(names)
print(names_lenght)
for i in range(names_lenght):
print(names[i])
CodePudding user response:
Assuming your code is correct and is locating those elements, you can do (keeping it within your code):
names = [...]
for name in names:
print(name.get_text())
BeautifulSoup documentation: https://beautiful-soup-4.readthedocs.io/en/latest/