Not sure where to go from here? (scraping)-CodePudding

url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p2380057.m570.l1313&_nkw=electronics"

response = requests.get(url)

soup = BeautifulSoup(response.text, "html.parser")

name = soup.find_all("div", class_="s-item__title")

price = soup.find_all("span", class_="s-item__price")

shipping = soup.find_all("span", class_="s-item__shipping s-item__logisticsCost")

I'm not sure where to go from here. The aim is to scrape: Name Price Shipping

And print it out like that. I'm just not quite sure how to start printing it (parsed)

CodePudding user response：

if you just want to print it out, try this

for n, p, s in zip(name, price, shipping):
    print(f"name: {n.span.contents} | price: {p.contents} | shipping: {s.contents}")

but the price and shpping info may not match the name.

CodePudding user response：

I would recommend to avoid all these lists if you are not controlling its length, store your data in a more structured way. Simply print it or transform it into dataframe for example:

for e in soup.select('li.s-item'):
    data.append({
        'name': e.find("div", class_="s-item__title").text,
        'price': e.find("span", class_="s-item__price").text,
        'shipping': e.find("span", class_="s-item__shipping s-item__logisticsCost").text if e.find("span", class_="s-item__shipping s-item__logisticsCost") else None
    })

Example

import requests
from bs4 import BeautifulSoup

url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p2380057.m570.l1313&_nkw=electronics"

response = requests.get(url)

soup = BeautifulSoup(response.text, "html.parser")

data = []

for e in soup.select('li.s-item'):
    data.append({
        'name': e.find("div", class_="s-item__title").text,
        'price': e.find("span", class_="s-item__price").text,
        'shipping': e.find("span", class_="s-item__shipping s-item__logisticsCost").text if e.find("span", class_="s-item__shipping s-item__logisticsCost") else None
    })
print(data)

Output

[{'name': 'Shop on eBay', 'price': '$20.00', 'shipping': None},
 {'name': 'Sony PSP 3000 & Charger Choose Color Fully Working REGION FREE NEW BATTERY',
  'price': '$29.99 to $119.99',
  'shipping': ' $25.45 shipping'},
 {'name': 'Nintendo Wii White Console - 2 Sets AUTHENTIC controllers- Gamecube - GUARANTEED',
  'price': '$139.99',
  'shipping': ' $16.77 shipping estimate'},
 {'name': 'Nintendo Ds Lite & OEM Charger Choose your Color Fully Working REGION FREE GBA',
  'price': '$12.99 to $99.99',
  'shipping': ' $25.45 shipping'},
 {'name': 'Sony Playstation 3 Console FAT Original Controller PS3',
  'price': '$34.99 to $54.99',
  'shipping': ' $19.99 shipping'},...]