Price won't show up in the html parsing-CodePudding

I'm trying to get the price of this object into a variable and print it out/put it in a CSV.

This is the section of html that I am trying to parse:

<span  data-js-product-price="">
 <span>$429.00 USD</span>
</span>

This is my python code (sorry im a little new to python and I've been trying to figure this out for a while, so sorry if the code is a little messy)

from bs4 import BeautifulSoup


url_to_scrape = "https://www.backfireboards.com/?gclid=CjwKCAjwjtOTBhAvEiwASG4bCGHPgmV4XjyqAIFrW0Lr0IiW0AvfTiC7sZ4E-HtM_qJ9k4ahAu2CHxoCH5YQAvD_BwE"

request_page = urlopen(url_to_scrape)
page_html = request_page.read()
request_page.close()

html_soup = BeautifulSoup(page_html, 'html.parser')

board_prices = html_soup.find_all('span', class_='price')

print("num of prices: "   str(len(board_prices)))
file_name = 'product.csv'
f = open(file_name,'w')

headers = 'Title, Price \n'

f.write(headers)

i = 1

for price in board_prices:
    currPrice = price.span.text
    print(i)
    i = i   1
    print(price)
    print(currPrice)


f.close()

Here is the error that I am getting:

Traceback (most recent call last):
  File "/Users/isaiah/PycharmProjects/Web_scrape/main.py", line 26, in <module>
    currPrice = price.span.text
AttributeError: 'NoneType' object has no attribute 'text'

I know this isn't a text object but when I print it without the .text it spits out this:

1
<span  data-js-popup-cart-subtotal=""></span>
None
2
<span  data-js-product-price=""><span></span></span>
<span></span>
3
<span  data-js-product-price=""><span></span></span>
<span></span>
4
<span ><span></span></span>
<span></span>
5
<span  data-js-product-price=""><span></span></span>
<span></span>
6
<span  data-js-product-price=""><span></span></span>
<span></span>

I don't know why the $429 is disappearing in the object, im a little new to webscraping and such. Is there something very simple I am ignorant too?

Also, from what I can tell the website actually has 8 prices listed on this page that i am pulling from yet the board_prices object is only size 6? Could someone explain that to me as well?

CodePudding user response：

The data you see on the page is loaded from external source. To get titles/prices of electric skateboards you can use following example:

import requests
from bs4 import BeautifulSoup


url = "https://www.backfireboards.com/collections/electric-skateboards"
soup = BeautifulSoup(requests.get(url).content, "html.parser")

for h4 in soup.select(".product-collection__content h4"):
    title = h4.get_text(strip=True)
    price = h4.find_next(class_="price").contents[-1].text
    print("{:<15} {}".format(price, title))

Prints:

$429.00 USD     Backfire G2 Black with Super Power Hobbywing Motors and 96mm Wheels with 180 Days Warranty Especially Suitable for Beginners
$649.00 USD     Backfire G3 with Super Flexible Deck
$799.00 USD     Backfire Zealot S Belt Drive Electric Skateboard
$1,899.00 USD   Backfire Hammer Sledge
$1,199.00 USD   Backfire Hammer Belt Drive All Terrain Electric Skateboard
$649.00 USD     Backfire Zealot Belt Drive Electric Skateboard
$1,399.00 USD   Backfire Ranger X3 All Terrain Electric Skateboard with 1500W X2 Ultra High Power Ultra High Torque Motors and 12S High Voltage High Efficiency Electronic System
$999.00 USD     Backfire Ranger X2 All Terrain Electric Skateboard with 1200W X2 Ultra High Power Ultra High Torque Motors and 12S High Voltage High Efficiency Electronic System
$599.00 USD     Backfire Mini Super Portable Electric Skateboard Best for City Commute
$999.00 USD     Backfire G3 Plus with Carbon Fiber Deck and Ultra Long Range
$499.00 USD     Backfire ERA Electric Skateboard