Selenium Cannot Locate Element Best Buy Prices-CodePudding

I'm currently learning Selenium and looking to practice on BestBuy. I am having trouble figuring out how to scrape the price either by XPATH or by CLASSNAME with Selenium.

What am I doing wrong? The error I keep getting is that Selenium cannot locate that element. Can someone point me in the direction of why Selenium can't locate the element?

Best Buy Link

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By

chrome_options = Options()
chrome_options.add_experimental_option("detach", True)

driver = webdriver.Chrome(r"Desktop\Selenium\chromedriver.exe")
website = 'https://www.bestbuy.com/site/apple-10-2-inch-ipad-latest-model-with-wi-fi-64gb-space-gray/4901809.p?skuId=4901809'
driver.get(website)

title = driver.find_element(By.CLASS_NAME,'sku-title')
price = driver.find_element(By.XPATH,'//*[@id="pricing-price-2768792"]/div/div/div/div/div/div[1]/div[1]/div/div/span[1]')
status = driver.find_element(By.CLASS_NAME,'fulfillment-add-to-cart-button')
status = status.text

print(price.text)

CodePudding user response：

Main issue is that suffix of attributes are dynamic and change from time to time or request to request.

Try to change your selection strategy and find something that is not that dynamic to get your goal. E.g. go with css selectors:

driver.find_element(By.CSS_SELECTOR,'[data-sticky-media-gallery] .priceView-hero-price span')

Note: Result also depence from country you are scraping from, so I had to click a link for the us, to get final product page

driver.find_element(By.CLASS_NAME,'us-link').click()

Example

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By

chrome_options = Options()
chrome_options.add_experimental_option("detach", True)

driver = webdriver.Chrome(r"Desktop\Selenium\chromedriver.exe")
website = 'https://www.bestbuy.com/site/apple-10-2-inch-ipad-latest-model-with-wi-fi-64gb-space-gray/4901809.p?skuId=4901809'
driver.get(website)

title = driver.find_element(By.CLASS_NAME,'sku-title')
price = driver.find_element(By.CSS_SELECTOR,'[data-sticky-media-gallery] .priceView-hero-price span')
status = driver.find_element(By.CLASS_NAME,'fulfillment-add-to-cart-button')
status = status.text

print(price.text)

Output

$329.99