Why can't I locate this element?-CodePudding

I am planning on web scraping GPU stock data from NVIDIA however when trying to load the full page by clicking on the load more button it says "selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".buy-link load-more-btn"}"

HTML The code

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import  expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException
import time

PATH = Service('C:\Program Files (x86)\chromedriver.exe')
driver = webdriver.Chrome(service=PATH)

driver.get("https://store.nvidia.com/en-gb/geforce/store/gpu/?page=1&limit=9&locale=en-gb&category=GPU&gpu=RTX 3090,RTX 3080 Ti,RTX 3080,RTX 3070 Ti,RTX 3070,RTX 3060 Ti,RTX 3060,RTX 3050 Ti,RTX 3050&gpu_filter=RTX 3090~30,RTX 3080 Ti~19,RTX 3080~24,RTX 3070 Ti~18,RTX 3070~22,RTX 3060 Ti~19,RTX 3060~28,RTX 3050 Ti~0,RTX 3050~0,RTX 2080 Ti~1,RTX 2080 SUPER~0,RTX 2080~0,RTX 2070 SUPER~0,RTX 2070~0,RTX 2060 SUPER~1,RTX 2060~8,GTX 1660 Ti~3,GTX 1660 SUPER~5,GTX 1660~5,GTX 1650 SUPER~2,GTX 1650~19")

load = driver.find_element(By.CLASS_NAME, "buy-link load-more-btn")

load.click()

CodePudding user response：

The call to find the element is executed before it is present on the page. Take a look at the wait timeout documentation. Here is an example from the docs

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
    element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "myDynamicElement"))
    )
finally:
    driver.quit()

you could also use implicitly_wait, but with this solution, you have to guess how long it will take to load

driver.implicitly_wait(20) # gives an implicit wait for 20 seconds