Unable to click and "Accept Cookies" pop up in python Selenium-CodePudding

I'm trying to scrape some information on prices from a website and get find the different prices for different states and cities. However, an "Accept Cookies" pop up shows up, and I need to click it for my code to be successful, but I am struggling to get Selenium to locate anything to click at all

here is the website www.kettlebellkings.com

here is the html for the button:

<button role="button" data-testid="uc-accept-all-button"  style="margin: 0px 6px;">Accept All</button>

I've been using Expected conditions but it just times out every time. Here is my code:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

path = "C:\Program Files (x86)\msedgedriver.exe"
driver = webdriver.Edge('www.kettlebellkings.com')
driver.get(url)

element_present = EC.element_to_be_clickable((By.CSS_SELECTOR, 'button[data-testid="uc-accept-all-button"]'))
WebDriverWait(driver, 30).until(element_present)
try:
    driver.find_element(By.CSS_SELECTOR, 'button[data-testid="uc-accept-all-button"]').click()
except:
    clicker = driver.find_element(By.CSS_SELECTOR, 'button[data-testid="uc-accept-all-button"]')
    driver.execute_script("arguments[0].click();", clicker)

But it just times out. If I leave out the expected conditions:

path = "C:\Program Files (x86)\msedgedriver.exe"
driver = webdriver.Edge('www.kettlebellkings.com')
driver.get(url)

driver.find_element(By.CSS_SELECTOR, 'button[data-testid="uc-accept-all-button"]').click()

Returns:

Message: no such element: Unable to locate element: {"method":"css selector","selector":"button[data-testid="uc-accept-all-button"]"}

It doesn't appear to be in an iframe either - what am I missing here?

CodePudding user response：

'Accept All' cookies button is inside the Shadow Dom, try the below code:

shadow_root = driver.find_element(By.CSS_SELECTOR, "#usercentrics-root").shadow_root
shadow_root.find_element(By.CSS_SELECTOR, "button[data-testid='uc-accept-all-button']").click()

You can refer how to handle shadow dom elements from the below links:

https://www.lambdatest.com/blog/shadow-dom-in-selenium/

https://www.seleniumeasy.com/selenium-tutorials/accessing-shadow-dom-elements-with-webdriver

CodePudding user response：

you could try setting a cookie that would stop the pop up just accept cookies on your normal browser, set them in the program and the website will recognize it as you/your account

driver.add_cookie({"name": "key", "value": "value"})