How to interact with Cookie pop within #shadow-root (open) using Selenium-CodePudding

I would like to scrap immowelt.de, but I could not get pass the cookie banner. I tried to wait for the banner to load using both sleep() and WebDriverWait, however none of them is working.

This is the code with the webdriver

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

chrome_driver_path = '.../chromedriver'
url = 'https://www.immowelt.de/immobilienpreise'
driver = webdriver.Chrome(executable_path=chrome_driver_path)

driver.get(url)

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="uc-center-container"]/div[2]/div/div/div/div[1]/button'))).click()

driver.close()

And this is the code with sleep

from selenium import webdriver
import time
from selenium.webdriver.common.by import By

chrome_driver_path = '.../chromedriver'
url = 'https://www.immowelt.de/immobilienpreise'
driver = webdriver.Chrome(executable_path=chrome_driver_path)

driver.get(url)
time.sleep(5)
driver.find_element(By.XPATH, '//*[@id="uc-center-container"]/div[2]/div/div/div/div[1]/button').click()

driver.close()

CodePudding user response：

The element OK is within

Solution

To click on OK you have to use shadowRoot.querySelector() and you can use the following Locator Strategy:

Code Block:

driver.get("https://www.immowelt.de/immobilienpreise")
time.sleep(5)
element = driver.execute_script("""return document.querySelector('#usercentrics-root').shadowRoot.querySelector("button[data-testid='uc-accept-all-button']")""")
element.click()

References

You can find a couple of relevant discussions in: