Unable to locate "Accept" Button - Selenium - Beginner Web Scraping-CodePudding

I am trying to use Selenium in order to learn different ways of web scraping.

When the code is executed Firefox starts and the "accept cookies" or what ever pops up. I am unable to locate the "accept" button when inspecting the page.

my code so far:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
import pandas as pd
import time

PATH = "C:/Users/myuser/Desktop/Driver/geckodriver.exe"

driver = webdriver.Firefox(executable_path=PATH)


driver.maximize_window() # For maximizing window
driver.get("https://www.immonet.de/")


button_pos = driver.find_element(by=By.CLASS_NAME, value="sc-gsDKAQ fILFKg")
button_pos.click()
print(driver.title)



input = input()

I get the following error: Unable to locate element: .sc-gsDKAQ fILFKg

My thought was locating the button via the inspect tool as follows:

What am I missing or doing wrong? How would i find the right element?

Thanks! Pat

CodePudding user response：

Class attribute in the html element can contain multiple classes separated by space. i.e. "sc-gsDKAQ fILFKg", contains two classes, sc-gsDKAQ and fILFKg.

You can user either but both are random and can be changed next time css is recompiled. I recommend to think of xpath using data-testid attribute

CodePudding user response：

First of all, to display this url,accepting the cookies is a must but to accept and click on the cookie button isn't a easy task because cookies button is under shadow root (open) selenium and webdriverWait can do nothing on shadow root,so to execute shadow root you need to apply JavaScript querySelector.

#To execute shadow root and accept cookies

driver.execute_script('''return document.querySelector('div#usercentrics-root').shadowRoot.querySelector('button[data-testid="uc-accept-all-button"]')''').click()