I'm trying to get data from the following page: https://www.robeco.com/ch/en/funds/prof-ch-en-11/robeco-qi-emerging-conservative-equities-f-eur-lu0940007007.html

I use the following script:

url="https://www.robeco.com/ch/en/funds/prof-ch-en-11/robeco-qi-emerging-conservative-equities-f-eur-lu0940007007.html"
driver.get(url)
time.sleep(5)

xpath1='/html/body/div[3]/div/div/div[2]/button'
driver.find_element(By.XPATH, xpath1).click()

time.sleep(2)

xpath2='/html/body/div[4]/div/div/div/div/ul/li'
driver.find_element(By.XPATH, xpath2).click()

The first click accepts the general conditions disclaimer. The second click accepts the cookie disclaimer. I have issues with the second click, as I sometimes get the following error. The behaviour looks random: sometimes the script works, other times the error occurs.

ElementClickInterceptedException: Message: element click intercepted: Element <li  id="sdgdpr_modal_buttons-agree">...</li> is not clickable at point (790, 521). Other element would receive the click: <p>...</p>
  (Session info: chrome)
Stacktrace:
Backtrace:
    Ordinal0 [0x00F1DF13 2219795]
    Ordinal0 [0x00EB2841 1779777]
    Ordinal0 [0x00DC423D 803389]
    Ordinal0 [0x00DF99D4 1022420]
    Ordinal0 [0x00DF78C4 1013956]
    Ordinal0 [0x00DF54AB 1004715]
    Ordinal0 [0x00DF4117 999703]
    Ordinal0 [0x00DE9B76 957302]
    Ordinal0 [0x00E0E7FC 1107964]
    Ordinal0 [0x00DE94B4 955572]
    Ordinal0 [0x00E0EA14 1108500]
    Ordinal0 [0x00E1F192 1175954]
    Ordinal0 [0x00E0E616 1107478]
    Ordinal0 [0x00DE7F89 950153]
    Ordinal0 [0x00DE8F56 954198]
    GetHandleVerifier [0x01212CB2 3040210]
    GetHandleVerifier [0x01202BB4 2974420]
    GetHandleVerifier [0x00FB6A0A 565546]
    GetHandleVerifier [0x00FB5680 560544]
    Ordinal0 [0x00EB9A5C 1808988]
    Ordinal0 [0x00EBE3A8 1827752]
    Ordinal0 [0x00EBE495 1827989]
    Ordinal0 [0x00EC80A4 1867940]
    BaseThreadInitThunk [0x75EFFA29 25]
    RtlGetAppContainerNamedObjectPath [0x77C17B5E 286]
    RtlGetAppContainerNamedObjectPath [0x77C17B2E 238]

What could I do? Thanks

CodePudding user response：

There are several issues with your code:

1. You need to learn how to create correct locators. Absolute XPath are extremely fragile and not reliable.
2. You should use WebDriverWait expected_conditions, not hardcoded pauses.
3. On the second cookies banner you should click the parent ul element, not a child li element.

The following code works:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 10)

url = "https://www.robeco.com/ch/en/funds/prof-ch-en-11/robeco-qi-emerging-conservative-equities-f-eur-lu0940007007.html"

driver.get(url)
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CLASS_NAME, "btn-turquoise"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, 'sdgdpr_modal_buttons'))).click()