Python Selenium cannot click the close button-CodePudding

I am trying to download the data from https://projects.propublica.org/nonprofits for my research. When the page is open, a notification window pops up. I tried to use python selenium to close it. My code is as follows,

from selenium.webdriver import Chrome
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

driver = Chrome()

driver.get('https://projects.propublica.org/nonprofits')
driver.find_element(By.XPATH, "/html/body/div[1]/div/div[2]/p[2]/a").click()

I got the error message: selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div/div[2]/p[2]/a"} (Session info: chrome=99.0.4844.51)

I revised my code as

driver.get('https://projects.propublica.org/nonprofits')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div/div[2]/button"))).click()

The error message is TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Stacktrace: Backtrace: Ordinal0 [0x005E9943 2595139] ...

Any suggestion to overpass the notification windows is highly appreciated. Thank you.

CodePudding user response：

The element you are trying to click is inside an iframe - you have to switch to that iframe first in order to access this element
You have to add waits to let the elements be loaded before accessing them.
This will work better:

from selenium.webdriver import Chrome
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

driver = Chrome()

driver.get('https://projects.propublica.org/nonprofits')
wait = WebDriverWait(driver, 20)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe.syndicated-modal")))
wait.until(EC.visibility_of_element_located((By.XPATH, "//a[contains(@href,'newsletter-roadblock')]"))).click()

When finished working inside the iframe you will need to switch to the default content with

driver.switch_to.default_content()