Home > Back-end >  selenium clicking accept button using XPath not working
selenium clicking accept button using XPath not working

Time:09-04

I would like to scrape data from this page: enter image description here

Checking the XPath of the button: XPath = //*[@id="onetrust-accept-btn-handler"]

Replicating the steps performed here: enter image description here XPath is //*[@id="leftColumn"]/div[9]/a[1]

wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id="leftColumn"]/div[9]/a[1]")))

also returned invalid Syntax.


Updated Code

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

company = 'nvidia-corp'
driver = webdriver.Chrome(path)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")

wait = WebDriverWait(driver, 2)
accept_link= wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="onetrust-accept-btn-handler"]')))
accept_link.click()

scrollDown = "window.scrollBy(0,500);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll

driver.maximize_window()
time.sleep(10)

wait = WebDriverWait(driver, 2)

scrollDown = "window.scrollBy(0,4000);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll

try:
    close_popup_link= wait.until(EC.element_to_be_clickable((By.XPATH,'//*[@id="PromoteSignUpPopUp"]/div[2]/i')))
    close_popup_link.click()
except NoSuchElementException:
    print('No such element')
    
wait = WebDriverWait(driver, 3)
try:
    annual_link = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="leftColumn"]/div[9]/a[1]')))
    annual_link()
    # break
except NoSuchElementException:
    print('No element of that id present!')

The first accept button was successfully clicked, but clicking the Annual button returns Timeout Exception error.


Annual button enter image description here

CodePudding user response:

You need to take care of a couple of things here as follows:

  • If you are supplying the xpath within double qoutes, i.e. "..." then the attribute values needs to be within single quotes, i.e. '...'
  • Similarly, if you are supplying the xpath within single qoutes, i.e. '...' then the attribute values needs to be within double quotes, i.e. "..."

This take care of both the SyntaxError: invalid syntax

Effectively, the lines of code will be:

link= wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='onetrust-accept-btn-handler')))

and

wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='leftColumn']/div[9]/a[1]")))

Solution

To click on the clickable elements you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

  • Clicking on I Accept:

    • Using CSS_SELECTOR:

      WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
      
    • Using XPATH:

      WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='onetrust-accept-btn-handler']"))).click()
      
  • Clicking on Annual:

    • Using CSS_SELECTOR:

      WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[data-ptype='Annual']"))).click()
      
    • Using XPATH:

      WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@data-ptype='Annual']"))).click()
      
  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    

CodePudding user response:

At least for me I saw we need to use another locator to access that element.
I used scrolling until I can click that element.
The following code works for me:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("--start-maximized")

s = Service('C:\webdrivers\chromedriver.exe')

driver = webdriver.Chrome(options=options, service=s)
company = 'nvidia-corp'

wait = WebDriverWait(driver, 5)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")
try:
     wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="PromoteSignUpPopUp"]/div[2]/i'))).click()
except:
    pass
while True:
    try:
        wait.until(EC.element_to_be_clickable((By.XPATH, "//div[@class='float_lang_base_1']//a[@data-ptype='Annual']"))).click()
        break
    except:
        driver.execute_script("window.scrollBy(0, arguments[0]);", 1000)
  • Related