selenium clicking accept button using XPath not working-CodePudding

I would like to scrape data from this page:

Checking the XPath of the button: XPath = //*[@id="onetrust-accept-btn-handler"]

Replicating the steps performed here: XPath is //*[@id="leftColumn"]/div[9]/a[1]

wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id="leftColumn"]/div[9]/a[1]")))

also returned invalid Syntax.

Updated Code

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

company = 'nvidia-corp'
driver = webdriver.Chrome(path)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")

wait = WebDriverWait(driver, 2)
accept_link= wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="onetrust-accept-btn-handler"]')))
accept_link.click()

scrollDown = "window.scrollBy(0,500);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll

driver.maximize_window()
time.sleep(10)

wait = WebDriverWait(driver, 2)

scrollDown = "window.scrollBy(0,4000);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll

try:
    close_popup_link= wait.until(EC.element_to_be_clickable((By.XPATH,'//*[@id="PromoteSignUpPopUp"]/div[2]/i')))
    close_popup_link.click()
except NoSuchElementException:
    print('No such element')
    
wait = WebDriverWait(driver, 3)
try:
    annual_link = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="leftColumn"]/div[9]/a[1]')))
    annual_link()
    # break
except NoSuchElementException:
    print('No element of that id present!')

The first accept button was successfully clicked, but clicking the Annual button returns Timeout Exception error.

Annual button

CodePudding user response：

You need to take care of a couple of things here as follows:

If you are supplying the xpath within double qoutes, i.e. "..." then the attribute values needs to be within single quotes, i.e. '...'
Similarly, if you are supplying the xpath within single qoutes, i.e. '...' then the attribute values needs to be within double quotes, i.e. "..."

This take care of both the SyntaxError: invalid syntax

Effectively, the lines of code will be:

link= wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='onetrust-accept-btn-handler')))

and

wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='leftColumn']/div[9]/a[1]")))

Solution

To click on the clickable elements you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

Clicking on I Accept:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()

Using XPATH:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='onetrust-accept-btn-handler']"))).click()

Clicking on Annual:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[data-ptype='Annual']"))).click()

Using XPATH:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@data-ptype='Annual']"))).click()

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

CodePudding user response：

At least for me I saw we need to use another locator to access that element.
I used scrolling until I can click that element.
The following code works for me:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("--start-maximized")

s = Service('C:\webdrivers\chromedriver.exe')

driver = webdriver.Chrome(options=options, service=s)
company = 'nvidia-corp'

wait = WebDriverWait(driver, 5)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")
try:
     wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="PromoteSignUpPopUp"]/div[2]/i'))).click()
except:
    pass
while True:
    try:
        wait.until(EC.element_to_be_clickable((By.XPATH, "//div[@class='float_lang_base_1']//a[@data-ptype='Annual']"))).click()
        break
    except:
        driver.execute_script("window.scrollBy(0, arguments[0]);", 1000)