Webdriverwait doesn't click the tab even with id selector-CodePudding

I am revert to this case and the next obstacle i found with the borrowed code. Surely i am new to this and please judge me with indulgence.I tried all available selectors and many syntax strategies but i am unable to convince Selenium for clicking the tab button. My first question was answered succesfully from a kind person and I hope in a similar generosity. The line with the controversy is:

elem = WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//ul[@class='fig-tabs__list']//[@id='tab_statistics'")))
elem.click()

First part of code:

from selenium import webdriver
from selenium.webdriver.opera.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from time import sleep
from datetime import datetime
import pandas as pd

errors = []
season = []
for id in range(2124889, 2124891):
    # Opening the connection and grabbing the page
    my_url = f'https://www.lefigaro.fr/sports/football/live/bundesliga/2020/{id}/*'
    option = Options()
    option.headless = False
    driver = webdriver.Opera(options=option)
    driver.get(my_url)
    driver.maximize_window()
    WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((
        By.XPATH,"//div[@id='appconsent']//iframe")))
    WebDriverWait(driver,10).until(EC.visibility_of_element_located((
        By.CSS_SELECTOR,'button.button__acceptAll'))).click()
    sleep(5)

    # Scraping the data
    try:

        date = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((
            By.XPATH, '//*[@id="section_match_sheet"]/div[1]/div[1]/div[3]/span[2]'))).text
        date = datetime.strptime(date, '%a %d %b %Y').strftime('%m/%d/%Y')

        home_team = driver.find_element_by_xpath(
            '//*[@id="fig-page"]/div[1]/div[4]/div/div[2]/div[1]/a/span').text
        away_team = driver.find_element_by_xpath(
            '//*[@id="fig-page"]/div[1]/div[4]/div/div[2]/div[3]/a/span').text

        scores = driver.find_element_by_xpath(
            '//*[@id="fig-page"]/div[1]/div[4]/div/div[2]/div[2]/div').text
        home_score = scores.split('-')[0]
        away_score = scores.split('-')[1]
        
        elem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((
            By.XPATH, "//ul[@class='fig-tabs__list']//[@id='tab_statistics'")))
        elem.click()

        dfs = pd.read_html(driver.page_source)
        stats = dfs[-1]

        driver.quit()

    except:
        driver.quit()
        errors.append(id)

CodePudding user response：

You've a syntax error here at

"//ul[@class='fig-tabs__list']//[@id='tab_statistics'"

You are missing ]

try this :

elem = WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//ul[@class='fig-tabs__list']//[@id='tab_statistics']")))
elem.click()

CodePudding user response：

I am suspecting that problem appear in websites with popup "accept cookies" banners that blocking the content initially. Although with the apropriate code you can click on accept and ged rid of the element, it seems that focusing is not straight at the content of the page.