I have a code to automatically click on the "Show more" button at the bottom of a page with Selenium e Firefox with proxy TOR, but I get an error:

     raise TimeoutException (message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

The code seems to be written well, I don't understand what the problem is. For greater clarity, I also share the connection with the proxy I use (everything ok, works fine) and then the code to click on the button automatically where I have the error. Can you help me please? Thanks

Code for connect Firefox with Proxy Tor

from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
import os
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait

#Connect Firefox with Proxy Tor
torexe_linux = os.popen('/home/xxxx/.local/share/torbrowser/tbb/x86_64/tor-browser_en-US') 

profile = FirefoxProfile('/home/xxxx/.local/share/torbrowser/tbb/x86_64/tor-browser_en-US/Browser/TorBrowser/Data/Browser/profile.default')
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
profile.set_preference("network.proxy.socks_remote_dns", False) #certi la tengono True
profile.update_preferences()

firefox_options = webdriver.FirefoxOptions()
firefox_options.binary_location = '/usr/bin/firefox' 

driver = webdriver.Firefox(
    firefox_profile=profile, options=firefox_options, 
    executable_path='/usr/bin/geckodriver')   

driver.get("https://www.diretta.it/calcio/svezia/allsvenskan/risultati/")
driver.maximize_window()

Code for automatic click (THE PROBLEM IS HERE)

from selenium.webdriver.common.action_chains import ActionChains

    driver.implicitly_wait(12)
    wait = WebDriverWait(driver, 12)
    actions = ActionChains(driver)
    
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
    
    while(driver.find_elements_by_css_selector('a.event__more.event__more--static')):
        show_more = driver.find_element_by_css_selector('a.event__more.event__more--static')
        driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
        actions.move_to_element(show_more).perform()
        time.sleep(0.5)
        show_more = driver.find_element_by_css_selector('a.event__more.event__more--static')
        show_more.click()
        time.sleep(3)

P.S: The code was set to click several times on the "Show more" button, because if you click on "Show more" the first time, then the page scrolls further down, but then I get another second "Show more" button. Sometimes even a third "Show more". So I would also like to click on the second and third "Show more" when they are loaded.

CodePudding user response：

Possibly the Show more button no more shows up as all the records are already being shown. In those cases, an ideal solution would be to:

1. Scroll the required height.

2. Move to the webelement. (this step isn't mandatory)

3. Click on Show More inducing WebDriverWait

4. Wrap up the code in a try-except{} block

5. Your optimum code block will be:

   WebDriverWait(driver, 12).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()        
   while True:
       try:
           driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
           ActionChains(driver).move_to_element(WebDriverWait(driver, 12).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "ba.event__more.event__more--static")))).perform()
           WebDriverWait(driver, 12).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.event__more.event__more--static"))).click()
           print("Show more button clicked")
           continue
       except TimeoutException:
           print("No more Show more buttons")
           break
   

CodePudding user response：

Try like below.

Use find_elements to store the Show more element in a list. Then compare the length of the list to 0, to determine if the Show more button is available to click.

driver.get("https://www.diretta.it/calcio/svezia/allsvenskan/risultati/")

wait = WebDriverWait(driver,30)

# Accept Cookies
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"button#onetrust-accept-btn-handler"))).click()

while len(driver.find_elements(By.CSS_SELECTOR,"a.event__more.event__more--static")) > 0:
    showmore = driver.find_element(By.CSS_SELECTOR, "a.event__more.event__more--static")
    driver.execute_script("arguments[0].scrollIntoView(true);", showmore)
    showmore.click()
    time.sleep(2)