Once reached last page, Python Selenium Script to continue instead of wait indefinitely-CodePudding

I am trying to click all next pages until last page which it does successfully on this website. However, it reaches the last page and then waits indefinitely. How can I best achieve this script to then proceed to the rest of the script once it reaches the last page? I could do an explicit wait time of 15 seconds timeout but this feels very slow and not the best way of doing this. Thanks Full code

i = 1
while i < 6:
    try:
        time.sleep(2)
        #time.sleep(random, 3)
        WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CSS_SELECTOR,  ".name:nth-child(1)")))
        WebDriverWait(driver, 100).until(lambda driver: driver.execute_script('return document.readyState') == 'complete')
        element = WebDriverWait(driver, 20).until(lambda driver: driver.find_element(By.CSS_SELECTOR, ".name:nth-child(1) , bf-coupon-table:nth-child(1) tr:nth-child(1) .matched-amount-value"))
        scroll = driver.find_element(By.CSS_SELECTOR, ".coupon-page-navigation__label--next")
        driver.execute_script("arguments[0].scrollIntoView();", scroll)
        link = driver.find_element_by_css_selector('[href^=http://somelink.com/]')
        WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "bf-coupon-page-navigation > ul > li:nth-child(4) > a")))
        NextStory = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'bf-coupon-page-navigation > ul > li:nth-child(4) > a')))
        link = driver.find_element_by_css_selector('bf-coupon-page-navigation > ul > li:nth-child(4) > a')
        NextStory.click()
    except:
        i = 6

CodePudding user response：

You are not getting to the except block, otherwise the loop would stop.

I think that this line is causing the slowness:

driver.execute_script("arguments[0].scrollIntoView();", scroll)

That is bacause all your other actions are driver actions with timeouts. Try clicking on the webpage and then just click page down key in a loop:

from selenium.webdriver.common.keys import Keys
# Click on a static element in your page
for _ in range(100):
    driver.send_keys(Keys.PAGE_DOWN)

CodePudding user response：

When i was working with playwright, I would make a variable that stores the HTML of the webpage initially, after click of button or u can say any function of page change or scroll, i would get the latest HTML content of page and compare it to the last stored HTML content, my while loop would do that and after comparing or in the beginning of the loop I would store the HTML content and after doing all the stuff in the end, page would change then again same comparision stuff and that's how it worked for me, the loop would stop when the last HTML content and the latest HTML content is same