Hello I wrote this selenium code to click Next button and give me url of the next page.

The Question is

1. I want to convert this code in a loop so I can click on the next button & collect all urls until next button disappears.

2. How do I out all the collected urls in a list?

next = driver.find_element(By.LINK_TEXT, "Next")
next.click()
urrl = driver.current_url
print(urrl)
driver.quit()

I tried While True loop for this.

while True:
    try:
        urrl = driver.current_url    **## I tried this line after clicking the next button as well**                       
        next = driver.find_element(By.LINK_TEXT,"Next")
        next.click()
    except:
        break

I was able to click on the next button until the end but I can not figure out how to collect url of the webpage and how to append them into a list.

Tried append but I think I am doing something wrong.

CodePudding user response：

You can write a function to test if the element exists:

def is_element_exists(xpath, id_flag=False):
    try:
        if id_flag:
            driver.find_element_by_id(xpath)
        else:
            driver.find_element_by_xpath(xpath)
        return True
    except Exception as e:
        # print("Excpetion:[%s][%s]" % (e, traceback.format_exc()))
        print('do not find the node')
    return False

CodePudding user response：

You can define a list object and append the collected URLs there as following.
The list should be defined before and out of the loop.

urls = []
while True:
    try:
        urrl = driver.current_url
        urls.append(urrl)
        next = driver.find_element(By.LINK_TEXT,"Next")
        next.click()
    except:
        break
print(urls)

The code above is generic. Probably you will need to scroll to the "next" button and wait for it to become clickable etc.