Getting href with Selenium and Python-CodePudding

I am trying to get the href with selenium and python.

This is my page:

Some class information are changing depending on which elements. So I am trying basically to get all href for <a id="job____ .....

links.append(job.find_element_by_xpath('//a[@aria-live="polite"]//span').get_attribute(name="href"))

I tried couple of things but can't figure out how. How can i get all my href from the screenshot above?

CodePudding user response：

Try this, but take care your xpath

"//a[@aria-live="polite"]//span"

will get a span, and i dont see any span with href on your html. Maybe this xpath solve it

//a[./span[@aria-live="polite"]]

links.append(job.find_element_by_xpath('//a[./span[@aria-live="polite"]]').get_attribute("href"))

But it wont get all urls, this with find_elements (return a list), extend your url list with list comprehension

links.extend([x.get_attribute("href") for x in job.find_elements_by_xpath('//a[./span[@aria-live="polite"]]')])

edit 1, other xpath solution

links.extend(["website_base_url" x.get_attribute("href") for x in job.find_elements_by_xpath('//a[contains(@id, "job_")]')])

CodePudding user response：

list_of_elements_with_href = wd.find_elements_by_xpath("//a[contains(@href,'')]")
for el_with_href in list_of_elements_with_href :
 links.append(el.with_href.get_attribute("href"))

or if you need more specify:

list_of_elements_with_href = wd.find_elements_by_xpath("//a[contains(@href,'') and contains(@id,'job_')]")

CodePudding user response：

Based on your description and attached image, I think you have got the wrong xpath. Try the following code.

find_links = driver.find_elements_by_xpath("//a[starts-with(@id,'job_')]")

links = []
for link in find_links:
        links.append(link.get_attribute("href"))

Please note elements in find_elements_by_xpath instead of element.

I am unable to test this solution as you have not provided the website.