Home > Software engineering >  How to extract href attribute of an element in html with Selenium in Ptyhon
How to extract href attribute of an element in html with Selenium in Ptyhon

Time:04-09

I need a make a list of URLs of pictures(after that I will download it ) I do not know how to extract elements from class and from elements extract a URLs

driver.get("https://pixabay.com/en/photos/search/"   tag  "/?orientation=horizontal&size=medium")
images = [my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='item']/a")))] 
print("Images on Pixabay were found")

        
for x in images:
    images = driver.find_element(By.XPATH,"/html/body/div[1]/div[2]/div/div[1]/div/div[1]/div/a[1]")
    images = images.get_attribute("href")
    print(images)
    driver.get(images)
    #this is important because if I did not open a URL(Url after open is change to another.) it did not work
    sec_url = driver.current_url
    print( "Sec URL:  "  sec_url)

CodePudding user response:

images is a list of all the <href> attributes which you have collected using:

images = [my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='item']/a")))] 

Moving ahead as you are iterating through the list as:

for x in images:

you need to avoid using the same variable name to identify an element and to store the <href> attribute within the loop. So you need to modify the for loop as follows:

for x in images:
    element = driver.find_element(By.XPATH,"/html/body/div[1]/div[2]/div/div[1]/div/div[1]/div/a[1]")
    element_href = element.get_attribute("href")
    print(element_href)
    driver.get(element_href)
    sec_url = driver.current_url
    print( "Sec URL:  "  sec_url)
  • Related