I have this in my code:

link_tag = "//div[@class= 'yuRUbf']//a/@href"

With this code I get this error

The result of the xpath expression "//div[@class= 'yuRUbf']//a/@href" is: [object Attr]. It should be an element.

I don't know any other way to scrape the URL from that div class. How can I fix this? This code works fie on the scraper chrome extension but not on python.

CodePudding user response：

You have to change your XPath to select an element node (as the error message suggests) - and not an attribute node - and, after that, get its attribute. So use

link_tag = "//div[@class= 'yuRUbf']//a"
links = driver.find_elements_by_xpath(link_tag)

and then extract the attribute with

links[0].get_attribute("href")

to get the @href attribute of the first matching element.

CodePudding user response：

This solution might work.

div = driver.find_element(By.CSS_SELECTOR,"div[class='yuRUbf']")
url = div.get_attribute("href")