Home > Blockchain >  How do I get all divs inside of this div using webdriver and selenium?
How do I get all divs inside of this div using webdriver and selenium?

Time:10-24

I am new to webscraping and am trying to scrape a website and would like to get all the elements that have data-feature-id=homepage/story with the ultimate goal of getting a subelement of each of the divs inside of the parent element. I am having trouble getting all of the elements with this parameter and am unsure how to go about doing so.

I've tried the outer div of all the elements that have data-feature-id=homepage/story; however that results in there being only one element in the list returned.

    def getLinks(self):
        container = self.driver.find_element(By.XPATH, '//*[@id="__next"]/div[3]/div/main/article/div')
        for item in container:
            headline = container.find_element(By.XPATH, '//*[@id="__next"]/div[3]/div/main/article/div/div[1]/div/div[1]/a/h3')
            print(headline)

The above returns

Type Error:'WebElement' object is not iterable

Can anyone please give some guidance on how to go about this problem?

css selector

For your Selenium script, try using the following line instead of your XPath Selector:

driver.find_element(By.CSS_SELECTOR, "div[data-feature-id='homepage/story']")
  • Related