So I want to return the links to all posts from a specific subreddit on my Reddit homepage. My intuition is to do this by looking for the next link after it finds an href = r/whatever.

CodePudding user response：

you can find all a tags with href attribute and after that, you can iterate through this list.
python implementation.

driver = webdriver.WhateverDriver
links = driver.find_elements(By.XPATH, "//a[@href]") # This will return all links

CodePudding user response：

I was using https://www.reddit.com/r/programming/

I would recommend using infinite scroll load.

Then after use this to grab all the links.

links = [x.get_attribute("href") for x in driver.find_elements(By.XPATH, "//a[@href and @data-click-id='body']")]