Trying to scrape a website, I created a loop and was able to locate all the elements. My problem is, that the next button id changes on every page. So I can not use the id as a locator.

This is the next button on page 1:

<a rel="nofollow" id="f_c7" href="#" ></a>

And this is the next button on page 2:

<a rel="nofollow" id="f_c9" href="#" ></a>

---

Idea:

next_button = browser.find_elements_by_class_name("nextLink jasty-link")
next_button.click

I get this error message:

Message: no such element: Unable to locate element

The problem here might be that there are two next buttons on the page. So I tried to create a list but the list is empty.

next_buttons = browser.find_elements_by_class_name("nextLink jasty-link")
print(next_buttons)

Any idea on how to solve my problem? Would really appreciate it.

This is the website:

https://fazarchiv.faz.net/faz-portal/faz-archiv?q=Kryptowährungen&source=&max=10&sort=&offset=0&_ts=1657629187558#hitlist

CodePudding user response：

There are two issues in my opinion:

• Depending from where you try to access the site there is a cookie banner that will get the click, so you may have to accept it first:

  browser.find_element_by_class_name('cb-enable').click()
  

• To locate a single element, one of the both next buttons, it doeas not matter, use browser.find_element() instead of browser.find_elements().

  Selecting your element by multiple class names use xpath:

  next_button = browser.find_element(By.XPATH, '//a[contains(@class, "nextLink jasty-link")]')
  

  or css selectors:

  next_button = browser.find_element(By.CSS_SELECTOR, '.nextLink.jasty-link')
  

Note: To avoid DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() import in addition from selenium.webdriver.common.by import By

CodePudding user response：

You can't get elements by multiple class names. So, you can use find_elements_by_css_selector instead.

next_buttons = browser.find_elements_by_css_selector(".nextLink.jasty-link")
print(next_buttons)

You can then loop through the list and click the buttons:

next_buttons = browser.find_elements_by_css_selector(".nextLink.jasty-link")
for button in next_buttons:
  button.click()

CodePudding user response：

Try below xPath

//a[contains(@class, 'step jasty-link')]/following-sibling::a