I'm having trouble trying to figure out how to locate the path for a "Load More Button" on this site I am trying to scrape.

https://www.ufc.com/athletes/all

The buttons html:

<a  href="?gender=All&amp;search=&amp;page=1" title="Load more items" 
 rel="next">Load More</a>  
 "Load More" == $0
 ::after == 0$
 

CodePudding user response：

You didn't provide any attempts and I can see 3 right off the top of my head that should work...

1. CSS selector, a[title='Load more items']
2. XPath, //a[text()='Load More']
3. By link text, "Load More"

In general you should prefer CSS selectors because they are more widely supported, faster, and have simpler syntax.

CodePudding user response：

To click on the element LOAD MORE you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

• Using CSS_SELECTOR:

  driver.get('https://www.ufc.com/athletes/all')
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[title='Load more items']"))).click()
  

• Using XPATH:

  driver.get('https://www.ufc.com/athletes/all')
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='onetrust-accept-btn-handler']"))).click()
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@title='Load more items']"))).click()
  

• Note: You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

CodePudding user response：

this is the XPATH for that button that you want you can right-click on an element to inspect and right-click on it to copy XPath

driver.find_element(By.XPATH, ' //*[@id="block- 
 mainpagecontent"]/div/div/div[2]/div/div/ul/li/a').click()