I just need to click the load more button once to reveal a bunch more information so that I can scrape more HTML than what is loaded.

The following "should" go to github.com/topics and find the one and only button element and click it one time.

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

driver = webdriver.Edge()

driver.get("https://github.com/topics")
time.sleep(5)
btn = driver.find_element(By.TAG_NAME, "button")
btn.click()
time.sleep(3)
driver.quit()

I'm told Message: element not interactable so I'm obviously doing something wrong but I'm not sure what.

CodePudding user response：

There are several issues with your code:

1. The "Load more" button is initially out of the view, so you have to scroll the page in order to click it.
2. Your locator is bad.
3. You need to wait for elements to appear on the page before accessing them. WebDriverWait expected_conditions explicit waits should be used for that, not hardcoded sleeps.

The following code works, it scrolls the page and clicks "Load more" 1 time.

import time

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 20)


url = "https://github.com/topics"
driver.get(url)

load_more = wait.until(EC.presence_of_element_located((By.XPATH, "//button[contains(.,'Load more')]")))
load_more.location_once_scrolled_into_view
time.sleep(1)
load_more.click()

CodePudding user response：

use

btn = driver.findElementsByXPath("//button[contains(text(),'Load more')]");

You are not finding the right element. This is the reason why it is not "interactable"