I've been trying to scrape this website Link I'm interested in clicking the first DONWLOAD button. However whenever I try to find any element that is a button, I can't find any.

Here is the code :

url = 'https://ember-climate.org/data/carbon-price-viewer/'
webdriver = create_driver()

with webdriver as driver:
    driver.maximize_window()
    driver.get(url)
    wait = WebDriverWait(driver, 30)
    try:
        wait.until(EC.element_to_be_clickable((By.XPATH, '//button')))
    except:
        pass
    ids = driver.find_elements_by_xpath('//button')
    for ii in ids:
        print (ii.tag_name, ii.get_attribute('class'))

Is there anything wrong with the XPath or is it an issue with the website itself?

CodePudding user response：

Your xpath is wrong, Download button is wrapped inside an span tag not button tag.

try this instead :

//span[contains(text(),'DOWNLOAD')]

also, I see it's in iframe, which can be located via

iframe[name='ETS']

CSS_SELECTOR, and we need to switch also to this iframe.

so in sequence the explanation would be :

1. You would have to click on cookies button.
2. Download button is in an iframe, we need to switch to iframe first and then we can interact with download button.
3. download button is a part of span tag not button tag.
4. Use Explicit waits.
5. Prefer id, css over xpath. (if they are unique in nature)
6. Launch browser in full screen mode.

Code :

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
#driver.implicitly_wait(30)
wait = WebDriverWait(driver, 50)
driver.get("https://ember-climate.org/data/carbon-price-viewer/")
wait.until(EC.element_to_be_clickable((By.ID, "cn-accept-cookie"))).click()
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[name='ETS']")))
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[contains(text(),'DOWNLOAD')]"))).click()

Imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC