Selenium with Python returning empty string for find

I am working on a web scraper to pull the current surf conditions from Surfline using Selenium, but when I attempt to pull the surf height from the "quiver-surf-height" class an empty string is returned.

This is what the Surfline html looks like.

<span >3-4<sup>FT</sup></span>

This is my code.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

surf = driver.find_element(By.CLASS_NAME, "quiver-surf-height").text
print (f"The contents of surf is: {surf}")

CodePudding user response：

After more experimenting, I couldn't find a solution as to why Selenium could not retrieve the data from "quiver-surf-height". However, by performing a search on an element a few levels up, "quiver-spot-forecast-summary__stat", I was able to pull all the subsequent data as text.

surf = driver.find_element(By.CLASS_NAME, "quiver-spot-forecast-summary__stat").text
print (f"The contents of surf is: {surf}")

Returns

The contents of surf is: Surf height
3-4ft
Waist to chest

CodePudding user response：

Try with Xpath instead, right-click on the HTML element and copy the Xpath and paste it into the code below.

driver.find_element(By.XPATH, "INSERT XPATH HERE")