Pulling all first instances of a tag from multiple classes of the same name with Selenium-CodePudding

I am running a Selenium script to scrape all of the first instances of h3s from multiple classes of the same name, using the index [0] to pull the first h3 (of 4). My script appears to run fine, but it's returning an empty array. I've tried the WebDriverWait solution to no avail.

Here's what I've got currently:

url = 'https://www.riversidemedgroup.com/riverside-urgent-care/'

data = []

driver = webdriver.Chrome(executable_path='/Library/Frameworks/Python.framework/Versions/3.9/bin/chromedriver')
driver.get(url)

centers = driver.find_elements_by_xpath("//h3[contains(@class,'elementor-widget-container')][0]")
for center in centers:
     data.append({
     "Center Name": center
     })
print(data)
driver.close()

Can't figure out where I've gone wrong. Any advice is much appreciated!

CodePudding user response：

I switched from XPath to CSS Selectors for simplicity. Below code should work properly.

from bs4 import BeautifulSoup
from selenium import webdriver
import time
url = 'https://www.riversidemedgroup.com/riverside-urgent-care/'

data = []

driver = webdriver.Chrome()
driver.get(url)

centers = driver.find_elements_by_css_selector(".elementor-widget-container > h3:first-of-type")
for center in centers:
     data.append({
     "Center Name": center.text
     })
print(data)
driver.close()

Output:

[{'Center Name': 'Bloomfield, NJ Urgent Care'}, 
{'Center Name': 'Cedar Knolls, NJ Urgent Care'}, 
{'Center Name': 'Cherry Hill, NJ Urgent Care'}, 
{'Center Name': 'Cinnaminson, NJ Urgent Care'}, 
{'Center Name': 'East Brunswick, NJ Urgent Care'}, 
{'Center Name': 'Ewing, NJ Urgent Care'}, 
{'Center Name': 'Hackettstown, NJ Urgent Care'}, 
{'Center Name': 'Hamilton, NJ Urgent Care'}, 
{'Center Name': 'Hazlet, NJ Urgent Care'}, 
{'Center Name': 'Howell, NJ Urgent Care'}, 
{'Center Name': 'Ledgwood, NJ Urgent Care'}, 
{'Center Name': 'Linden, NJ Urgent Care'}, 
{'Center Name': 'Lodi, NJ Urgent Care'}, 
{'Center Name': 'Mount Ephraim, NJ Urgent Care'}, 
{'Center Name': 'Nutley, NJ Urgent Care'}, 
{'Center Name': 'Pennsville, NJ Urgent Care'}, 
{'Center Name': 'Rockaway, NJ Urgent Care'}, 
{'Center Name': 'Runnemede, NJ Urgent Care'}, 
{'Center Name': 'Springfield, NJ Urgent Care'}, 
{'Center Name': 'Totowa, NJ Urgent Care'}, 
{'Center Name': 'Vineland South, NJ Urgent Care'}, 
{'Center Name': 'Vineland, NJ Urgent Care'}, 
{'Center Name': 'Wall, NJ Urgent Care'}, 
{'Center Name': 'Watchung, NJ Urgent Care'}, 
{'Center Name': 'Willingboro, NJ Urgent Care'}, 
{'Center Name': 'Woodbury, NJ Urgent Care'}, 
{'Center Name': ''}, 
{'Center Name': 'Derby, CT Urgent Care'}, 
{'Center Name': 'Meriden, CT Urgent Care'}, 
{'Center Name': 'Middletown, CT Urgent Care'}]