Selenium WebDriver find_elements methods don't find empty elements-CodePudding

I'm using Selenium WebDriver with python, with ChromeDriver.

When trying to find th elements in a thead, find_elements method only find 3 of them, while there are 4 in the DOM. One of them is not found, it turns out it's an empty element :

<table>
       <thead>
          <tr>
             <th  scope="col"></th>
             <th scope="col">
                <div >Whatever</div>
             </th>
             <th scope="col">
                <div >Whatever2</div>
             </th>
             <th scope="col">
                <div >Whatever3</div>
             </th>
          </tr>
       </thead>
       <tbody >
          <tr>
             <td width="16">
                <svg><!-- some svg --></svg>
             </td>
             <td>Whatever</td>
             <td>Whatever2</td>
             <td>Whatever3</td>
          </tr>
       </tbody>
    </table>

I've used the following find_elements calls :

tr = thead.find_element_by_tag_name("tr")
table_headers = tr.find_elements_by_tag_name("th")  
table_headers2 = tr.find_elements(By.TAG_NAME,"th")  
table_headers3 = tr.find_elements(By.XPATH,"//th")  
table_headers4 = tr.find_elements(By.XPATH,"//node()[contains(@scope,'col')]")  
table_headers5 = driver.find_elements_by_xpath("//thead/tr/*")

They're all finding only 3 elements.

CodePudding user response：

Edit: One more thing:

Selenium 4.3.0

Deprecated find_element_by_* and find_elements_by_* are now removed (#10712)

Not sure how kosher this is, but let's give it a try:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.relative_locator import locate_with
import pandas as pd
import time as t

chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")


webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)

url = 'https://stackoverflow.com/questions/73068196/selenium-webdriver-find-elements-methods-dont-find-empty-elements#73068196'
browser.get(url)
button = browser.find_element(By.XPATH, "//*[contains(text(), 'Run code snippet')]")
button.click()
iframe = WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//iframe[@name='sif2']")))
browser.switch_to.frame(iframe)
t.sleep(1)
# table_head = browser.find_element(By.TAG_NAME, "thead").get_attribute("outerHTML")
table_head = browser.find_element(By.TAG_NAME, "thead")
th_cells = table_head.find_elements(By.TAG_NAME, "th")
for x in range(len(th_cells)):
    print('th:', x, th_cells[x].get_attribute("outerHTML"))
browser.quit()

This returns 4 elements:

th: 0 <th  scope="col"></th>
th: 1 <th scope="col">
                <div >Whatever</div>
             </th>
th: 2 <th scope="col">
                <div >Whatever2</div>
             </th>
th: 3 <th scope="col">
                <div >Whatever3</div>
             </th>

So .. your original table must be different, or I'm missing something in your code.