Home > Software engineering >  Selenium can not find elements in dynamic web page, page source does not be loaded completely
Selenium can not find elements in dynamic web page, page source does not be loaded completely

Time:11-30

I try to recover some elements in a web page with selenium but the page_source I'm getting it does not have that elements loaded.

Find element returns elem.text empty and driver.page_source does not have the id titulotramitedocu.

What am I missing?

Code:

URL = "https://seu.conselldemallorca.net/fitxa?key=91913"
driver = webdriver.Chrome()
driver.get(URL)
try:
    driver.implicitly_wait(20)
    elem = driver.find_element(By.ID,"titulotramitedocu")
    print(elem.text)
finally:
    driver.quit()

I also tried with a wait..

element = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, "titulotramitedocu"))
)

CodePudding user response:

To locate and print the text from the visible element instead of presence_of_element_located() you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

  • Using CSS_SELECTOR:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.titulotramitedocu#titulotramitedocu > h1"))).text)
    
  • Using XPATH:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='titulotramitedocu' and @id='titulotramitedocu']//h1"))).text)
    
  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    
  • Console Output:

    Concurs de mèrits de personal funcionari del Consell de Mallorca per a la categoria d'enginyer-a tècnic-a industrial d'administració especial (codi CFCEA2/024)
    
  • Related