python selenium : impossible to find element (input in a table)-CodePudding

I ve tried many ways to find the element, but the retrieved element gives me an empty list. This is the page : https://www.avocatparis.org/annuaire I try to locate the "nom" input form. When i copy the xpath i get

//*[@id="_ctl0_Corps_txtRSNom"]

and when i copy the full xpath i get

/html/body/form/table/tbody/tr/td[2]/table/tbody/tr[2]/td/table/tbody/tr/td/div/div/div[2]/table/tbody/tr[1]/td[2]/input

When i put that into my code :

input = self.driver.find_elements(by=By.XPATH,value='//*[@id="_ctl0_Corps_txtRSNom"]')

I get an empty list.

Am i missing something?

Thank you.

CodePudding user response：

NEW

The desired element is within an <iframe>, so you have to use WebDriverWait for waiting the iframe to be available, and then switch to it. Then you can get the element with the usual find_elements command.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver.get('https://www.avocatparis.org/annuaire')
WebDriverWait(driver, 30).until(EC.frame_to_be_available_and_switch_to_it((By.TAG_NAME, "iframe")))
element = driver.find_elements(By.XPATH, '//*[@id="_ctl0_Corps_txtRSNom"]')
print(len(element))

and you will see that the print is 1.

OLD

This is not a true answer since it doesn't solve the problem, however it was too long to be written in the comments.

By inspecting the text version of the HTML downloaded by selenium (see below) it turns out that it lacks a lot of the HTML that you find by manually opening the inspector tool of the browser. In particular the table containing the element you are interested in, i.e. #Table4, is not contained in the selenium page source. I guess it's a problem related to some javascript not loading. I tried both with chrome and firefox.

To see selenium page source run

print(driver.page_source)

Then search for <table (CTRL F) you will see that there are only 5 results, all related to CookiebotDialog. Instead, if you go in the browser inspector tool and search //table you will see 11 results.

If I found a way to let selenium download the complete HTML code, I will update the answer.

From selenium documentation

WebDriver Get the source of the last loaded page. If the page has been modified after loading (for example, by Javascript) there is no guarantee that the returned text is that of the modified page. Please consult the documentation of the particular driver being used to determine whether the returned text reflects the current state of the page or the text last sent by the web server.