Selenium get text by xpath-CodePudding

I am trying to copy a text element from a webpage and print it in my console just as a test for a future project.

These are the lines when I get the error:

elem = driver.find_element_by_xpath("/html/body/text()[2]")
print(elem.text)

And the error says:

C:\Users\hp\Desktop\facebook-creator-studio-bot-master\get_cnp.py:12: DeprecationWarning: find_element_by_xpath is deprecated. Please use find_element(by=By.XPATH, value=xpath) instead
  driver.find_element_by_xpath("/html/body/form/input[2]").click()
C:\Users\hp\Desktop\facebook-creator-studio-bot-master\get_cnp.py:13: DeprecationWarning: find_element_by_xpath is deprecated. Please use find_element(by=By.XPATH, value=xpath) instead
  elem = driver.find_element_by_xpath("/html/body/text()[2]")
Traceback (most recent call last):
  File "C:\Users\hp\Desktop\facebook-creator-studio-bot-master\get_cnp.py", line 13, in <module>
    elem = driver.find_element_by_xpath("/html/body/text()[2]")
  File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 521, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1248, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 425, in execute
    self.error_handler.check_response(response)
  File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: The result of the xpath expression "/html/body/text()[2]" is: [object Text]. It should be an element.
  (Session info: chrome=99.0.4844.82)
Stacktrace:
Backtrace:
        Ordinal0 [0x00769943 2595139]
        Ordinal0 [0x006FC9F1 2148849]
        Ordinal0 [0x005F4528 1066280]
        Ordinal0 [0x005F6E04 1076740]
        Ordinal0 [0x005F6CBE 1076414]
        Ordinal0 [0x005F6F50 1077072]
        Ordinal0 [0x00620D1E 1248542]
        Ordinal0 [0x006211CB 1249739]
        Ordinal0 [0x0064D812 1431570]
        Ordinal0 [0x0063BA34 1358388]
        Ordinal0 [0x0064BAF2 1424114]
        Ordinal0 [0x0063B806 1357830]
        Ordinal0 [0x00616086 1204358]
        Ordinal0 [0x00616F96 1208214]
        GetHandleVerifier [0x0090B232 1658114]
        GetHandleVerifier [0x009C312C 2411516]
        GetHandleVerifier [0x007FF261 560433]
        GetHandleVerifier [0x007FE366 556598]
        Ordinal0 [0x0070286B 2173035]
        Ordinal0 [0x007075F8 2192888]
        Ordinal0 [0x007076E5 2193125]
        Ordinal0 [0x007111FC 2232828]
        BaseThreadInitThunk [0x76CA6359 25]
        RtlGetAppContainerNamedObjectPath [0x77827C24 228]
        RtlGetAppContainerNamedObjectPath [0x77827BF4 180]

Then my chomedrive close. What should be the problem ?

CodePudding user response：

This error message...

selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: The result of the xpath expression "/html/body/text()[2]" is: [object Text]. It should be an element.

...implies that the locator strategies you have used is an invalid selector as,

driver.find_element_by_xpath("/html/body/text()[2]")

would return the second matching text node where as Selenium supports only elements.

This usecase

If your usecase is to retrieve the text from an element, you need to locate the element uniquely within the DOM Tree and then extract the innerText using get_attribute() as follows:

Using css_selector and get_attribute("innerHTML"):

print(driver.find_element(By.CSS_SELECTOR, "element_cssSelector").get_attribute("innerHTML"))

Using xpath and text attribute:

print(driver.find_element(By.XPATH, "element_xpath").text)

tl; dr

Difference between text and innerHTML using Selenium

CodePudding user response：

As clearly described in the error trace the problem here is

invalid selector: The result of the xpath expression "/html/body/text()[2]" is: [object Text]. It should be an element.

With print(elem.text) you are trying to apply .text method on elem web element object so driver.find_element_by_xpath() needs a locator to a web element on the web page while "/html/body/text()[2]" is not a valid XPath locator to a web element.
For example ``"/html/body"may be a valid locator to a web element, while/text()refers to atext` attribute value of some web element, but not to a web element object.
UPD
What you can do here is to get the web element, extract it text and then extract the desired text part from there as following:

elem = driver.find_element_by_xpath("/body")
print(elem.text)

This will give you several text string, not only the generated code, unfortunately we can't make better work with selenium since the text you are looking for is sitting inside the body element itself.
You can split the received text after that to extract the code from it.