I'm Getting an error while implementing the following scenario: Driver goes to a url gets a list of elements and iterates over them, for each iteration it has to go on another link and get some information, This is working fine but once we come back from the page in the next iterations all the previous elements stored seems to be throwing the following exception: "StaleElementReferenceException: Message: stale element reference: element is not attached to the page document"
Following is my code:
driver.get('https://www.jobs.nhs.uk/xi/search_vacancy/')
input=driver.find_elements(By.XPATH,'//*[@id="typeahead_key"]')
if (len (input)>0):
input[0].send_keys('nurse')
else:
print ('Not found')
button = driver.find_element(By.NAME, 'searchBtn1')
button.click()
i=1
d=[]
while (i<=6):
vacancies= driver.find_elements(By.XPATH,"//*[@class='vacancy']")
for vac in vacancies:
print (len(vacancies))
print (vacancies[1].text)
print("-----------------------" str(i) "-------------------------")
ls = vac.text.split("\n")
print (ls[10][14:])
driver.get('https://beta.jobs.nhs.uk/candidate/jobadvert/' ls[10][14:])
content=driver.find_element(By.XPATH,"//*[@class='nhsuk-grid-column-one-third']")
print (content.text)
# print (vac.text)
# d.append(vac.text)
driver.back()
print("----------------------------------------------------")
i =1
try:
driver.find_element(By.XPATH,'//*[@title="Next page"]').click()
except WebDriverException:
exit(1)
Error Log:
Traceback (most recent call last):
File "C:\Users\Lenovo T14\Documents\jo\sel\sample.py", line 46, in <module>
print (vacancies[1].text)
File "C:\Users\Lenovo T14\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webelement.py", line 85, in text
return self._execute(Command.GET_ELEMENT_TEXT)['value']
File "C:\Users\Lenovo T14\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webelement.py", line 773, in _execute
return self._parent.execute(command, params)
File "C:\Users\Lenovo T14\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 430, in execute
self.error_handler.check_response(response)
File "C:\Users\Lenovo T14\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=103.0.5060.114)
Stacktrace:
Backtrace:
Ordinal0 [0x00FDD953 2414931]
Ordinal0 [0x00F6F5E1 1963489]
Ordinal0 [0x00E5C6B8 837304]
Ordinal0 [0x00E5F0B4 848052]
Ordinal0 [0x00E5EF72 847730]
Ordinal0 [0x00E5F200 848384]
Ordinal0 [0x00E84FDF 1003487]
Ordinal0 [0x00EA449C 1131676]
Ordinal0 [0x00E7FC74 982132]
Ordinal0 [0x00EA46B4 1132212]
Ordinal0 [0x00EB4812 1198098]
Ordinal0 [0x00EA42B6 1131190]
Ordinal0 [0x00E7E860 976992]
Ordinal0 [0x00E7F756 980822]
GetHandleVerifier [0x0124CC62 2510274]
GetHandleVerifier [0x0123F760 2455744]
GetHandleVerifier [0x0106EABA 551962]
GetHandleVerifier [0x0106D916 547446]
Ordinal0 [0x00F75F3B 1990459]
Ordinal0 [0x00F7A898 2009240]
Ordinal0 [0x00F7A985 2009477]
Ordinal0 [0x00F83AD1 2046673]
BaseThreadInitThunk [0x76136739 25]
RtlGetFullPathName_UEx [0x77C18FEF 1215]
RtlGetFullPathName_UEx [0x77C18FBD 1165]
Update: I have found a workaround by opening further links into other tabs, Get the information needed and switch back to the previous tab.
CodePudding user response：
DOM Issue:
This has to do with the vacancies element not being attached to the DOM. See Selenium Docs on this error.
A common technique used for simulating a tabbed UI in a web app is to prepare DIVs for each tab, but only attach one at a time, storing the rest in variables. In this case, it's entirely possible that your code might have a reference to an element that is no longer attached to the DOM (that is, that has an ancestor which is document.documentElement).
If WebDriver throws a stale element exception in this case, even though the element still exists, the reference is lost. You should discard the current reference you hold and replace it, possibly by locating the element again once it is attached to the DOM.
You cannot perform operations on an element after the webpage has been changed. You need to re-establish your variables after every DOM change to have the element be a part of the current DOM structure so that you can interact with the elements. For your particular example, you reference "vacancies" once, yet change the webpage several times. For this I could recommend putting the vacancies variable within the loop and find by a single element:
driver.find_element(By.XPATH, ...)
CodePudding user response：
Your workaround is the solution I would recommend. Elements on the webpage are tied to that particulate instance of the page, due to stuff involving the DOM as @LukeHamilton said. If you ever refresh a web page or navigate away and back to it you will need to relocate all elements.
I recommend using your workaround. Just open a new window or tab and switch between them.
main_tab = driver.curren_window_handle
while (i<=6):
vacancies= driver.find_elements(By.XPATH,"//*[@class='vacancy']")
for vac in vacancies:
print (len(vacancies))
print (vacancies[1].text)
print("-----------------------" str(i) "-------------------------")
ls = vac.text.split("\n")
print (ls[10][14:])
driver.switch_to.new_window('tab')
driver.get('https://beta.jobs.nhs.uk/candidate/jobadvert/' ls[10][14:])
content=driver.find_element(By.XPATH,"//*[@class='nhsuk-grid-column-one-third']")
print (content.text)
# print (vac.text)
# d.append(vac.text)
driver.close()
driver.switch_to.window(main_tab)