Home > database >  StaleElementReferenceException: Message: stale element reference: element is not attached to the pag
StaleElementReferenceException: Message: stale element reference: element is not attached to the pag

Time：01-01

I'm writing a selenium program in python to get links from a website, it runs the first time but when I re-run it, it opens the website in browser properly then it gives an error

This is my code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service

headers = {
    "User-Agent":
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19582",
    "Accept-Language": "fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3",
    "Connection": "keep-alive",
    "Accept-Encoding": "gzip, deflate, br",
}

edge_driver = './msedgedriver.exe'
s = Service( edge_driver )
browser = webdriver.Edge( service=s )

browser.get( "https://www.jumia.ug/always/" )

all_links = browser.find_elements( By.CLASS_NAME, "core" )
working_links = []
for l in all_links:
    if l.get_attribute( "href" ) is not None:
        working_links.append(l.get_attribute('href'))

print(working_links)

this is the error i got after running it every other time

C:\Users\eliHeist\PycharmProjects\webscraping\venv\Scripts\python.exe C:/Users/eliHeist/PycharmProjects/webscraping/getlinks.py
Traceback (most recent call last):
  File "C:/Users/eliHeist/PycharmProjects/webscraping/getlinks.py", line 34, in <module>
    if l.get_attribute( "href" ) is not None:
  File "C:\Users\eliHeist\PycharmProjects\webscraping\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 155, in get_attribute
    attribute_value = self.parent.execute_script(
  File "C:\Users\eliHeist\PycharmProjects\webscraping\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 878, in execute_script
    return self.execute(command, {
  File "C:\Users\eliHeist\PycharmProjects\webscraping\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "C:\Users\eliHeist\PycharmProjects\webscraping\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: MicrosoftEdge=96.0.1054.57)
Stacktrace:
Backtrace:
    Microsoft::Applications::Events::ILogConfiguration::ILogConfiguration [0x00007FF60587CB62 56946]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054AA597 949863]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054AD788 962648]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054AE3EA 965818]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF605516D9F 1394287]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF605501546 1306134]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF605515D5D 1390125]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6055013BF 1305743]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054D6AA9 1131385]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6054D7B8F 1135711]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF60556C2E5 1743797]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF60556A771 1736769]
    Microsoft::Applications::Events::EventProperty::EventProperty [0x00007FF60574B519 2521]
    Microsoft::Applications::Events::EventProperty::to_string [0x00007FF6055420AE 1571198]
    Microsoft::Applications::Events::ILogConfiguration::ILogConfiguration [0x00007FF60587576C 27260]
    Microsoft::Applications::Events::ILogConfiguration::ILogConfiguration [0x00007FF6058750E4 25588]
    Microsoft::Applications::Events::ILogConfiguration::ILogConfiguration [0x00007FF605874F36 25158]
    Microsoft::Applications::Events::EventProperties::GetName [0x00007FF6057D02DC 211724]
    BaseThreadInitThunk [0x00007FFF89E254E0 16]
    RtlUserThreadStart [0x00007FFF8AF2485B 43]


Process finished with exit code 1

What should I be changing or not doing, the page does not refresh at all?

CodePudding user response：

To print the value of the href attribute you have to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

  • Using CSS_SELECTOR:

    driver.get("https://www.jumia.ug/always/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label='newsletter_popup_close-cta']"))).click()
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.core[data-brand='Always'][href]")))])

  • Using XPATH:

    driver.get("https://www.jumia.ug/always/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@aria-label='newsletter_popup_close-cta']"))).click()
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[@class='core' and @data-brand='Always'][@href]")))])

  • Console Output:

    ['https://www.jumia.ug/always-maxi-long-7s-15736010.html', 'https://www.jumia.ug/always-roxanne-maxi-duo-long-16s-15736015.html', 'https://www.jumia.ug/always-ultra-long-s3-8s-15736020.html', 'https://www.jumia.ug/always-roxanne-dreamz-maxi-ex-long-8s-15736016.html', 'https://www.jumia.ug/stainless-steel-thermos-flask-3-litres-silver-always-mpg51110.html', 'https://www.jumia.ug/always-roxanne-dreamz-maxi-ex-long-16s-15736017.html', 'https://www.jumia.ug/always-ultra-new-12s-vp-sup-pr-african-15736026.html', 'https://www.jumia.ug/travel-vacuum-thermo-cup-450ml-red-always-mpg59272.html', 'https://www.jumia.ug/pressing-flask-3litres-silver-always-mpg49765.html', 'https://www.jumia.ug/stainless-steel-travel-mug-silver-always-mpg69987.html', 'https://www.jumia.ug/unbreakable-2.5-litres-vaccum-flask-silver-always-mpg54323.html', 'https://www.jumia.ug/stainless-steel-vacuum-flask-pressing-3.5l-silver-always-mpg56188.html', 'https://www.jumia.ug/always-dailies-flexistyle-slim-panty-liners-breathable-flexible-with-fresh-scent-pack-of-26-10968170.html', 'https://www.jumia.ug/stainless-steel-vacuum-flask-500ml-black-always-mpg48945.html', 'https://www.jumia.ug/always-flask-pressing-colour-silver-16418821.html', 'https://www.jumia.ug/vaccum-travel-flask-450ml-blue-always-mpg73899.html', 'https://www.jumia.ug/500mls-vacuum-hot-cold-bottle-flask-sliver-always-mpg54381.html', 'https://www.jumia.ug/always-ultra-lw-14s-vp-sup-pr-african-15736025.html', 'https://www.jumia.ug/pressing-2.0-litres-unbreakable-vaccum-jar-flask-stainless-steel-always-mpg56170.html', 'https://www.jumia.ug/stainless-steel-travel-cap-black-always-mpg65745.html', 'https://www.jumia.ug/always-stainless-steel-travel-mug-0.48l-navy-blue-8224366.html', 'https://www.jumia.ug/always-thermal-flask-cup-silver-black-20572534.html', 'https://www.jumia.ug/stainless-vacuum-travel-mug-450ml-black-always-mpg56189.html', 'https://www.jumia.ug/0.5l-insulated-stainless-steel-travel-mug-colour-varies-always-mpg73677.html', 'https://www.jumia.ug/stainless-steel-travel-mug-0.5l-gold-always-mpg67096.html', 'https://www.jumia.ug/ultra-platinum-long-7-pads-always-mpg59273.html', 'https://www.jumia.ug/life-travel-flask-450ml-red-always-mpg72907.html', 'https://www.jumia.ug/always-xtra-protection-feminine-panty-liners-extra-long-92-pieces-us-4766946.html', 'https://www.jumia.ug/always-zzz-overnight-pads-for-women-size-6-with-wings-for-worry-free-nights-20ct-14336082.html', 'https://www.jumia.ug/portable-vacuum-thermos-flask-bottle-0.5ltr-silver-always-mpg48676.html', 'https://www.jumia.ug/stainless-steel-vacuum-flask-700ml-silver-always-mpg59271.html', 'https://www.jumia.ug/always-vacuum-hot-cold-bottle-flask-500mls-always-mpg46382.html', 'https://www.jumia.ug/stainlesss-steel-side-pressing-vaccum-flask-3litres-silver-always-mpg65748.html', 'https://www.jumia.ug/always-stainless-steel-thermos-flask-jug-silver-always-mpg42506.html', 'https://www.jumia.ug/always-450ml-thermal-flask-maroon-5494832.html', 'https://www.jumia.ug/always-food-flask-18949481.html', 'https://www.jumia.ug/always-portable-vacuum-1.0ltr-thermos-flask-bottle-silver-8032566.html', 'https://www.jumia.ug/always-stainless-vaccum-flask-3-litre-silver-black-always-mpg46820.html', 'https://www.jumia.ug/stainless-steel-vaccum-flask-3.5l-silver-always-mpg48943.html', 'https://www.jumia.ug/always-stainless-steel-vacuum-flask-silver-5196899.html']

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

CodePudding user response：

wait=WebDriverWait(browser,10)                                 
browser.get("https://www.jumia.ug/always/")
hrefs=[x.get_attribute('href') for x in wait.until(EC.visibility_of_all_elements_located((By.XPATH,"//a[@class='core']")))]
print(hrefs)

Just simply wait and grab all the href attribute with that class.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC

Page link：https//www.codepudding.com/database/246821.html

Prev:My user control should only be added to a certain type of control
Next:Message: stale element reference: element is not attached to the page document after returning a pag
  • Related

About Us:  Contact Us      Terms of Service       Privacy Policy

Copyright © 2010-2023，Powered By CodePudding