Selenium: element is not attached to the page document restaurant get free food script-CodePudding

Trying to make a free food finder but get unknown error

I'm trying to make this code to get every free product in this food delivery restaurant I expect it to iterate through this 'hbaEIe.sc-5674cfe4-2' elements, that look like this: Restaurant div

url = 'https://www.rappi.com.ar/restaurantes'

for restaurant in all_restaurant:
    link = restaurant.get_attribute("href")
    full_link = base_url   link
    name = restaurant.get_attribute("aria-label")

    # open tab
    driver.find_element(By.TAG_NAME, 'body').send_keys(Keys.CONTROL   't')

    # Load a page
    driver.get(full_link)

    getFreeStuff(name, full_link)

    # close the tab
    driver.find_element(By.TAG_NAME, 'body').send_keys(Keys.CONTROL   'w')

    print(name)

Then, for each restaurant i want to iterate through all the product list and get the price, comparing it to 0.

def getFreeStuff(restaurant, link):
    time.sleep(1)
    prices = driver.find_elements(By.CLASS_NAME, "css-kowr8")
    names = driver.find_elements(By.CLASS_NAME, "css-puxjan")
    for i in range(0, len(prices)):
        price = prices[i]
        if price == "$ 0,00":
            restaurants.append(restaurant)
            links.append(link)
            products.append(names[i])
    return 0

But when i run it it gives me the following error:

BOULEVARD HONORIO
Traceback (most recent call last):
  File "C:\Users\Usuario\Desktop\Web Scraping\practica2\main.py", line 39, in <module>
    link = restaurant.get_attribute("href")
  File "C:\Users\Usuario\Desktop\Web Scraping\practica1\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 155, in get_attribute
    attribute_value = self.parent.execute_script(...

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=107.0.5304.107)
Stacktrace:
Backtrace:
    Ordinal0 [0x00A3ACD3 2075859]
    Ordinal0 [0x009CEE61 1633889]
        ...
    RtlGetAppContainerNamedObjectPath [0x771A7B8E 238]


Process finished with exit code 1

I've tried many things, but i don't know how to proceed

CodePudding user response：

This happens when the page is refreshed but you keep the old reference in your varibale. Try rediscovering element in the dom after refresh.

CodePudding user response：

I suspect that your attempt to open the individual restaurant link in a new tab is not successful. As a result you are no longer on the main page when iterating to the second item in the list. As you are only taking the name and link for each restaurant from the main page I would suggest it is best to store this data and then simply step through all of the restaurant pages adding free items (sadly there don't seem to be any!) to the restaurant objects:

all_restaurant = driver.find_elements(By.XPATH, "//a[@class='sc-5674cfe4-2 hbaEIe']")
for restaurant in all_restaurant:
    link = restaurant.get_attribute("href")
    name = restaurant.get_attribute("aria-label")
    restaurants.append({"name":name, "link":link})
for restaurant in restaurants:
    # Load a page
    driver.get(restaurant["link"])
    products = getFreeStuff(driver, name, link)
    if len(products) > 0:
        restaurant["products"] = products 

def getFreeStuff(driver):
    time.sleep(1)
    products = []
    prices = driver.find_elements(By.CLASS_NAME, "css-kowr8")
    names = driver.find_elements(By.CLASS_NAME, "css-puxjan")
    for i in range(0, len(prices)):
        price = prices[i]
        if price == "$ 0,00":
            products.append(names[i])
    return products