xpath inside loop does not locate properly-CodePudding

i have a code snippet where I go to url, to scrape data by blocks of div elements. I am easily locating div bocks by different data-asin numbers. And inside loop locator changes accordingly. then strangely, x_block_of_index=driver.find_element(By.XPATH,locator) works well, i mean according to new data-asin number finds appropriate block but afterwards, y_title=x_block_of_index.find_element(By.XPATH,'//div/h2/a/span') finds only first div block text value. Does not change according loop. I tried many ways, to get why, as you will see i tried to get attribute of span element, as it is DOM property of HTML element

wait=WebDriverWait(driver,timeout=77)

try:
    x_indexes=wait.until(EC.visibility_of_all_elements_located((By.XPATH,'//div[@data-asin]')))
    print(len(x_indexes),'X_INDEX')

    counter=1
    for i in range(len(x_indexes)):
        x_data_asin=x_indexes[i].get_attribute('data-asin')
        if x_data_asin!="":
            print(x_data_asin,i,counter)
            counter =1

            #locating title
            locator='//div[@data-asin="' x_data_asin '"]'
            x_block_of_index=driver.find_element(By.XPATH,locator)
            x_title=x_indexes[i].find_element(By.XPATH,'//div/h2/a/span')
            #print(x_block_of_index.text)
            y_title=x_block_of_index.find_element(By.XPATH,'//div/h2/a/span')
            print(y_title.text)
            print(y_title.get_attribute('textContent'))

except:
    y_indexes=driver.find_elements(By.XPATH,'//div[@data-asin]')
    print(len(y_indexes))

CodePudding user response：

You probably need to add a dot . at the beginning of the XPath to make it relative.
I mean instead of x_title=x_indexes[i].find_element(By.XPATH,'//div/h2/a/span') probably it should be x_title=x_indexes[i].find_element(By.XPATH,'.//div/h2/a/span')
The same about y_title=x_block_of_index.find_element(By.XPATH,'//div/h2/a/span') -> change it to be y_title=x_block_of_index.find_element(By.XPATH,'.//div/h2/a/span').
When you apply XPath locator, driver will start searching from the DOM upper element and it will return the first match. But when you put a dot . before the XPath expression and apply it on some specific element (node) it will start searching form (inside) that specific element (node)