Locating an element using Python and Selenium via innerHTML-CodePudding

I'm new to Selenium and I'm trying to write my first real script using the package for Python.

I'm using:

Windows 10
Python 3.10.5
Selenium 4.3.0

So far I've been able to do everything I need with different selectors, like ID, name, XPATH etc.

However I've stumbled upon an issue where I need to find a specific element by using the innerHTML of it.

The issue I'm facing is that I need to find an element with the innerHTML-value of "Changed" as seen in the HTML below.

The first challenge I'm facing is that the element doesn't have a unique ID, name or otherwise to identify it and there's many objects/elements of "dlx-treeview-node". The second challenge is that XPATH won't work because the element changes position depending on where you are on the website (the number of "dlx-treeview-node"-elements change), so if I use XPATH I'll get the wrong element depending on where I am.

I can successfully get the name by using the below XPATH, "get_attribute" and printing to console, which is why I know it's innerHTML and not innerText, but as mentioned this will change depending on where I am on the website.

I would really appreciate any help I can get to solve this challenge and to learn more about the use of Selenium with Python.

Code trials:

select_filter_name = wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/div/app-root/dlx-select-filter-attribute-dialog/dlx-dialog-window/div/div[2]/div/div/div[5]/div/div/dlx-view-column-selector-component/div[1]/dlx-treeview/div/dlx-treeview-nodes/div/dlx-treeview-nodes/div/dlx-treeview-node[16]/div/div/div/div[2]/div/dlx-text-truncater/div")))
filter_name = select_filter_name.get_attribute("innerHTML")
print(filter_name)

HTML:

<dlx-treeview-node _nghost-nrk-c188="" >
  <div _ngcontent-nrk-c188="" dlx-droppable="" dlx-draggable="" dlx-file-drop="" >
    <div _ngcontent-nrk-c188="" >
      <div _ngcontent-nrk-c188=""  style="padding-left: 15px;">
        <!---->
        <div _ngcontent-nrk-c188=""  style="min-width: 16px; margin-left: 3px;">
          <!---->
        </div>
        <!---->
        <div _ngcontent-nrk-c188=""  style="padding: 3.5px 0px;">
          <div >
            <!---->
            <dlx-text-truncater >
              <div >Changed</div>
              <!---->
              <!---->
            </dlx-text-truncater>
            <!---->
          </div>
          <!---->
          <!---->
          <!---->
        </div>
      </div>
      <!---->
      <!---->
    </div>
  </div>
  <!---->
  <dlx-attachment-content _ngcontent-nrk-c188="">
    <div style="position: fixed; z-index: 10001; left: -10000px; top: -10000px; pointer-events: auto;">
      <!---->
      <!---->
    </div>
  </dlx-attachment-content>
</dlx-treeview-node>

Edit-1:

NOTE: I'm not sure I'm using the correct terms for HTML, so please correct me if I'm wrong.

I've learned that I have 3 follow up questions:

How do I search for the text as described, but only searching in the "dlx-treeview-node" (there's about 100 of these)? So basically searching in the "children" of these.
A new issue: It seems that I can find the element, but I can't select it - even though I can see it and select it on screen?
The element I'm searching for is in a drop-down-menu and by running some tests today I've now come across a new issue where in some places I get "ElementClickInterceptedException" because the element appears lower in the list in the drop-down where a user would normally scroll down the menu and click it. Any ideas?

CodePudding user response：

Presuming the innerText of the <div> element as a unique text within the HTML DOM to locate the element with the innerHTML as Changed you can use either of the following xpath based locator strategies:

Using xpath and text():

element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[text()='Changed']")))

Using xpath and contains():

element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(., 'Changed')]")))

CodePudding user response：

just run this code on your page and you will get an array of all elements which are a div with the value of Changed

# Define XPath Function (used in the next step)
driver.execute_script("function getXPathOfElement(elt) {var path = "";for (; elt && elt.nodeType == 1; elt = elt.parentNode) { idx = getElementIdx(elt); xname = elt.tagName; if (idx > 1) xname  = "["   idx   "]"; path = "/"   xname   path;} return path;}")

# Get all XPaths for all nodes which are a div with the text of "changed"
xpaths = driver.execute_script("return Array.from(document.querySelectorAll(\"div\")).find(el => el.textContent.includes('Changed')).map((node)=>{ return getXPathOfElement(node)});');

Tips

check if xpaths length is more than or equal to 1
index xpaths such as xpaths[0] or do loops to make your changes
you will now have an xpath which can be used like a normal selector.

good luck