Finding value using Xpath (with no unique identifiers) in Python Selenium-CodePudding

I'm having trouble trying to get the colour of a vehicle in Selenium using Python. I've checked YouTube, stackoverflow and all the usual resources but can't seem to find an answer that makes sense (I'm relatively new to Python and Selenium). I'm currently undertaking a project to automate the fetching of vehicle colour from the gov.uk website into an excel sheet based on the Vehicle Registration number already present on the spreadsheet. The code isn't finished yet, as I want to get over this Xpath hurdle first!

I need to fetch the 'Blue' value from this code:

<dl >
<div >
<dt>Registration number</dt>
<dd>
<div >WJ06 HYF</div>
</dd>
</div>
<div >
<dt>Make</dt>
<dd>VOLKSWAGEN</dd>
</div>
<div >
<dt>Colour</dt>
<dd>BLUE</dd>
</div>
</dl>

However, as you can see, they have made it very difficult as there is no specific ID, class, tag name, etc to work with so I'm assuming Xpath is my only option? could anyone help me as to the best implementation of this? My assumption is to find the first 'dd' tag underneath 'Colour' but I don't know how to write this! Here is the code snippet I'm working on that I have so far:

try:
main = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, "summary-no-action"))
    )

div = main.find_element(By.LINK_TEXT, "Colour")
for article in div:
    header = article.find_element(By.TAG_NAME, "dd")
    print(header.text)
finally:
driver.quit()

I'm aware the line 'div = main.find_element(By.LINK_TEXT, "Colour")' is incorrect, but I need to replace it with something so that I may fetch the colour present in the 'dd' tag underneath.

This is what I had originally, but it brings back all the values in the "summary-no-action" class name:

try:
main = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, "summary-no-action"))
    )

div = main.find_elements(By.CLASS_NAME, "govuk-summary-list__row")
for article in div:
    header = article.find_element(By.TAG_NAME, "dd")
    print(header.text)
finally:
driver.quit()

Any help would be appreciated!

EDIT:

For reference, here is the whole code:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

driver =     
webdriver.Chrome(service=Service(ChromeDriverManager().install()))
driver.get("https://www.google.com")

driver.get ("https://vehicleenquiry.service.gov.uk/")

time.sleep(5)

search = driver.find_element(By.ID ,                 
"wizard_vehicle_enquiry_capture_vrn_vrn")
search.send_keys("wj06hyf")
search.send_keys(Keys.RETURN)

try:
main = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, "summary-no-action"))
    )

div = main.find_elements(By.CLASS_NAME, "govuk-summary-list__row")
for article in div:
    header = article.find_element(By.TAG_NAME, "dd")
    print(header.text)
finally:
driver.quit()

CodePudding user response：

Use the following xpath to get the value BLUE. first identify the dt tag with text colour and following dd tag

//dt[text()='Colour']/following::dd[1]

code:

print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH, "//dt[text()='Colour']/following::dd[1]"))).text)

CodePudding user response：

To fetch the text Blue you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

Using CSS_SELECTOR and get_attribute("innerHTML"):

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "dl.summary-no-action div:last-child dd"))).get_attribute("innerHTML"))

Using XPATH and text attribute:

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//dl[@class='summary-no-action']//div[@class='govuk-summary-list__row']/dt[text()='Colour']//following-sibling::dd[1]"))).text)

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC