Printing web search results won't work in Selenium script, but works when I type it into the sh-CodePudding

I'm very new and learning web scraping in python by trying to get the search results from the website below after a user types in some information, and then print the results. Everything works great up until the very last 2 lines of this script. When I include them in the script, nothing happens. However, when I remove them and then just try typing them into the shell after the script is done running, they work exactly as I'd intended. Can you think of a reason this is happening? As I'm a beginner I'm also super open if you see a much easier solution. All feedback is welcome. Thank you!

#Setup
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
import time

#Open Chrome
driver = webdriver.Chrome()
driver.get("https://myutilities.seattle.gov/eportal/#/accountlookup/calendar")

#Wait for site to load
time.sleep(10)

#Click on street address search box
elem = driver.find_element(By.ID, 'sa')
elem.click()

#Get input from the user
addr = input('Please enter part of your address and press enter to search.\n')

#Enter user input into search box
elem.send_keys(addr)

#Get search results
elem = driver.find_element(By.XPATH, ('/html/body/app-root/main/div/div/account-lookup/div/section/div[2]/div[2]/div/div/form/div/ul/li/div/div[1]'))
print(elem.text)

CodePudding user response：

You are using an absolute XPATH, what you should be looking into are relative XPATHs

Something like this should do it:

elems = driver.find_elements(By.XPATH, ("//[@id='addressResults']/div"))

for elem in elems:
   ...

CodePudding user response：

I haven't used Selenium in a while, so I can only point you in the right direction. It seems to me you need to iterate over the individual entries, and print those, as opposed to printing the entire div as one element.

CodePudding user response：

You should remove the parentheses from the xpath expression
You can shorten the xpath expression as follows:

Code:

elems = driver.find_element(By.XPATH, '//*[@]/div')
for elem in elems:
    print(elem.text)