input suggestions webscraping selenium python-CodePudding

Input Element:

<input type="text" auto_complete_item_format="<b xmlns=&quot;none&quot;>[_full_name]</b>" auto_complete_display_field="_full_name" auto_complete_id_field="suburb_id" auto_complete_search_fields="_full_name,suburb" auto_complete_minimum_characters="1" auto_complete_data_source_web_method="" auto_complete_data_source_object="" auto_complete_data_source_function="get_suburb_suggestions" auto_complete_width="300px" onblur="smart_update(this);" onkeypress="smart_keypress(this, event); " onchange="smart_date_onchange(this); " selected_id="-1" original_value="" value="3000" smart_field_type="auto_complete"  id="suburb" name="suburb" field="jims.addresses.smart_table.suburb" record_id="-1" bind="True" auto_complete_cursor="2" autocomplete="off" fdprocessedid="5bithf" auto_complete_last_search_string="300" selected_row_index="-1" style="width: 300px;">

Suggestions List:

<div id="suburb_list" name="suburb_list" style="display: none; position: absolute; border: 1px solid rgb(0, 0, 0); z-index: 100; left: 533px; height: 400px; width: 300px; background-color: white; opacity: 0.93; overflow: auto;"><div id="suburb_list_item_0" list_index="0" row_index="0" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: rgb(221, 221, 221); padding: 3px;"><b xmlns="none">150 Lonsdale Street, Melbourne 3000 VIC</b></div><div id="suburb_list_item_1" list_index="1" row_index="1" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: rgb(255, 255, 255); padding: 3px;"><b xmlns="none">1 Elizabeth Street, Melbourne 3000 VIC</b></div><div id="suburb_list_item_2" list_index="2" row_index="4" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: lightblue; padding: 3px;"><b xmlns="none">Carlton 3000 VIC</b></div><div id="suburb_list_item_3" list_index="3" row_index="5" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: rgb(255, 255, 255); padding: 3px;"><b xmlns="none">Docklands 3000 VIC</b></div><div id="suburb_list_item_4" list_index="4" row_index="10" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: rgb(221, 221, 221); padding: 3px;"><b xmlns="none">East Melbourne 3000 VIC</b></div><div id="suburb_list_item_5" list_index="5" row_index="12" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: rgb(255, 255, 255); padding: 3px;"><b xmlns="none">Footscray 3000 VIC</b></div><div id="suburb_list_item_6" list_index="6" row_index="15" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: rgb(221, 221, 221); padding: 3px;"><b xmlns="none">Melbourne 3000 VIC</b></div><div id="suburb_list_item_7" list_index="7" row_index="19" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: rgb(255, 255, 255); padding: 3px;"><b xmlns="none">Roxburgh Park 3000 VIC</b></div><div id="suburb_list_item_8" list_index="8" row_index="28" style="border-width: 1px; border-color: rgb(170, 170, 170); border-style: solid; background-color: rgb(221, 221, 221); padding: 3px;"><b xmlns="none">West Melbourne 3000 VIC</b></div></div>

My Code:

        for num in range(30):
            try:
                Element = browser.find_element(By.XPATH, f'//*[@id="suburb_list_item_{num}"]')
                print(Element)
            except:
                break

The two html code at the top are for a input suggestion list and i am trying to scrape of the website. The python code is the code i used and it should print all the elements. But it does not because the suggestion list is clicked. How could i click the text box because this didn't work:

Element = browser.find_element(By.XPATH, '//*[@id="suburb"]')

the element could not be found.

I am not allowed to share the website link.

CodePudding user response：

You can try:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# <insert other code here for setting up the browser>

input_element = wait.until(EC.visibility_of_element_located((By.ID, "suburb")))
input_element.click()

wait = WebDriverWait(browser, 10)
element = wait.until(EC.visibility_of_element_located((By.ID, "suburb_list")))