How to scroll the element until a certain word appears?-CodePudding

I'm scraping Google Maps and I need to know how to scroll the query column until the word appears

"You've reached the end of the list".
I am using selenium for scraping.
Code I currently use:

for a in range(100):
    barraRolagem = wait.until(EC.presence_of_element_located((By.XPATH, 
    "//div[@role='main']//div[contains(@aria-label,'"   procurar   "')]")))
     driver.execute_script("arguments[0].scroll(0, arguments[0].scrollHeight);", barraRolagem)

this code works, however the range is variable and is not certain, that is, it can scroll a lot and it can also scroll a little, for this reason I need to stop scrolling when the code finds the phrase

"You've reached the end of the list"

Link

https://www.google.com.br/maps/search/contabilidade balneario camboriu/@-26.9905418,-48.6289914,15z

CodePudding user response：

You can scroll in a loop until "You've reached the end of the list" text is visible.
When text is found visible - break the loop. Otherwise do a scroll.
Since in case element not visible exception is thrown try-except block is needed here.
Additional scroll is added after the element is found visible since Selenium detects that element visible while it is still not actually visible, so one more scroll should be done finally.
The following code works correctly:

import time

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")
options.add_argument('--disable-notifications')

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 1)

url = "https://www.google.com.br/maps/search/contabilidade balneario camboriu/@-26.9905418,-48.6289914,15z"
driver.get(url)
while True:
    try:
        wait.until(EC.visibility_of_element_located((By.XPATH, "//span[contains(text(),'reached the end')]")))
        barraRolagem = wait.until(EC.presence_of_element_located((By.XPATH, "//div[@role='main']//div[@aria-label]")))
        driver.execute_script("arguments[0].scroll(0, arguments[0].scrollHeight);", barraRolagem)
        break
    except:
        barraRolagem = wait.until(EC.presence_of_element_located((By.XPATH, "//div[@role='main']//div[@aria-label]")))
        driver.execute_script("arguments[0].scroll(0, arguments[0].scrollHeight);", barraRolagem)
        time.sleep(0.5)