Selenium Web Scraping-CodePudding

I would like my web scraper to go to a web site and press the log in button, then put random credentials and submit, here is the html code, please help

<div >
        <div >
            <div >
                <h1>
                    <a href="/" style="text-decoration: none">Quotes to Scrape</a>
                </h1>
            </div>
            <div >
                <p>
                
                    <a href="/login">Login</a>
                
                </p>
            </div>
        </div>

Here is the code I have by far

# Start driver
driver_path = "chromedriver"
driver = webdriver.Chrome(driver_path)
# Navigate to the website
driver.get('http://quotes.toscrape.com/')
driver.maximize_window()

driver.find_element('//bento/orange[contains(@Class,"small")]').click()

To press login button and put random credentials in

CodePudding user response：

Use WebDriverWait() and wait for element to be clickable. User following xpath options

driver.get("http://quotes.toscrape.com/")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//a[text()='Login']"))).click()
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[@id='username']"))).send_keys("testuser")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[@id='password']"))).send_keys("testuser")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//input[@value='Login']"))).click()

Import below libraries

from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

CodePudding user response：

To click on the Login link, fill in Username and Password and click on Login you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following locator strategies:

driver.get("https://quotes.toscrape.com/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href='/login']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#username"))).send_keys("Orkhan")
driver.find_element(By.CSS_SELECTOR, "input#password").send_keys("Karimov")
driver.find_element(By.CSS_SELECTOR, "input[value='Login']").click()