Use Python Selenium to extract span text-CodePudding

Hi i'm new at selenium and webscraping and i need some help. i try to scrape one site and i need and i dont know how to get span class.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

PATCH = "/Users/bobo/Downloads/chromedriver"
driver = webdriver.Chrome(PATCH)



driver.get("https://neonet.pl")
print(driver.title)

search = driver.find_element_by_class_name("inputCss-input__label-263")
search.send_keys(Keys.RETURN)

time.sleep(5)

i try to extract this span

<span >Szukaj produktu</span>

CodePudding user response：

I can see that you are trying to search something in the search bar. First I recommend you to use the xpath instead of the class name, here is a simple technique to get the xpath of every element on a webpage:

right-click/ inspect element/ select the mouse in a box element on the upper left/ click on the element on the webpage/ it will directly show you the corresponding html/ then right click on the selected html/ copy options and then xpath.

Here is a code example that searches an element on the webpage, I also included the 'Webdriver-wait' option because sometimes the code runs to fast and can't find the next element so this function make the code wait till the element is visible:

from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome(executable_path="/Users/bobo/Downloads/chromedriver")

driver.get("https://neonet.pl") #loading page
wait = WebDriverWait(driver, 20) #defining webdriver wait

search_word = 'iphone\n' # \n is going to act as an enter key

wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="root"]/main/div[1]/div[4]/div/div/div[2]/div/button[1]'))).click() #clicking on cookies popup
wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="root"]/main/header/div[2]/button'))).click() #clicking on search button
wait.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="root"]/aside[2]/section/form/label/input'))).send_keys(search_word) #searching on input button
print('done!')
sleep(10)

Hope this helped you!

CodePudding user response：

wait=WebDriverWait(driver,10)
driver.get('https://neonet.pl')
elem=wait.until(EC.visibility_of_element_located((By.XPATH, "//span[contains(@class,'inputCss-input')]"))).text
print(elem)

To output the value of the search bar you use .text on the Selenium Webelement.

Imports:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC