I know that this question was asked many times on stackoverflow. I tried different solutions but did not get it work. Here is a simple MWE to automate the search on Youtube. Any body familiar with this can help explain the raison ?
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
options = Options()
options.add_argument('--disable-extensions')
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--remote-debugging-port=9515')
options.add_argument('--disable-setuid-sandbox')
options.add_argument("--start-maximized")
driver = webdriver.Chrome(service=Service("/usr/bin/chromedriver"), options=options)
url = "https://www.youtube.com/"
driver.get(url)
search_area = driver.find_element(By.XPATH, '//*[@id="search"]')
driver.implicitly_wait(10)
search_area.send_keys('Lionel Messi', Keys.ENTER)
print(search_area.text)
CodePudding user response:
There are 2 problems with your code:
- You should use unique locator.
The locator you are using here matching 5 elements on that page, so Selenium returns you the first element on the page matching the locator you passing here while you need the second match.
This locator will work better//input[@id="search"]
- You need to add a delay.
Selenium returns yousearch_area
element at the moment that element is found on the page while it is still may be not ready to be interacted with.
The best way to overcome this issue is to use Explicit Waits.
So, this code should work better:
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
options = Options()
options.add_argument('--disable-extensions')
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--remote-debugging-port=9515')
options.add_argument('--disable-setuid-sandbox')
options.add_argument("--start-maximized")
driver = webdriver.Chrome(service=Service("/usr/bin/chromedriver"), options=options)
wait = WebDriverWait(driver, 20)
url = "https://www.youtube.com/"
driver.get(url)
search_area = wait.until(EC.visibility_of_element_located((By.XPATH, "//input[@id='search']")))
search_area.send_keys('Lionel Messi', Keys.ENTER)
print(search_area.text)
P.S.
driver.implicitly_wait(10)
is NOT a delay command, this will not put a delay of 10 seconds on the line where you put it. This only defines the timeout for Selenium to way for element presence. Also, you put if AFTER locating the element, so this could not affect the previously located element.