How to fix selenium unable to find element when run on Heroku but is able to find the same element w-CodePudding

I am running a simple python selenium script on Heroku, Its build is successful but the script is unable to locate a specific element when run on the server but can locate the same element when run locally.

CODE-

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By

s=Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=s)
driver.maximize_window()
driver.get('https://www.google.com')
a=driver.find_element(By.XPATH, '//*[@id="SIvCob"]')
print(a.get_attribute('innerHTML'))

ERROR-

2022-06-29T09:07:47.865986 00:00 app[worker.1]:File "/app/tempTry.py", line 33, in <module>
2022-06-29T09:07:47.866124 00:00 app[worker.1]:     a=driver.find_element(By.XPATH, '//*[@id="SIvCob"]')
2022-06-29T09:07:47.866127 00:00 app[worker.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 857, in find_element
2022-06-29T09:07:47.866423 00:00 app[worker.1]:     return self.execute(Command.FIND_ELEMENT, {
2022-06-29T09:07:47.866433 00:00 app[worker.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
2022-06-29T09:07:47.866630 00:00 app[worker.1]:     self.error_handler.check_response(response)
2022-06-29T09:07:47.866642 00:00 app[worker.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
2022-06-29T09:07:47.866782 00:00 app[worker.1]:     raise exception_class(message, screen, stacktrace)
2022-06-29T09:07:47.866825 00:00 app[worker.1]: selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="SIvCob"]"}
2022-06-29T09:07:47.866825 00:00 app[worker.1]:   (Session info: headless chrome=103.0.5060.53)
2022-06-29T09:07:47.866826 00:00 app[worker.1]: Stacktrace:
2022-06-29T09:07:47.866827 00:00 app[worker.1]: #0 0x55701fad7b13 <unknown>
2022-06-29T09:07:47.866827 00:00 app[worker.1]: #1 0x55701f8de688 <unknown>
2022-06-29T09:07:47.866827 00:00 app[worker.1]: #2 0x55701f915cc7 <unknown>
2022-06-29T09:07:47.866827 00:00 app[worker.1]: #3 0x55701f915e91 <unknown>
2022-06-29T09:07:47.866828 00:00 app[worker.1]: #4 0x55701f948e34 <unknown>
2022-06-29T09:07:47.866828 00:00 app[worker.1]: #5 0x55701f9338dd <unknown>
2022-06-29T09:07:47.866828 00:00 app[worker.1]: #6 0x55701f946b94 <unknown>
2022-06-29T09:07:47.866829 00:00 app[worker.1]: #7 0x55701f9337a3 <unknown>
2022-06-29T09:07:47.866829 00:00 app[worker.1]: #8 0x55701f9090ea <unknown>
2022-06-29T09:07:47.866829 00:00 app[worker.1]: #9 0x55701f90a225 <unknown>
2022-06-29T09:07:47.866830 00:00 app[worker.1]: #10 0x55701fb1f2dd <unknown>
2022-06-29T09:07:47.866830 00:00 app[worker.1]: #11 0x55701fb232c7 <unknown>
2022-06-29T09:07:47.866830 00:00 app[worker.1]: #12 0x55701fb0922e <unknown>
2022-06-29T09:07:47.866831 00:00 app[worker.1]: #13 0x55701fb240a8 <unknown>
2022-06-29T09:07:47.866831 00:00 app[worker.1]: #14 0x55701fafdbc0 <unknown>
2022-06-29T09:07:47.866831 00:00 app[worker.1]: #15 0x55701fb406c8 <unknown>
2022-06-29T09:07:47.866832 00:00 app[worker.1]: #16 0x55701fb40848 <unknown>
2022-06-29T09:07:47.866832 00:00 app[worker.1]: #17 0x55701fb5ac0d <unknown>
2022-06-29T09:07:47.866832 00:00 app[worker.1]: #18 0x7fae2c9be609 <unknown>
2022-06-29T09:07:47.866834 00:00 app[worker.1]:

CodePudding user response：

Google gives you page depending on your location. For example here is my local browser (on my computer):

And here is how it looks like on my private VPS server located in USA (there is no element with xpath=//*[@id="SIvCob"]):

For more details you can paste print(driver.page_source) after the line driver.get('https://www.google.com') and save the output to the .html file. Copy it from the server and open locally. You'll see that element doesn't exist on that page.