Python version: 3.10; Selenium Webdriver: Firefox; IDE: PyCharm 2021.3.2 (CE); OS: Fedora 35 VM

I am writing a python selenium script to scrape data from a website. I would like to navigate through a website, find an element & print it. I am able to do this when the element is present. My problem is that sometimes the element is not present & I get an exception. If the element is not present when I use this code:

 RemainDeductible = (WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))

the script produces the following exception:

 Traceback (most recent call last):
  
 File "...PythonSeleniumScript.py", line 152, in <module>
    RemainDue = (WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))

 File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/support/wait.py", line 89, in until
    raise TimeoutException(message, screen, stacktrace)

 selenium.common.exceptions.TimeoutException: Message: 
Stacktrace:
WebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:183:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.jsm:395:5
element.find/</<@chrome://remote/content/marionette/element.js:300:16

If the Timeout Exception occurs I would like to catch the exception & set the "Remain Due" variable to the string "BLANK". I still get the following Timeout Exception if I use Try in my code as follows:

try:
    RemainDue = (WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))
except TimeoutException:
    RemainDue = "BLANK"

I still get the Timeout Exception. I thought my code would catch the exception. Why is it not catching it???

 Traceback (most recent call last):
  
 File "...PythonSeleniumScript.py", line 155, in <module>
    RemainDue = (WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))

  File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/support/wait.py", line 89, in until

    raise TimeoutException(message, screen, stacktrace)

 selenium.common.exceptions.TimeoutException: Message: 
Stacktrace:
WebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:183:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.jsm:395:5
element.find/</<@chrome://remote/content/marionette/element.js:300:16

The relevant HTML for the element I am trying to locate is:

<input id="b8-b36-Input_RemainAmtYr1"
 data-
input="" disabled="" type="text"
style="margin-top: 5px;" value="$10.50">
event

Of course the code does work when the element is present. My issue is that the code crashes with the Timeout Exception when the element is not present. When the Timeout Exception occurs how do I catch the Timeout Exception & set the "Remain Due" variable to the string "BLANK"?

CodePudding user response：

I would suggest several things here:

1. instead of

except TimeoutException:

Try using

except:

I understand that it's preferable to catch a specific exception type rather any exception, but I will still advice doing that here.
2) I would suggest using visibility_of_element_located instead of presence_of_element_located since presence_of_element_located will wait for presence of such element on the page only while it may still not be completely rendered. While visibility_of_element_located will wait for much more mature element state, when it is visible.
So instead of

try:
    RemainDue = (WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))
except TimeoutException:
    RemainDue = "BLANK"

I would suggest you using

try:
    RemainDue = (WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))
except:
    RemainDue = "BLANK"

Also make sure the id value of b8-b36-Input_RemainAmtYr1 is fixed, not changing.