I am learning web scrapping with Selenium for Finance team project. The idea is:

1. Login to HR system
2. Search for Purchase Order Number
3. System display list of attachments
4. Download the attachments

Below are my codes:

# interaction with Google Chrome
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# specify chromedriver location
PATH = './chromedriver_win32/chromedriver.exe'

# open Google Chrome browser & visit Purchase Order page within HRIS
browser = webdriver.Chrome(PATH)
browser.get('https://purchase.sea.com/#/list')

< user input ID & password >

# user interface shows "My Purhcase Request" & "My Purchase Order" tabs
# click on the Purchase Order tab

try:
    po_tab = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.LINK_TEXT, "My Purchase Orders"))
    )
    po_tab.click()
except:
    print('HTML element not found!!')

# locate PO Number field and key in PO number
try:
    po_input_field = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "input-field"))
    )
    po_input_field.send_keys(<dummy PO num#>) ## any PO number
except:
    print("PO field not found!!")

# locate Search button and click search

try:
    search_button = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.LINK_TEXT, "Search"))
    )
    search_button.click()
except:
    print("Search button not found!!")

I stuck at the step # click on the Purchase Order tab and following steps. I can find the elements but I can see error after executing the py script. The most interesting part is....I can do it perfectly in Jupyter Notebook.

Python Script Execute Error

Here are the elements after inspection screens:

Purchase Orders tab

PO Number input field

Search button

CodePudding user response：

The first error in your log is HTML element not found!! You are performing a click before element is visible on DOM. Please try below possible solutions,

• Wait till element is visible then perform click operation.

EC.visibility_of_element_located((By.LINK_TEXT, "My Purchase Orders"));

• If still you are not able click with above code then wait for element to be clickable.

EC.element_to_be_clickable((By.LINK_TEXT, "My Purchase Orders"));

I would suggest to create reusable methods for actions like click() getText() etc.

CodePudding user response：

See you are using presence_of_element_located which is basically

""" An expectation for checking that an element is present on the DOM
of a page. This does not necessarily mean that the element is visible.
locator - used to find the element
returns the WebElement once it is located
"""

What I would suggest to you is to use element_to_be_clickable

""" An expectation for checking that an element is present on the DOM of a
page and visible. Visibility means that the element is not only displayed
but also has a height and width that is greater than 0.
locator - used to find the element
returns the WebElement once it is located and visible

so, in your code it'd be something like this :

WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.LINK_TEXT, "My Purchase Orders"))).click()

Also, we could try with different locators if this does not work, locators like CSS, XPATH. But I'll let you comment first if this works or not.