Why can't I grab an element on a webpage with selenium in python? Tried by id, xpath, full xpat-CodePudding

I'm trying to write a script that will auto-fill an online form with python and selenium, but I'm having trouble grabbing an element from the html code.

This is my starting code. Everything works until I try to identify the element. The element's id changes for every new instance of the website, so I'm trying a workaround which I believe grabs the correct id for each instance:

from selenium import webdriver
from selenium.webdriver.common.by import By
import requests
web = webdriver.Chrome()
url = 'https://lincdoc.ou.edu/lincdoc/doc/run/ouathletics/OU_AdvisingForm2#ldTimeoutUri'
web.get(url)
html_text = requests.get(url).text
instance_id_index = html_text.index("<div id=")
instance_id = html_text[instance_id_index 9]   html_text[instance_id_index 10]   html_text[instance_id_index 11]   html_text[instance_id_index 12] 
student_id = "ID NUMBER"
student_id_box_id = instance_id   "q4"

I've tried the following, which all come back with a no such element exception:

(1) tried and failed to grab student id textbox element by id

web.find_element(By.ID, student_id_box_id).send_keys(student_id)

(2) tried and failed to grab student id textbox element by xpath

student_id_box_xpath = '//*[@id="'   instance_id   'q4"]'
web.find_element(By.XPATH, student_id_box_xpath).send_keys(student_id)

(3) tried and failed to grab student id textbox element by full xpath

student_id_box_xpath = '//*[@id="'   instance_id   'q4"]'
student_id_box_full_xpath = student_id_box_xpath   '/html/body/div/div[2]/div/div[2]/div[2]/div[3]/div/div[3]/table/tbody[2]/tr[1]/td[3]/div/input'
web.find_element(By.XPATH, student_id_box_full_xpath).send_keys(student_id)

Can anyone shed some light on this? I'm very very new to programming and python and it took awhile for me to get even this far. Thanks in advance!

CodePudding user response：

You can easily use an attribute = value css selector with $ ends with operator to specify the input element with id ending with q4

from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait 

web = webdriver.Chrome()
web.get('https://lincdoc.ou.edu/lincdoc/doc/run/ouathletics/OU_AdvisingForm2#ldTimeoutUri')
student_id_field = WebDriverWait(web, 5).until(EC.presence_of_element_located((By.CSS_SELECTOR, "input[id$=q4]")))
student_id_field.send_keys("some id goes here")

CodePudding user response：

the id of this page is dynamic, it is better to use table, row as anchor element, I wrote one sample to solve this issue, you can view the full code from here

sample as the following:

from clicknium import clicknium as cc, locator, ui

tab = cc.chrome.open("https://lincdoc.ou.edu/lincdoc/doc/run/ouathletics/OU_AdvisingForm2#ldTimeoutUri")
tab.find_element(locator.chrome.lincdoc.text_advising_unit).set_text("Athletics-OMS 256/(405) 325-8373")
tab.find_element(locator.chrome.lincdoc.text_advising_name).set_text("AJ [email protected]")

tab.find_element(locator.chrome.lincdoc.text_student_id).set_text("student1")
tab.find_element(locator.chrome.lincdoc.text_first_name).set_text("tom")
tab.find_element(locator.chrome.lincdoc.text_last_name).set_text("jack")
tab.find_element(locator.chrome.lincdoc.text_phone_number).set_text("12345")
tab.find_element(locator.chrome.lincdoc.text_ou_email).set_text("[email protected]")
tab.find_element(locator.chrome.lincdoc.text_student_id).set_text("Junior")

tab.find_element(locator.chrome.lincdoc.checkbox_topic, {'topic':'Academic Contract'}).set_checkbox()

print('done')