I'm using selenium to automate a task on a very dynamic website with pyhton. For this reason certain HTML elements of the current loaded page may or may not be present at the moment of the request from my code. How exactly the webdriver instance gets updated and receives the new data from the web page?
Is it constantly connected and receive the change in the HTML code instantly?
Or it first download a first verion of the page when driver.get() is called, and then updates it whenever a function such as .find_element_by_class_name() is called?
CodePudding user response：
Web page is loaded by driver.get().
But the driver doesn't "know" what elements are existing there. It just opens, loads the web page.
To access any element, to check element presence etc. you will need to do that particularly per each specific element / elements using commands like .find_element_by_class_name() with a specific element locator.
CodePudding user response：
Q. Is it constantly connected and receives the change in the HTML code instantly?
Ans. Well, for each Selenium command, there will be an HTTP request that will send to the browser driver and then to the server and the command will get, A HTTP response will be returned and the command/commands will get executed based on the response.
Now let's say in a situation you do,
driver.get()
Basically, this is a selenium command.
It would fire an HTTP request stating to launch the URL provided. Based on the response (200-Ok or anything else), you would either see the WebPage getting loaded or an error message.
Same way in Sequence all the Selenium commands will get executed.
It's obvious that we need locators to locate web elements in the UI.
Once we have them, we can tightly couple them with
driver.find_element_by_***
methods.
Things to be noted down here
You need to learn/understand :
- Implicit Waits.
- Explicit Waits.
- Fluent Waits.
Implicit Waits :
By implicitly waiting, WebDriver polls the DOM for a certain duration when trying to find any element. This can be useful when certain elements on the webpage are not available immediately and need some time to load.
Basically what it means is, whenever you use drive.find_element it gonna look for implicit waits if you have defined one.
If you've not defined one implicit wait, then the default value is 0.
Explicit wait
They allow your code to halt program execution, or freeze the thread, until the condition you pass it resolves. The condition is called with a certain frequency until the timeout of the wait is elapsed. This means that for as long as the condition returns a falsy value, it will keep trying and waiting.
FluentWait
FluentWait instance defines the maximum amount of time to wait for a condition, as well as the frequency with which to check the condition.
Reference Link