Selenium's element returns text too slow-CodePudding

Chrome version: 105.0.5195.102
Selenium == 4.4.3
Python == 3.9.12

In a certain page, 'element.text' takes ~0.x seconds which is unbearable. I suppose 'element.text' should return literally just a text from a cached page so couldn't understand it takes so long time. How can I make it faster?

Here are similar QNAs but I need to solve the problem just with Selenium.

Another question: Why every 'element.text' takes different times?

For example,

import chromedriver_autoinstaller
import time
from selenium import webdriver

chromedriver_autoinstaller.install(cwd=True)

options = webdriver.ChromeOptions()
options.add_argument("--headless")
options.add_argument('--no-sandbox')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--disable-dev-shm-usage')
options.add_experimental_option("excludeSwitches", ["enable-logging"])

wd = webdriver.Chrome(options=options)
wd.get("https://www.bbc.com/")

t0 = time.time()
e = wd.find_element(By.CSS_SELECTOR, "#page > section.module.module--header > h2")
print(time.time()-t0)

for i in range(10):
    t0 = time.time()
    txt = e.text
    print(time.time()-t0)
    # This prints different result for every loop.

wd.quit()

CodePudding user response：

Selenium can be a bit slow because it does NOT work directly with Chrome. The communication is made via channel which is the Chrome web driver.

If you wish to work with a faster and better plugin for Automation try using PlayWright.

Another thing you can try is to find your element directly, and not using a long CSS or long Xpath expression. The longer your expression will be -> the longer it will take to find it and its text

CodePudding user response：

I see the following output for your code:

0.0139617919921875
0.01196908950805664
0.003988742828369141
0.004987955093383789
0.003988027572631836
0.0039899349212646484
0.003989219665527344
0.004987955093383789
0.003987789154052734
0.003989696502685547
0.0049860477447509766

The first 2 times are about 12-14 milliseconds while the others are about 4 milliseconds.
The first action

wd.find_element(By.CSS_SELECTOR, "#page > section.module.module--header > h2")

is polling the DOM until it finds there element matching the given locator.
While the txt = e.text line uses already existing reference to the element on the page, so it does not perform any polling / searching, just access an element on the page via the existing reference (pointer) that's why it takes significantly less time.
Why the second time is long as the first I don't sure I know.
I run this test several times, got different outputs but mainly the picture was the - same.