Getting the HTML element using Selenium WebDriver-CodePudding

I'm trying to get price of a product on amazon using Selenium:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By

url = \
"https://www.amazon.in/Celevida-Kesar-Elaichi-Flavor-Metal/dp/B081WJ6536/ref=sr_1_5?crid=3NRZERQ8H4T8L&keywords=dr reddys celevida&qid=1672124472&sprefix=,aps,5801&sr=8-5"

services = Service(r"C:\Users\Deepak Shetter\chromedriver_win32\chromedriver.exe")
driver = webdriver.Chrome(service=services)
driver.get(url)
price = driver.find_element(By.CLASS_NAME, "a-offscreen")
print("price is " price.text)

As you can see in this image the html for the price is of . But when I run my code on pycharm it return None. How can I get the price string? (btw I checked it using Beautiful soup and it worked fine)

Edit : This time I used another url : https://www.amazon.in/Avvatar-Alpha-Choco-Latte-Shaker/dp/B08S3TNGYK/?_encoding=UTF8&pd_rd_w=ofFKu&content-id=amzn1.sym.1f592895-6b7a-4b03-9d72-1a40ea8fbeca&pf_rd_p=1f592895-6b7a-4b03-9d72-1a40ea8fbeca&pf_rd_r=PT3Y6GWJ7YHADW09VKNK&pd_rd_wg=lBWZa&pd_rd_r=0a44c278-bcfa-49c2-806b-cf8eb292038a&ref_=pd_gw_ci_mcx_mr_hp_atf_m

In this case it has 2 price elements one with the and another one with calss="a-price-whole".

my code :

price = driver.find_element(By.CLASS_NAME, "a-price-whole")

this time return value is 1,580.

CodePudding user response：

Using Safari, the following code works:

from selenium import webdriver
from selenium.webdriver.common.by import By

with webdriver.Safari() as driver:
    driver.get('https://www.amazon.in/Celevida-Kesar-Elaichi-Flavor-Metal/dp/B081WJ6536/ref=sr_1_5?crid=3NRZERQ8H4T8L&keywords=dr reddys celevida&qid=1672124472&sprefix=,aps,5801&sr=8-5')
    price = driver.find_element(By.CLASS_NAME, "a-offscreen")
    print(price.text)

Which gives this output:

₹566.00

Therefore it appears that your use of the ChromeDriver may be flawed

CodePudding user response：

you can use execute_script function and run js and get the value

result = driver.execute_script(
    "return document.querySelector('.a-offscreen').textContent"
)
print(result)