Home > front end >  How to scrape text value in label tag which is inside div tag using selenium with python?
How to scrape text value in label tag which is inside div tag using selenium with python?

Time:06-09

I have tried multiple things present on stackoverflow but nothing worked out for me.

Here is the pattern of the HTML

<div >
<label> MacBook<lable>
<div>

<div >
<label> MacBook Air<lable>
<div>

<div >
<label> MacBook Air Pro<lable>
<div>

I want to extract out the model names present in <label> tag inside <div class= "single-model">

Here is my code:

from selenium import webdriver
import time
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service

Brand = ["Apple"]
model_name = []
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
driver.get("https://www.google.com")

search_url="https://northladder.com/en/ae/electronics/laptop/apple"
driver.get(search_url)

#Scroll to the end of the page
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(5)

modelname = driver.find_elements(by=By.CLASS_NAME, value="single-model")
for m in modelname:
    model = m.find_elements(by=By.TAG_NAME, value="label")
    print(model.text)

Here print(model.text) is giving me an error

AttributeError: 'list' object has no attribute 'text'

CodePudding user response:

Try This

modelname = driver.find_elements(By.XPATH, "//label")
for m in modelname:
    model= m.get_attribute('innerHTML')
    Model_name.append(model)
print(Model_name)

CodePudding user response:

find_elements gives you a list of elements not a single element (even if there's only 1 match found) so you would need to loop over the elements to get the text inside it

for m in modelname:
    models = m.find_elements(by=By.TAG_NAME, value="label")
    for modal in modals:
        print(model.text)

alternatively if you only want to find 1 label per loop you can use find_element instead of find_elements and then it will only give you 1

  • Related