How to print the hidden product names using Selenium and Python-CodePudding

Second ever post here. Still new to all this. Having trouble with a simple print call in a for loop not displaying any text, yet moving the cursor as if it did. If I check the length, it matches with how many spaces are moved in the print call (12)

from selenium import webdriver
from selenium.webdriver.common.by import By

browser= webdriver.Chrome()
browser.get('https://www.amazon.com/s/ref=nb_sb_noss?url=search-alias=computers-intl-ship&field-keywords=&ref=nb_sb_noss&crid=JKZZMSQ4Q71E&sprefix=,computers-intl-ship,114')
elem_list= browser.find_element(By.CSS_SELECTOR, "div.a-section.a-spacing-medium._octopus-search-result-card_style_apbSearchResultsContainer__bCqjb")
items = elem_list.find_elements(By.XPATH, '//span[@data-component-type="s-product-image"]')
for item in items:
    print(item.text)

if I run

print(len(items))

I get 12. However once I run the above mentioned for loop, I simply get 12 blank empty spaces. What am I doing wrong?

Update

I'm thinking the problem is in the data component type being an image. But after inspecting all the elements, what's best practice for displaying the list of items on the page that I am trying to do?

CodePudding user response：

The product names are contained within the alt attribute of the <img> elements.

Solution

To extract the product names using List Comprehension you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following locator strategies:

Using CSS_SELECTOR:

driver.get("https://www.amazon.com/s/ref=nb_sb_noss?url=search-alias=computers-intl-ship&field-keywords=&ref=nb_sb_noss&crid=JKZZMSQ4Q71E&sprefix=,computers-intl-ship,114")
print([my_elem.get_attribute("alt") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "span[data-component-type='s-product-image']>a>div>img")))])

Using XPATH:

driver.get("https://www.amazon.com/s/ref=nb_sb_noss?url=search-alias=computers-intl-ship&field-keywords=&ref=nb_sb_noss&crid=JKZZMSQ4Q71E&sprefix=,computers-intl-ship,114")
print([my_elem.get_attribute("alt") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//span[@data-component-type='s-product-image']/a/div/img")))])

Console Output:

['Apple Pencil (2nd Generation)', 'Sceptre 24" Professional Thin 75Hz 1080p LED Monitor 2x HDMI VGA Build-in Speakers, Machine Black (E248W-19203R Series)', 'Roku Streaming Stick 4K 2021 | Streaming Device 4K/HDR/Dolby Vision with Roku Voice Remote and TV Controls', 'Original HP 67XL Black High-yield Ink Cartridge | Works with HP DeskJet 1255, 2700, 4100 Series, HP ENVY 6000, 6400 Series...', 'Seagate Portable 2TB External Hard Drive Portable HDD – USB 3.0 for PC, Mac, PlayStation, & Xbox - 1-Year Rescue Service (...', 'Logitech MK270 Wireless Keyboard and Mouse Combo for Windows, 2.4 GHz Wireless, Compact Mouse, 8 Multimedia and Shortcut K...', 'Original HP 67 Black/Tri-color Ink Cartridges (2-pack) | Works with HP DeskJet 1255, 2700, 4100 Series, HP ENVY 6000, 6400...', 'HP 24mh FHD Monitor - Computer Monitor with 23.8-Inch IPS Display (1080p) - Built-In Speakers and VESA Mounting - Height/T...', 'iPhone Charger, TAKAGI Lightning Cable 3PACK 6FT Nylon Braided USB Charging Cable High Speed Data Sync Transfer Cord Compa...', 'Original HP 63XL Black High-yield Ink Cartridge | Works with HP DeskJet 1112, 2130, 3630 Series; HP ENVY 4510, 4520 Serie...', 'Roku Express 4K  2021 | Streaming Media Player HD/4K/HDR with Smooth Wireless Streaming and Roku Voice Remote with TV Cont...', 'Logitech C920x HD Pro Webcam, Full HD 1080p/30fps Video Calling, Clear Stereo Audio, HD Light Correction, Works with Skyp...']