I am trying to scrap video details from a youtube channel using selenium but as a output i only get-CodePudding

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

driver = webdriver.Chrome()
driver.get('https://www.youtube.com/channel/UC9B_ywLK2Vne4mtmGgWZklg/videos')

videos = driver.find_elements(By.CLASS_NAME, 'style-scope ytd-grid-video-renderer')

for video in videos:
    title = driver.find_element(By.XPATH, './/*[@id="video-title"]').text
    views = driver.find_element(By.XPATH, './/*[@id="metadata-line"]/span[1]').text
    when = driver.find_element(By.XPATH, './/*[@id="metadata-line"]/span[2]').text

    print(title, views)

Above code was expected to scrap videos details from targeted youtube channel but it only returns only details of first video multiple times!

Here is the output: This is the output!

CodePudding user response：

 title = driver.find_element(By.XPATH, './/*[@id="video-title"]').text

Should be

 title = video.find_element(By.XPATH, './/*[@id="video-title"]').text

.// Is what you'll use when xpathing from another element.

CodePudding user response：

You can also use list comprehension and do the job with something like this

driver.get('https://www.youtube.com/channel/UC9B_ywLK2Vne4mtmGgWZklg/videos')

title = [x.text for x in driver.find_elements(By.CSS_SELECTOR, "#contents #items #video-title")]
views = [x.text for x in driver.find_elements(By.CSS_SELECTOR, "#contents #items #metadata-line span:first-child")]
when = [x.text for x in driver.find_elements(By.CSS_SELECTOR, "#contents #items #metadata-line span:nth-child(2)")]
for i in range(len(title)):
    print(f'{title[i]} - {views[i]} - {when[i]}')