noob I am following this python selenium webscraping tutorial /watch?v=lTypMlVBFM4 I don't know-CodePudding

I am trying to scrap this site https://www.youtube.com/@JohnWatsonRooney/videos for the video titles.

This short explination does not help with the problem of the code but I thought I would say it anyway. I am doing this so I can get my foot in the door to then scrap other sites for prices on stuff so I can make a program that gives me a link to the cheapest priced product from a list several sites as well as a ranking of all the other products and there names incase that wasn't the one I was looking for.

here is the terminal output.

Traceback (most recent call last):
File "C:\\Users\\PRO\\Documents\\dynamic_web_scraping_test.py", line 11, in \<module\>
videos = driver.find_element("name", "style-scope ytd-rich-grid-media")
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\webdriver.py", line 830, in find_element
return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})\["value"\]
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\webdriver.py", line 440, in execute
self.error_handler.check_response(response)
File "C:\\Users\\PRO\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\selenium\\webdriver\\remote\\errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: \[name="style-scope ytd-rich-grid-media"\]
Stacktrace:
RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:180:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:392:5
element.find/\</\<@chrome://remote/content/marionette/element.sys.mjs:275:16

C:\\Users\\PRO\\Documents\>

here is my code

import time
from selenium import webdriver
url1 = 'https://www.youtube.com/@JohnWatsonRooney/videos'
url2 = 'https://www.youtube.com/@MyCraftyDen/videos'
driver = webdriver.Firefox()
driver.get(url1)
time.sleep(10)
videos = driver.find_element("name", "style-scope ytd-rich-grid-media")
for video in videos:
titles = video.find_element_by_xpath('//\*\[@id="video-title-link"\]').text
print(title)

I don't know what to try I did my best looking up this problem before posting. I know waiting for the browser to load the page doesnt work hense the time.sleep(10) in my code.

CodePudding user response：

If you are looking to find the links of all the videos in the given page, you can use below code

import time
from selenium import webdriver
from selenium.webdriver.common.by import By

url1 = 'https://www.youtube.com/@JohnWatsonRooney/videos'
url2 = 'https://www.youtube.com/@MyCraftyDen/videos'
driver = webdriver.Firefox()
driver.get(url1)
time.sleep(10)
videos = driver.find_elements(By.ID, "video-title-link")
for video in videos:
    title = video.get_attribute("href")
    print(title)