Selenium xpath no result-CodePudding

I am trying to do some web scrawling through Selenium. However, when I run the code, it does not show the result.

Here is my code:

import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

from bs4 import BeautifulSoup
import time
import pandas as pd


driver = webdriver.Chrome()
url = 'https://vimeo.com/510879223'
driver.get(url)

#head > meta:nth-child(14)
#/html/head/meta[8]

title = driver.find_element(By.CSS_SELECTOR,"head > meta:nth-child(14)")
print (title.text)
description = driver.find_element(By.XPATH,"//meta[@property='og:description']").text
print (description)

Result:

Process finished with exit code 0

In this case, what should I add or delete? Is it happened because the site that I want to scrape does not support xpath scrape option?

If I do print (title), the result is:

<selenium.webdriver.remote.webelement.WebElement (session="6f182a4afb7c1173f1e74f1cd6a40d87", element="e10f1407-3a09-4f3e-96e4-19071cda7d8e")>

Feel like it has a result but I cannot check the result as text. In this case, what is the best way to fix it? Thank you!

CodePudding user response：

In your case I would recommend finding title by Xpath since the css selector you are trying to use is showing me the description tag. Notice that the text you are looking for is not stored as text on the page but rather in the content attribute. Using the .get_attribute() method should help.

title = driver.find_element(By.XPATH,"//meta[@property='og:title']").get_attribute('content')
print (title)
description = driver.find_element(By.XPATH,"//meta[@property='og:description']").get_attribute('content')
print (description)

CodePudding user response：

Hi sometimes errors may occur in xpath copying. There may be inconsistencies with the command you want to take action.You can try using commands.

find_element_by_css_selector
find_element_by_class_name

My example project may help you.

https://github.com/kaayaumutt/instagramBotApp/blob/main/instagramBotApp/instagramBotApp.py