couldn't find the element by soup.find-CodePudding

I couldn't find the value. the value location in html is (span 161 /span) Should I try xpath method to get theelement ? Or better suggetion ?

!pip install Selenium
from selenium import webdriver

# 要把chromedriver 放在同一層資料夾裡
browser = webdriver.Chrome(executable_path='./chromedriver.exe')

browser.get("https://shopee.tw/shop/10228173/search?page=0&sortBy=ctime")
source = browser.page_source

import time
import requests
from bs4 import BeautifulSoup

soup = BeautifulSoup(source)
browser.get("https://www.rakuten.com.tw/shop/watsons/product/956957/")
soup = BeautifulSoup(browser.page_source)
soup
products =[]
product = {}
#     product['網址'] = link

product['購買次數'] = soup.find('div',class_="b-container-child").span
products.append(product)
print(products)

CodePudding user response：

You can use headers to get full data from soup and also you have to select which element you have to find so from b-container-child we have to select 2nd index and from it we have to find last span tag and then it will return 161 as output.

import requests
from bs4 import BeautifulSoup
headers={"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36"}
res=requests.get("https://www.rakuten.com.tw/shop/watsons/product/956957/",headers=headers)

soup=BeautifulSoup(res.text,"html.parser")
all_data=soup.find_all("div",class_="b-container-child")[2]
main_data=all_data.find_all("span")[-1]
print(main_data.text)

Output: