Home > Software engineering >  Getting none in beautiful soup web-scraping
Getting none in beautiful soup web-scraping

Time:10-16

My intend

I want to scrap commits of user from github using beautiful soup with python.

My issue

Getting none as result of my script.

My code

from bs4 import BeautifulSoup
import requests

html = requests.get('https://github.com/pnp/cli-microsoft365').text
soup = BeautifulSoup(html, 'html.parser')
commits = soup.find('strong', class_='repo-content-pjax-container > div > div.gutter-condensed.gutter-lg.flex-column.flex-md-row.d-flex > div.flex-shrink-0.col-12.col-md-9.mb-4.mb-md-0 > div.Box.mb-3 > div.Box-header.position-relative > div > div:nth-child(4) > ul > li > a > span > strong')
print(commits)

CodePudding user response:

What happens?

Your using a "wild mix" in your find() and this will not lead to the element you are expected to find, thats why you get a None

How to fix?

Use the css selector to chain the parts you are looking for, in this case it will pick the <svg> in front of commits and its next <span> element that contains the <strong>:

soup.select_one('svg.octicon.octicon-history   span strong').text 

Output (in moment of my request)

1,664
  • Related