Home > Back-end >  Retriving data from div and header classes using pyhton urllib.request and bs4
Retriving data from div and header classes using pyhton urllib.request and bs4

Time:06-06

I am trying to create a python program which is capable of getting the objects title and its cost from 'https://coinmarketcap.com/' website. in this image i have my initial code. I keep getting an error that says:

AttributeError: 'NoneType' object has no attribute 'text'

however, both the 'priceHeading' & 'priceValue' class have values of 'bitcoin price' and '29,000' respectively. How do I get those values?

here's my initial code:

# import libraries
import urllib.request as ur
from bs4 import BeautifulSoup

quote_page = 'https://coinmarketcap.com/currencies/bitcoin/'

page = ur.urlopen(quote_page)

soup = BeautifulSoup(page, 'html.parser')

name_box = soup.find('h1', attrs={'class': 'priceHeading'})

name = name_box.text.strip() 
print (name)

price_box = soup.find('div', attrs={'class':'priceValue'})
price = price_box.text
print (price)

CodePudding user response:

You probably received compressed response from the server which urllib.request cannot handle. Instead of urllib.request use requests module:

import requests
from bs4 import BeautifulSoup

quote_page = "https://coinmarketcap.com/currencies/bitcoin/"

soup = BeautifulSoup(requests.get(quote_page).content, "html.parser")
name_box = soup.find("h1", attrs={"class": "priceHeading"})

name = name_box.text.strip()
print(name)

price_box = soup.find("div", attrs={"class": "priceValue"})
price = price_box.text
print(price)

Prints:

Bitcoin Price (BTC)
$29,919.86
  • Related