It is returning none while scraping the data-CodePudding

when i am using the findAll with corrrect class and tag it is returning None ans sometime empty list when i change the class..

from bs4 import BeautifulSoup as bs
import requests
s=requests.get('https://ineuron.ai/courses')
soup=bs(s.text,'html.parser')
t=soup.find('div',{"class":"AllCoursesMobile_course-filter-mobile__atbp2"})
print(t)

I want to extract all the data from particular class

CodePudding user response：

As mentioned content is rendered dynamically and requests only deals with static content - Take a look at the devloper tools of the browser and check the XHR request that is fired to get the course information:

import pandas as pd
pd.json_normalize(pd.read_json('https://ineuron.ai/_next/data/RCswhPX7tXYKTfUB0-2MP/courses.json')['pageProps']['initialState']['filter']['initCourses'])