I am trying to import the world population figure from website 'https://countrymeters.info/en', but unable to import the same by using Python

Thank & much appreciated for help! Below screenshot of my code

CodePudding user response：

the main page fetches it first but https://countrymeters.info/en/World#Population_clock doesn't. I got this to work:

res = requests.get("https://countrymeters.info/en/World#Population_clock").text
res = res.split('"cp1">')[1]
res = res.split("<")[0]
print(res)

CodePudding user response：

The reason that this doesn't work is because beautiful soup fetches the html (this describes the content of a webpage; text, links and images) which yes is displayed on the screen but can be modified by js (this is a program that can make the page do something like show a popup) and in this example the html start's with a loading text and then the js replaces that loading text with the data and since beautiful soup just fetches html you get the loading text originally shown instead of the data. I don't have a solution since I don't use beautiful soup to extract webpage content but hopefully this helps.