I have been trying to extract live data from worldometer.com(https://www.worldometers.info/), particularly the health section data. I was able to extract the title (example:'Communicable disease deaths today' but I cannot extract the live data(numbers). Can anyone please help me on this?

CodePudding user response：

The live data(numbers) is populated by JavaScript and you can grab it easily using automation tool something like selenium. Here is an example. Please just run the code.

Script:

import time
from bs4 import BeautifulSoup
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager

url = "https://www.worldometers.info/"

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
time.sleep(5)
driver.get(url)
time.sleep(5)


soup = BeautifulSoup(driver.page_source, 'lxml')

num  = soup.select_one('div#c49 > div > span.counter-number')
print(num.text)

 

Output:

2,134,658