Date search and date output from the same class name when parsing-CodePudding

I'm trying to parse a site. It has the same named classes, while the number of such classes varies from page to page. I'm interested in the class that contains the date, which is written in the following pattern: 24 January 2020.

Code:

import pandas as pd
import numpy as np
import requests
from bs4 import BeautifulSoup
import re
from user_agent import generate_user_agent
import time

my_user_agent = generate_user_agent(os=('mac', 'linux'))

headers = {
    "Accept": "*/*",
    "user-agent": my_user_agent}

source = 'https://coronavirus-graph.ru/kanada'
req = requests.get(source, headers=headers)
html = req.content
soup = BeautifulSoup(html, 'lxml')
items_list = soup.find_all('div', class_='rr')

print(items_list)
[<div ><span>In Canada, ≈</span><b>11</b><span> people die out of </span><b>1000</b><span> infected ( 1.1%)</span></div>, <div ><b>99</b><span> a person in serious condition</span></div>, <div ><span>First person infected</span><b>24 January 2020</b></div>]

In this case, I'm interested in the last class and its value in <b>. That is the date. How can I get it?

CodePudding user response：

Here is one way of getting that data:

[...]
headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36'
}

source = 'https://coronavirus-graph.ru/kanada'
r = requests.get(source, headers=headers)
soup = BeautifulSoup(r.text, 'lxml')
items_list = soup.select('div[]')[-1].select_one('b')

print(items_list.text)

Result in terminal:

24 января 2020 г.

CodePudding user response：

Use css selectors and pseudo class last-child:

soup.select_one('.rr b:last-child').text

Example

from bs4 import BeautifulSoup

html = '''<div ><span>In Canada, ≈</span><b>11</b><span> people die out of </span><b>1000</b><span> infected ( 1.1%)</span></div>, <div ><b>99</b><span> a person in serious condition</span></div>, <div ><span>First person infected</span><b>24 January 2020</b></div>'''
soup = BeautifulSoup(html)

soup.select_one('.rr b:last-child').text