Extract about into from hipages-CodePudding

I want to extract this info from

This is what my code looks like:

description = []
    try:
        description.append(
            soup.select('div', class_="sc-bdnxRM col__Col-sc-15n4ng3-0 icIRSI itbbRL").text)
        print(f"Description: {description}")
    except:
        description.append(" ")

But this does not work.

Can someone help me with this?

Thanks :)

CodePudding user response：

Desired tag's class name gets changed dynamically with js, and ends up being <div color="tint6" class="sc-dlnjwi btKpNj"> (check website page source). So grabbing that tag will give us desired output:

from bs4 import BeautifulSoup
import requests

res = requests.get('https://hipages.com.au/connect/tigerelectricalservices')

soup = BeautifulSoup(res.text, 'lxml')
parent = soup.find('div', class_='sc-dlnjwi btKpNj').select('div> div:nth-child(2)')

text = parent[0].get_text()

Note that select returns list containing given selectors, so we need to use indexing to get the actual item. Output would look like:

' Welcome to Tiger Electrical Services  Our electricians have extensive experience in the Electrical industry offering a wide range of services. With over 12 years of experience we can offer a wide range of services and understand that our customers expect honesty, integrity and reliability.\r Our business was initially established to provide electrical solutions to the outer suburbs of Berwick, Narre Warren, Beaconsfield, Pakenham and Cranbourne.\r .... '