Cannot grab data inside a div ContentPlaceHolder1_contractCodeDiv in python-CodePudding

Good day. I am trying to grab some data from a url. Specifically, I wanted to grab what is inside div id="ContentPlaceHolder1_contractCodeDiv" and produce the output below. However, I am just getting an error. Thanks in advance.

import requests
from bs4 import BeautifulSoup

header = {"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:92.0) Gecko/20100101 Firefox/92.0"}
url = "https://bscscan.com/address/0x264ca80271e944c80ee81a3c8ed71acaa7d15b15#code"
url = requests.get(url, headers=header, timeout=5)
ca = BeautifulSoup(url.content, 'html.parser')

constatus = ca.find('h3', class_='h6 text-dark font-weight-bold mb-4').get_text().strip()
print (constatus)

conFlag = ca.find('div', class_='alert alert-warning text-dark mb-2').get_text()
print ("conFlag: ", conFlag)

contractCode = ca.find('div', class_='ContentPlaceHolder1_contractCodeDiv').get_text()
print (contractCode)

Wanted Output:

Warning:  This contract contains unverified libraries: IterableMapping
Status: Contract Source Code Verified (Exact Match)
Contract Name: ElonVSPutin
Compiler Version:  v0.6.12 commit.27d51765
Optimization Enabled:  No with 200 runs
Other Settings: default evmVersion, MIT license

Compiler specific version warnings:       #-- text from the exclaimation icon
The compiled contract might be susceptible to SignedImmutables (very low-severity),
ABIDecodeTwoDimensionalArrayMemory (very low-severity), EmptyByteArrayCopy (medium-severity),
DynamicArrayCleanup (medium-severity) Solidity Compiler Bugs.

Current Output:

Contract Source Code Verified (Exact Match)
Warning:  ×This contract contains unverified libraries: IterableMapping
Traceback (most recent call last):

CodePudding user response：

Just like to print the texts? Take a look at the example - If you like to get the information separately you can access them by css selector that check for a string, get the parent and extract the text:

contract = soup.select_one('div:-soup-contains-own("Contract Name:")').parent.get_text(' ', strip=True)
compilerVersion = soup.select_one('div:-soup-contains-own("Compiler Version")').parent.get_text(' ', strip=True)

Example

import requests
from bs4 import BeautifulSoup

header = {"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:92.0) Gecko/20100101 Firefox/92.0"}
url = "https://bscscan.com/address/0x264ca80271e944c80ee81a3c8ed71acaa7d15b15#code"
url = requests.get(url, headers=header, timeout=5)
soup = BeautifulSoup(url.content, 'html.parser')

info = '\n'.join([e.replace('\n\n',' ').strip('\n ×') for e in soup.select_one('#ContentPlaceHolder1_contractCodeDiv').text.split('\n\n\n\n') if e != ''])
warning = soup.select_one('#ContentPlaceHolder1_divWarningMsg').text.strip().replace(':','\n')

print(f'{info}\n{warning}')

Output

This contract contains unverified libraries: IterableMapping
Contract Source Code Verified (Exact Match)
Contract Name: ElonVSPutin
Compiler Version v0.6.12 commit.27d51765
Optimization Enabled: No with 200 runs
Other Settings: default evmVersion, MIT license

Compiler specific version warnings
The compiled contract might be susceptible to SignedImmutables (very low-severity), ABIDecodeTwoDimensionalArrayMemory (very low-severity), EmptyByteArrayCopy (medium-severity), DynamicArrayCleanup (medium-severity) Solidity Compiler Bugs.