Grabbing data from table does not produce any result-CodePudding

I am trying to make the script work to grab a row of data from a table using what I have search in the internet. This script produce no output and no errors. Any help will do. Thak you.

import requests
from bs4 import BeautifulSoup

headers = {"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:92.0) Gecko/20100101 Firefox/92.0"}
url = "https://bscscan.com/token/generic-tokentxns2?m=normal&contractAddress=0x0D0b63b32595957ae58D4dD60aa5409E79A5Aa96&a=0x81e0ef68e103ee65002d3cf766240ed1c070334d&sid=87aaaf5bf5aad80385b57f5f3fa2aa3a&p=1"

s = requests.Session()
r = s.get(url,headers=headers, timeout=5)
soupblockdetails = BeautifulSoup(r.content, 'html.parser')

for row in soupblockdetails.select("tr:has(td)")[:3]:
    item1 = row.find_all("td")[0].text[0:].strip()
    item2 = row.find_all("td")[1].text[0:].strip()
    item3 = row.find_all("td")[2].text[0:].strip()
    item4 = row.find_all("td")[1].a.get('href').split('a=')[-1]
    print ("{:<2} {:<43}   {:>25} {}".format(item1, item2, item3, item4))

Current Output:

no display

Intended Output:

0xb07b68f72f0b58e8cfb8c8e896736f49b13775ebda25301475d24554a601ff97   115 days 12 hrs ago   Yooshiba Inu: Deployer   KIPS: Locked Wallet  1,870.82

CodePudding user response：

What happens?

Main issue here the table is embedded in an iframe thats src is generated dynamically by JavaScript

How to fix?

One option is to use Selenium navigating to rendered iframe and grab the table:

...
driver.get(url)
driver.switch_to.frame('tokentxnsiframe')

soupblockdetails = BeautifulSoup(driver.page_source, 'html.parser')

for row in soupblockdetails.select("tr:has(td)")[:3]:
    print(f"{' '.join(row.stripped_strings)}\t{row.a.get('href').split('/')[-1]}")
...

Example

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from bs4 import BeautifulSoup

url = 'https://bscscan.com/token/0x0D0b63b32595957ae58D4dD60aa5409E79A5Aa96?a=0x81e0ef68e103ee65002d3cf766240ed1c070334d'

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get(url)
driver.switch_to.frame('tokentxnsiframe')

soupblockdetails = BeautifulSoup(driver.page_source, 'html.parser')

for row in soupblockdetails.select("tr:has(td)")[:3]:
    print(f"{' '.join(row.stripped_strings)}\t{row.a.get('href').split('/')[-1]}")

Output

0xb07b68f72f0b58e8cfb8c8e896736f49b13775ebda25301475d24554a601ff97 0x129fa7e8 2021-12-09 17:23:58 115 days 14 hrs ago Yooshiba Inu: Deployer IN KIPS: Locked Wallet 1,870.828693386970691791    0xb07b68f72f0b58e8cfb8c8e896736f49b13775ebda25301475d24554a601ff97