My code:

from bs4 import BeautifulSoup

import requests

url = "https://www.binance.com/en/support/announcement/c-48?navId=48"

result = requests.get(url)

doc = BeautifulSoup(result.text, "lxml")

a = doc.find_all("div", class_="css-1q4wrpt")

print(a)

This is the screenshot of webpage with elements

I tried "html.parser", "lxml", and "html5lib". Had no luck. Also read through many posts about find_all return empty list but still confused why my element is right there in the page but I cant access it through bs4

Thank you for your patience.

CodePudding user response：

The data you see is embedded within the page in Json format. To parse it, you can use following example:

import json
import requests
from bs4 import BeautifulSoup

url = "https://www.binance.com/en/support/announcement/c-48?navId=48"
soup = BeautifulSoup(requests.get(url).content, "html.parser")

data = json.loads(soup.select_one("#__APP_DATA").text)

# uncomment to print all data:
# print(json.dumps(data, indent=4))

for catalog in data["routeProps"]["b723"]["catalogs"]:
    print(catalog["catalogName"])
    print("-" * 80)
    for i, a in enumerate(catalog["articles"], 1):
        print("{:<4} {}".format(i, a["title"]))
    print()

Prints:

New Cryptocurrency Listing
--------------------------------------------------------------------------------
1    Nexo (NEXO) Listing Postponed
2    Binance Will List MobileCoin (MOB) and Nexo (NEXO)
3    Binance Futures Will Launch USDT-Margined DAR Perpetual Contracts with Up to 25X Leverage
4    Introducing Project Galaxy (GAL) on Binance Launchpool! Farm GAL By Staking BNB, BUSD and CAKE
5    Binance Adds APE/BRL, AUDIO/TRY, BTTC/BUSD, GMT/AUD & MBL/BUSD Trading Pairs
6    Binance Futures Will Launch Coin-Margined GMT Perpetual Contracts with Up to 20X Leverage
7    Binance Adds KNC on Cross Margin and ASTR on Isolated Margin
8    Binance Adds ASTR/BTC, ASTR/ETH, BSW/TRY, FTT/ETH, FUN/BNB, PORTO/BUSD, STEEM/USDT & ZIL/EUR Trading Pairs
9    Binance Futures Will Launch USDT-Margined JASMY Perpetual Contracts and Coin-Margined XMR Perpetual Contracts
10   Binance Adds GMT/TRY and WAVES/TRY Trading Pairs
11   Binance to Delist UST/BTC Trading Pair and Add BTC/UST Trading Pair
12   Binance Futures Will Launch Coin-Margined KNC Perpetual Contracts and USDT-Margined FTT Perpetual Contracts
13   Binance Adds APE/ETH, BSW/ETH, FIL/TRY, FTM/EUR, GMT/GBP & ZIL/TRY Trading Pairs
14   Binance Futures Will Launch USDT-Margined WOO Perpetual Contracts with Up to 25X Leverage
15   Binance Adds KDA, MBL on Cross Margin and KDA, IMX on Isolated Margin, Delisting HEGIC/BUSD Isolated Margin Pair
16   Binance Futures Will Launch Coin-Margined ZIL Perpetual Contracts with Up to 20X Leverage
17   Binance Adds APE/AUD, AVAX/GBP, BIFI/USDT, GMT/EUR, IMX/BNB & RUNE/ETH Trading Pairs
18   Binance Futures Will Launch USDT-Margined BNX Perpetual Contracts with Up to 25X Leverage
19   Binance Futures Will Launch Coin-Margined VET Perpetual Contracts with Up to 20X Leverage
20   Binance Futures Will Launch Coin-Margined APE Perpetual Contracts with Up to 20X Leverage

Latest Binance News
--------------------------------------------------------------------------------
1    Binance Concludes 6 Promos: Register & Share ApeCoin (APE), Learn & Trade ApeCoin (APE), Gift Card Sharing Promo, AVAX Community Giveaway, and More
2    Dual Investment - Subscription Quota Increased by Up to 10X
3    Binance Will Support the TerraUSD (UST) Contract Swap
4    Binance Liquid Swap Adds ANT/USDT, ANT/BNB, GMT/ETH and GMT/BNB Liquidity Pools
5    Binance P2P Supports New Payment Methods for Southeast Asia and MENA
6    Binance Futures Relaunches Open Order Modification Feature
7    Binance NFT Launches Password-Free Purchase Limit and Pay PIN Features
8    Nine Tokens Supported for Binance Bridge 2.0
9    Earn Wednesday: High-Yield Offers Available Now! (2022-04-27)
10   New Dual Investment Products Launched: Earn Up to 148% APY (2022-04-27)
11   Binance Savings Offers Limited Welcome Bonus: Earn 20% APY with GMT!
12   PROM, LOOM and AERGO Added to the Convert & OTC Portal, With 75 New Trading Pairs Supported
13   Binance Completes Four Network Integrations And Opens Deposits
14   Binance Concludes 2 Promos: Layer 1 Tokens Trading Competition & BNX Trading Competition
15   Binance Will Support the Ellipsis (EPS) Token Migration, Redenomination and Ticker Change to EPX
16   New Assets Added to Flexible Savings on Binance (2022-04-22)
17   Binance Loans Adds Collateral Assets JASMY & RUNE
18   Changes of Services to Users in Russia
19   OM Staking Special: Enjoy Up to 106.57% APY and Share 300,000 OM in Rewards!
20   Updates on Tick Size for GMT USDⓈ-M Perpetual Futures Contracts

Latest Activities
--------------------------------------------------------------------------------
1    Get 5 BUSD & Stand a Chance to Win 1 Bitcoin with Binance Card!
2    Claim Exclusive POAP NFT Rewards During Alpine Esports Series Championship 2022 Powered by Binance
3    Learn & Earn with Esteban Ocon from BWT Alpine F1® Team: $10,000 in ALPINE and 100 ALPINE NFTs Up for Grabs!
4    Binance Lite Referral Activity: Refer Friends and Win Puzzle Pieces Worth Up to $500 Each from a Pool of $500,000 Equivalent in Tokens & 3,000 NFTs!
5    Learn & Trade ERN - $100,000 Rewards Up for Grabs!
6    Existing Users Bonus - Spend More with Binance Card to Receive 100 BUSD!
7    Game On! Learn & Trade BNX with $100,000 Up for Grabs!
8    Kickstart Your Binance Journey with Fiat & Auto-Invest: Share Up to 90,000 BUSD & Win 200 Binance NFT Mystery Boxes
9    BSW Trading Tournament - $100,000 in BSW to Be Shared!
10   Crypto Busy Bees: Collect the Tron Eco Hive Puzzles, Share the Pot of Honey with $350,000 in Rewards!
11   Binance P2P Philippines Exclusive: Trade Now to Share a Prize Pool Worth $4,000, Including a Trip to Boracay!
12   Harvest Up to 100 DOGE with Your First Gift Card!
13   Auto-Invest Star Portfolio Competition - $100,000 in BNB Up for Grabs!
14   WOO USDⓈ-M Futures Trading Tournament with 50,000 BUSD to Share!
15   MOve It with MBOX Learn & Trade: Share $100,000 and 1,000 MOMOverse NFTs!
16   Auto-Invest Giveaway - $30,000 in ONE to Be Shared
17   Collect Easter Eggs to Win a Basket of Eggs-quisite Rewards with $138,000 in Gift Cards and 1,000 Binance NFT Mystery Boxes!
18   Activate and Spend with Binance Card for Additional 5% Cashback This April!
19   Get 5 BUSD When Spending with Your Binance Card!
20   Register, Complete KYC, and Share Rewards of $45,000 in ApeCoin (APE)!

New Fiat Listings
--------------------------------------------------------------------------------
1    Buy PROM & More Directly Using Your Credit/Debit Card
2    Buy DNT & More Directly Using Your Credit/Debit Card, and MULTI & VTHO on Recurring Buy
3    Buy MULTI, RAMP & More Directly Using Your Credit/Debit Card
4    Buy BSW, GMT & More Directly Using Your Credit/Debit Card, and KDA & MBL on Recurring Buy
5    Buy APE, ATM, MBL, TROY & More Directly Using Your Credit/Debit Card
6    Buy BAR, BTCST, JUV, T & More Directly Using Your Credit/Debit Card, and AMP & KEY on Recurring Buy
7    Buy EPS, ONG & More Directly Using Your Credit/Debit Card, and ACA, ALPINE, WOO & More on Recurring Buy
8    Buy ALPINE, ACA, WOO & More Directly Using Your Credit/Debit Card
9    Buy API3, LOKA, RARE & More Directly Using Your Credit/Debit Card, and BTTC & FIDA on Recurring Buy
10   Buy CLV, CREAM, HIGH & More Directly Using Your Credit/Debit Card
11   Buy BETA, OCEAN, POND, SCRT & More Directly Using Your Credit/Debit Card, and COS & More on Recurring Buy
12   Buy COS, USDC & More Directly Using Your Credit/Debit Card
13   Buy CELR, ENS, FLUX, FTT, KP3R & POWR Directly Using Your Credit/Debit Card, and DUSK, FARM, JOE, MDT, OOKI & VOXEL on Recurring Buy
14   Buy DUSK, FARM, JOE, MDT, OOKI & VOXEL Directly Using Your Credit/Debit Card
15   Buy PEOPLE, SPELL & More Tokens Directly Using Your Credit/Debit Card, and EGLD & ROSE on Recurring Buy
16   Buy BICO, EGLD, GXS & ROSE Directly Using Your Credit/Debit Card
17   Buy CITY, SUSHI & More Tokens Directly Using Your Credit/Debit Card, and AKRO on Recurring Buy
18   Buy COCOS and 27 Other Newly Added Tokens Directly Using Your Credit/Debit Card
19   Buy SANTOS & More Tokens Directly Using Your Credit/Debit Card, and 1INCH, ARPA, DYDX, HBAR, ICP & QNT on Recurring Buy
20   Buy ARPA, CRV, FET, GALA, HNT, IOTX, ONE, SAND & TRIBE Directly Using Your Credit/Debit Card

Delisting
--------------------------------------------------------------------------------
1    Notice of Removal of Trading Pairs - 2022-04-28
2    Notice of Removal of Trading Pairs - 2022-04-21
3    Binance to Delist and Cease Redemption for XTZUP, XTZDOWN, LTCUP & LTCDOWN Leveraged Tokens (2022-04-07)
4    Binance Will Delist HEGIC, RENBTC and SUSD on 2022-04-11
5    Binance Futures Will Delist 1000BTTC and YFII USDT-Margined Contracts
6    Notice of Removal of Trading Pairs - 2022-04-01
7    Notice of Removal of Trading Pairs - 2022-03-18
8    Notice of Removal of Trading Pairs - 2022-03-10
9    Binance Will Delist BCD, CND, MTH, NCASH and YOYO on 2022-03-08
10   Notice of Removal of Trading Pairs - 2022-03-03
11   Notice of Removal of AXS/BIDR Trading Pair - 2022-02-25
12   Notice of Removal of Trading Pairs - 2022-02-24

Wallet Maintenance Updates
--------------------------------------------------------------------------------
1    Notice on the Suspension of Withdrawals on Solana (SOL) Network
2    Binance Will Support the Decred (DCR) Upgrade & Hard Fork
3    Binance Will Support the Ardor (ARDR) Upgrade & Hard Fork
4    Withdrawals for MOBOX NFT Avatars Suspended
5    Wallet Maintenance Second Extension for Multiple Networks - 2022-04-14
6    Wallet Maintenance Extension for Multiple Networks - 2022-04-14
7    Binance Will Support the New BitShares (NBS) Network Upgrade & Hard Fork
8    Binance Will Support the Cosmos (ATOM) Network Upgrade
9    Wallet Maintenance for Multiple Networks - 2022-04-14
10   Binance Will Support the BNB Beacon Chain (BNB) Upgrade & Hard Fork
11   Wallet Maintenance for BNB Beacon Chain (BEP2) - 2022-04-08
12   Deposits & Withdrawals on Ronin Network Fully Resumed
13   Binance Will Support the Flow (FLOW) Network Upgrade & Hard Fork
14   Deposits & Withdrawals on Tezos (XTZ) Network Temporarily Suspended
15   Binance Will Support the Horizen (ZEN) Upgrade & Hard Fork
16   Deposits & Withdrawals on Ronin Network Partially Resumed
17   Binance Will Support the Elrond (EGLD) Network Upgrade
18   Wallet Maintenance for Cardano Network (ADA)
19   Wallet Maintenance for Ethereum Network (ERC20) - 2022-02-16

API Updates
--------------------------------------------------------------------------------
1    Binance Futures Launches Time-Weighted Average Price Trading Algorithm on API
2    Binance Futures Launches Volume Participation Trading Algorithm on API
3    Updates on SAPI Rules (2021-11-04)
4    Updates on SAPI Transfer Types (2021-10-27)
5    Updates to the Time Range of Data Retrieval via Binance Margin SAPI
6    Updates to API Services
7    Updates to Binance Margin SAPI Create Margin Account
8    Notice on Adjusting the Request Weight of Rest API Endpoints
9    Binance Futures API Websocket User Data Stream Upgrade (2021-04-23)
10   Notice to API Users on Delisting WAPI Endpoints
11   Updates to the Sub-Account Transfer History Query in Binance SAPI 
12   API Functionality Now Enabled for Binance Vanilla Options
13   Update to the Binance USDT-Margined Futures Contract API
14   Updates to the Binance Savings API
15   API Functionality Now Enabled for Binance Liquid Swap
16   Updates to Scheduled Changes for the User Data Stream Websocket
17   Updates to the User Data Stream Websocket Payloads
18   API Functionality Now Enabled for Binance Isolated Margin Trading
19   New Binance Futures API Endpoint With 10x Performance 
20   Notice Regarding Updates to the Binance Futures API

Crypto Airdrop
--------------------------------------------------------------------------------
1    Binance Completes the Tenth Round of APENFT (NFT) Airdrop Distribution to TRON (TRX), BitTorrent OLD (BTTOLD), BitTorrent (BTTC), JUST (JST), and WINkLink (WIN) Holders
2    Binance Completes the Ninth Round of APENFT (NFT) Airdrop Distribution to TRON (TRX), BitTorrent OLD (BTTOLD), BitTorrent (BTTC), JUST (JST), and WINkLink (WIN) Holders
3    Binance Completes the Sixth Round of BEL Airdrop Distribution
4    Binance Completes the Eighth Round of APENFT (NFT) Airdrop Distribution to TRON (TRX), BitTorrent (BTT), JUST (JST), and WINkLink (WIN) Holders
5    Binance Completes the Seventh Round of APENFT (NFT) Airdrop Distribution to TRON (TRX), BitTorrent (BTT), JUST (JST), and WINkLink (WIN) Holders
6    Binance Will Support the Pundi X PURSE (PURSE) Airdrop Program for Pundi X (PUNDIX) Holders
7    Binance Completes the Songbird (SGB) Airdrop Distribution to XRP (XRP) Holders
8    Binance Will Support the Sologenic (SOLO) Airdrop Program for XRP (XRP) Holders
9    Binance Will Support the Songbird (SGB) Airdrop Program for XRP (XRP) Holders
10   Important Updates to the APENFT (NFT) Airdrop Program
11   Binance Will Support the Boba Token (BOBA) Airdrop Program for OMG Network (OMG) Holders
12   Binance Completes the Fourth Round of APENFT (NFT) Airdrop Distribution  to TRON (TRX), BitTorrent (BTT), and JUST (JST) Holders
13   Binance Will Support the Second Round of Donnie Finance (DON) Airdrop Program for IOST (IOST) Holders
14   Binance Will Support the APENFT (NFT) Airdrop Program for TRON (TRX), BitTorrent (BTT) and JUST (JST) Holders
15   Venus Reward Token (VRT) Distribution Is Now Complete
16   Binance Will Support the Venus Reward Token (VRT) Airdrop Program for Venus (XVS) Holders
17   Binance Will Support the Mettalex (MTLX) Airdrop Program for Fetch.ai (FET) Holders
18   Snapshot Taken for Symbol (XYM) Airdrop Program for NEM (XEM) Holders
19   Snapshot Schedule Update to the Symbol (XYM) Airdrop Program for NEM (XEM) Holders
20   Binance Will Support the Donnie Finance (DON) Airdrop Program for IOST (IOST) Holders

CodePudding user response：

Selenium and beautiful soup used.

from selenium import webdriver
from bs4 import BeautifulSoup as bs

url='https://www.binance.com/en/support/announcement/c-48?navId=48'
driver = webdriver.Chrome('C:/path/to/chromedriver.exe')
driver.get(url)
driver.implicitly_wait(10)

html = driver.page_source
soup = bs(html)

for tag in soup.find_all("div", class_="css-1q4wrpt"):
    print(tag.text)
driver.close()

Check this how-to-retrieve-the-values-of-dynamic-html-content-using-python