Home > Net >  Trying to create Dataframe from lists of zip using Pandas. wanted data table result
Trying to create Dataframe from lists of zip using Pandas. wanted data table result

Time:02-11

I'm scraping website and come to the part where to put it in Dataframe. I tried to follow this answer but no expected output.

Here's my whole code

#driver chrome def
website = 'https://www.bitkub.com/fee/cryptocurrency'
path = r"C:\\Users\\USER\\Downloads\\chromedriver.exe"
options = Options()
options.add_argument("start-maximized")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.get(website)

#giving variable
coin_name = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//td[2]//span")))]
chain_name = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//td[3]//div")))]
withdrawal_fees = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//td[4]//div")))]
#print(coin_name)
#print(chain_name)
#print(withdrawal_fees)

#for loop make list
for coin, chains, wdf in zip(coin_name, chain_name, withdrawal_fees):
    print("Coin name: {} Chain: {} Fee: {}".format(coin, chains, wdf))
#driver.quit

The output would be like this without being column yet. So I want it separate column by 'Coin name', 'Chain' and 'Fee'

Coin name: Civic(CVC) Chain: ETH (ERC20) Fee: 97.00000000 (CVC)
Coin name: (CVC) Chain: BSV Fee: 0.0004 (BSV)
Coin name: Bitcoin SV(BSV) Chain: ETH (ERC20) Fee: 0.00625000 (ETH)
Coin name: (BSV) Chain: BKC (KAP20) Fee: 0.01 (KUB)
Coin name: Ethereum(ETH) Chain: ETH (ERC20) Fee: 0.22000000 (COMP)
.
.
.

This is the input of variable I've scraped

print(coin_name)
['Civic(CVC)', '(CVC)', 'Bitcoin SV(BSV)', '(BSV)', 'Ethereum(ETH)', '(ETH)', 'Bitkub Coin(KUB)', '(KUB)', 'Compound(COMP)', '(COMP)', 'Curve DAO Token(CRV)', '(CRV)', 'Axie Infinity(AXS)', '(AXS)', 'The Sandbox(SAND)', '(SAND)', 'XRP(XRP)', '(XRP)', 'ChainLink Token(LINK)', '(LINK)', 'KyberNetwork(KNC)', '(KNC)', 'Stellar(XLM)', '(XLM)', 'Six(SIX)', '(SIX)', 'Everex(EVX)', '(EVX)', 'Alpha Finance Lab(ALPHA)', '(ALPHA)', 'Balancer(BAL)', '(BAL)', 'Litecoin(LTC)', '(LTC)', 'IOSToken(IOST)', '(IOST)', 'Cortex Coin(CTXC)', '(CTXC)', 'NEAR(NEAR)', '(NEAR)', 'Aave(AAVE)', '(AAVE)', 'Raiden Network Token(RDN)', '(RDN)', 'JFIN Coin(JFIN)', '(JFIN)', 'Basic Attention Token(BAT)', '(BAT)', 'Zilliqa(ZIL)', '(ZIL)', 'Polkadot(DOT)', '(DOT)', 'SushiSwap(SUSHI)', '(SUSHI)', 'Bitcoin Cash(BCH)', '(BCH)', 'Cardano(ADA)', '(ADA)', 'BNB Coin(BNB)', '(BNB)', 'Power Ledger(POW)', '(POW)', 'Secret Network(SCRT)', '(SCRT)', 'yearn.finance(YFI)', '(YFI)', 'Bitcoin(BTC)', '(BTC)', 'OmiseGO(OMG)', '(OMG)', 'Tether(USDT)', '(USDT)', 'Dogecoin(DOGE)', '(DOGE)', 'USD Coin(USDC)', '(USDC)', 'Maker(MKR)', '(MKR)', 'Enjin Coin(ENJ)', '(ENJ)', 'Kusama(KSM)', '(KSM)', 'Donnie Finance(DON)', '(DON)', 'Wancoin(WAN)', '(WAN)', 'StatusNetwork(SNT)', '(SNT)', '0x(ZRX)', '(ZRX)', 'Arcblock(ABT)', '(ABT)', 'Decentraland(MANA)', '(MANA)', 'Dai Stablecoin(DAI)', '(DAI)', 'Band Protocol(BAND)', '(BAND)', 'Golem Network Token(GLM)', '(GLM)', 'Uniswap(UNI)', '(UNI)', 'FTX Token(FTT)', '(FTT)', 'Ocean Protocol(OCEAN)', '(OCEAN)', 'Synthetix(SNX)', '(SNX)', 'Boba Token(BOBA)', '(BOBA)', 'Gala(GALA)', '(GALA)', 'GateToken(GT)', '(GT)', 'Songbird(SGB)', '(SGB)', 'GuildFi(GF)', '(GF)', 'Immutable X(IMX)', '(IMX)', 'Ethereum Name Service(ENS)', '(ENS)', 'dYdX(DYDX)', '(DYDX)', 'Illuvium(ILV)', '(ILV)', 'Aavegotchi(GHST)', '(GHST)', 'Chiliz(CHZ)', '(CHZ)', 'The Graph(GRT)', '(GRT)', 'Gods Unchained(GODS)', '(GODS)', 'Convex Finance(CVX)', '(CVX)', 'LUKSO(LYXE)', '(LYXE)', 'Crypto.com Coin(CRO)', '(CRO)', 'Yield Guild Game(YGG)', '(YGG)', 'Smooth Love Potion(SLP)', '(SLP)', 'Coin98(C98)', '(C98)', 'ShuttleOne(SZO)', '(SZO)', 'Popcoin(POP)', '(POP)', 'Experimental Finance(EXFI)', '(EXFI)', 'Solana(SOL)', '(SOL)', 'Avalanche(AVAX)', '(AVAX)', 'Polygon(MATIC)', '(MATIC)', 'Fantom(FTM)', '(FTM)', 'Terra(LUNA)', '(LUNA)']

print(chain_name)
['ETH (ERC20)', 'BSV', 'ETH (ERC20)', 'BKC (KAP20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'XRP', 'ETH (ERC20)', 'ETH (ERC20)', 'XLM', 'XLM', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'LTC', 'IOST', 'ETH (ERC20)', 'NEAR', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ZIL', 'DOT', 'ETH (ERC20)', 'BCH', 'ADA', 'BNB (BEP2)', 'ETH (ERC20)', 'SCRT', 'ETH (ERC20)', 'BTC', 'ETH (ERC20)', 'ETH (ERC20)', 'DOGE', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'KSM', 'IOST', 'WAN', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'BAND', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'SGB', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'ETH (ERC20)', 'BKC (KAP20)', 'SGB', 'SOL', 'AVAX_C (AVAX)', 'MATIC (Polygon)', 'FTM (Fantom)', 'LUNA (Terra)']

print(withdrawal_fees)
['97.00000000 (CVC)', '0.0004 (BSV)', '0.00625000 (ETH)', '0.01 (KUB)', '0.22000000 (COMP)', '9.32000000 (CRV)', '0.49000000 (AXS)', '6.95000000 (SAND)', '0.25 (XRP)', '1.75000000 (LINK)', '4 (KNC)', '0.02 (XLM)', '1 (SIX)', '114.00000000 (EVX)', '72.00000000 (ALPHA)', '2.14000000 (BAL)', '0.001 (LTC)', '1 (IOST)', '0.1 (CTXC)', '0.01 (NEAR)', '0.28000000 (AAVE)', '55.00000000 (RDN)', '5.75000000 (JFIN)', '34.00000000 (BAT)', '0.2 (ZIL)', '0.1 (DOT)', '6.80000000 (SUSHI)', '0.001 (BCH)', '1 (ADA)', '0.00100000 (BNB)', '54.00000000 (POW)', '0.1 (SCRT)', '0.00120000 (YFI)', '0.0005 (BTC)', '5.99000000 (OMG)', '35.00000000 (USDT)', '5 (DOGE)', '35.00000000 (USDC)', '0.01400000 (MKR)', '15.00000000 (ENJ)', '0.01 (KSM)', '0.00840000 (DON)', '0.1 (WAN)', '514.00000000 (SNT)', '45.00000000 (ZRX)', '35 (ABT)', '9.82000000 (MANA)', '15 (DAI)', '0.01 (BAND)', '79.00000000 (GLM)', '2.64000000 (UNI)', '0.68000000 (FTT)', '39 (OCEAN)', '7.05 (SNX)', '10.00000000 (BOBA)', '90.00000000 (GALA)', '9.5 (GT)', '1 (SGB)', '10 (GF)', '11.00000000 (IMX)', '1.54000000 (ENS)', '11.00000000 (DYDX)', '0.04400000 (ILV)', '13 (GHST)', '105 (CHZ)', '48 (GRT)', '12 (GODS)', '1.8 (CVX)', '1.5 (LYXE)', '31 (CRO)', '5.42 (YGG)', '519 (SLP)', '11 (C98)', '170 (SZO)', '40 (POP)', '0.1 (EXFI)', '0.01 (SOL)', '0.01 (AVAX)', '0.1 (MATIC)', '0.01 (FTM)', '0.02 (LUNA)']

CodePudding user response:

Some how coin_name is twice as long as your other lists. Once you fix that you can do this:

pd.DataFrame({'coin_name': coin_name[0:81], 'chain_name': chain_name, 'withdrawal_fees':withdrawal_fees})

          coin_name       chain_name    withdrawal_fees
0        Civic(CVC)      ETH (ERC20)  97.00000000 (CVC)
1             (CVC)              BSV       0.0004 (BSV)
2   Bitcoin SV(BSV)      ETH (ERC20)   0.00625000 (ETH)
3             (BSV)      BKC (KAP20)         0.01 (KUB)
4     Ethereum(ETH)      ETH (ERC20)  0.22000000 (COMP)
..              ...              ...                ...
76       Maker(MKR)              SOL         0.01 (SOL)
77            (MKR)    AVAX_C (AVAX)        0.01 (AVAX)
78  Enjin Coin(ENJ)  MATIC (Polygon)        0.1 (MATIC)
79            (ENJ)     FTM (Fantom)         0.01 (FTM)
80      Kusama(KSM)     LUNA (Terra)        0.02 (LUNA)

[81 rows x 3 columns]
  • Related