Scrapping HTML table from with beautifulsoup-CodePudding

I have been trying to do some web scrapping on this tables website (

CodePudding user response：

You also can grab dynamic table using pandas with selenium

import pandas as pd
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC


driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
table=driver.get('https://prosettings.net/cs-go-pro-settings-gear-list/')
driver.maximize_window()
table = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '(//table)[1]'))).get_attribute("outerHTML")
df1 = pd.read_html(table)[0]
print(df1)

Output:

         Team       Player  ...                            Headset     CFG
0        NaVi      Boombl4  ...           Logitech G Pro X Headset  CONFIG
1        NaVi   electronic  ...  Logitech G Pro X Wireless Headset  CONFIG
2        NaVi       s1mple  ...           Logitech G Pro X Headset  CONFIG
3        NaVi     Perfecto  ...           Logitech G Pro X Headset  CONFIG
4        NaVi          b1t  ...           Logitech G Pro X Headset  CONFIG
..        ...          ...  ...                                ...     ...
466  Streamer    seangares  ...              HyperX Cloud Alpha GE  CONFIG
467  Streamer    Lobanjica  ...         Sennheiser Momentum In-Ear  CONFIG
468  Streamer       FlipiN  ...             AIM Gaming Auriculares  CONFIG
469  Streamer  swag / brax  ...               Sennheiser GAME ZERO  CONFIG
470  Streamer       Znorux  ...                   Corsair Virtuoso  CONFIG

[471 rows x 22 columns]

webdriverManager