Selenium WebDriver/ChromeDriver Error During WebScraping with Selenium-CodePudding

I am trying to scrape a table from the following website - https://www.cmegroup.com/markets/energy/crude-oil/brent-crude-oil.settlements.html#tradeDate=03/31/2022

Tried the following block of code, but it seems to return an error message.

I am new to web scraping and any help on the below issue would be greatly appreciated.

Code:

import time
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import pandas as pd
from bs4 import BeautifulSoup

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get(url)
time.sleep(5)

soup = BeautifulSoup(driver.page_source, 'lxml')

price = pd.read_html(str(soup))[0]
print(price)

Output:

===== WebDriver manager ======
Could not get version for google-chrome.Is google-chrome installed?
Get LATEST chromedriver version for None google-chrome
There is no [linux64] chromedriver for browser None in cache
Trying to download new driver from https://chromedriver.storage.googleapis.com/100.0.4896.60/chromedriver_linux64.zip
Driver has been saved in cache [/root/.wdm/drivers/chromedriver/linux64/100.0.4896.60]
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:1: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  """Entry point for launching an IPython kernel.
---------------------------------------------------------------------------
WebDriverException                        Traceback (most recent call last)
<ipython-input-4-c026f0c15647> in <module>()
----> 1 driver = webdriver.Chrome(ChromeDriverManager().install())
3 frames
/usr/local/lib/python3.7/dist-packages/selenium/webdriver/common/service.py in assert_process_still_running(self)
    110             raise WebDriverException(
    111                 'Service %s unexpectedly exited. Status code was: %s'
--> 112                 % (self.path, return_code)
    113             )
    114 
WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: -6

CodePudding user response：

You must install the browser driver for your browser as explained here https://www.selenium.dev/documentation/webdriver/getting_started/install_drivers/