Home > Software design >  How to convert HTML table to CSV? [Python]
How to convert HTML table to CSV? [Python]

Time:12-29

I have a fixed stracture of data array that comes in a table I saved the html with the table now im trying to take the table that is inside the html file and put it in a csv file to organize the data (with a code I saw here) and im keep getting an error

this is the code (taken from this site)

# Importing the required modules 
import os
import sys
import pandas as pd
from bs4 import BeautifulSoup

path = 'file location'   'file.html'

# empty list
data = []

# for getting the header from
# the HTML file
list_header = []
soup = BeautifulSoup(open(path),'html.parser')
header = soup.find_all("table")[0].find("tr")

for items in header:
    try:
        list_header.append(items.get_text())
    except:
        continue

# for getting the data 
HTML_data = soup.find_all("table")[0].find_all("tr")[1:]

for element in HTML_data:
    sub_data = []
    for sub_element in element:
        try:
            sub_data.append(sub_element.get_text())
        except:
            continue
    data.append(sub_data)

# Storing the data into Pandas
# DataFrame 
dataFrame = pd.DataFrame(data = data, columns = list_header)

# Converting Pandas DataFrame
# into CSV file
dataFrame.to_csv('Geeks.csv')

but get this error

File, line 38, in <module>
dataFrame = pd.DataFrame(data = data, columns = list_header)
File, line 509, in __init__
arrays, columns = to_arrays(data, columns, dtype=dtype)
File, line 524, in to_arrays
return _list_to_arrays(data, columns, coerce_float=coerce_float, dtype=dtype)
File, line 567, in _list_to_arrays
raise ValueError(e) from e
ValueError: 1 columns passed, passed data had 7 columns

what am I doing wrong ?

CodePudding user response:

The big thing I see that's missing in your sample code is that you're not iterating td elements inside every row element, maybe the for sub_element in element line does the cell iteration, but it's not clear. For yourself, and anyone else who needs to read your code (like, us

  • Related