I have a dataset from the State Security Department in my county that has some problems.

I can't read the records at all from the file that is made available in CSV, bringing up only empty records. When I convert the file to XLSX it does get read.

I would like to know if there is any possible solution to the above problem.

The dataset is available at: here or here.

I tried the code below, but i only get nulls, except for the first row in the first column:

df = pd.read_csv('mensal_ss.csv', sep=';', names=cols, encoding='latin1')

image

Thank you!

CodePudding user response：

If you try with utf-16 as the encoding, it seems to work. However, note that the year rows complicates the parsing, so you may need some extra manipulation of the csv to circumvent that depending on what you want to do with the data

df = pd.read_csv('mensal_ss.csv', sep=';', encoding='utf-16')

CodePudding user response：

try to use 'utf-16-le':

import pandas as pd

df = pd.read_csv('mensal_ss.csv', sep=';', encoding='utf-16-le')
print(df.head())