Home > Software design >  Importing a list of dictionaries from a CSV file in python
Importing a list of dictionaries from a CSV file in python

Time:12-29

I have playlists that are lists of dictionaries, that look aprox like this: [{'url':url1,'title':title1}, {'url':url2,'title':title2}] Of course, the playlist has a name, name1 What I want to do is in one function, to save it to a CSV file. My wish is that the whole dictionary can just be one item in the CSV file, instead of dividing up the dictionary over the columns. Then I want to have another function to import the playlists. I litteraly spent hours to do this, and I am in desperation because I don't manage.

I show what I tried: To save the playist:

def SavePlaylist():
    name = SearchBox.get()   "§"
    if len(playlist) >1 and name != "":
        newrow = [name, playlist]
        with open(os.path.join(sys.path[0], "AntiTubePlaylists.txt"), "a") as f_object:
            writer_object = writer(f_object)
            writer_object.writerow(newrow)
            f_object.close()

To import the playlist:

with open(os.path.join(sys.path[0], "AntiTubePlaylists.txt"), "r") as csv_file:
    csv_reader = csv.reader(csv_file, quotechar='"', delimiter='§')
    for row in csv_reader:
        playlistt = row[1][2:][:-1]
        print(playlistt[0])
        playlistmenu.add_command(label=row[0], command=(lambda playlistt = playlistt:  ImportPlaylist(playlist)))

What it does is, it creates a list that has every single character of my list. (i.e. the print command prints just "["

CodePudding user response:

Eval solves the problem:

with open(os.path.join(sys.path[0], "AntiTubePlaylists.txt"), "r") as csv_file:
    csv_reader = csv.reader(csv_file, quotechar='"', delimiter='§')
    for row in csv_reader:
        playlistt = row[1][2:][:-1]
        playlistt = eval(playlistt)
        print(playlistt[0])
        playlistmenu.add_command(label=row[0], command=(lambda playlistt = playlistt:  ImportPlaylist(playlist)))

CodePudding user response:

do you want a csv having 2 cols (url, title)? I would personally use pandas: create a dataframe from your list of dicts and then save to csv

for example

import pandas as pd

name1 = [{'url':'url1','title':'title1'}, 
         {'url':'url2','title':'title2'},
         {'url':'url3','title':'title3'},
         {'url':'url4','title':'title4'}]

df = pd.DataFrame(name1)

df.to_csv('PATH/name1.csv')

CodePudding user response:

Though using eval may work, it is a big security risk! You should only use it for testing purposes, and only on files of which you are certain that nobody untrusted has tampered with. The reason for this is that eval actually runs everything in the file as code, so if someone injected some evil code into the csv file you're reading, you would (unknowingly) execute it when you load the file!

Indeed, saving an object using JSON is much easier. Specifically json.dump() and json.load() (https://docs.python.org/3/library/json.html#basic-usage) should be what you want.

An alternative is using the pickle library, which is made for storing Python objects, but as the page says, it is less secure, so you're probably better off sticking to JSON. Added benefit is that JSON is human-readable, whereas pickle will output some garbage:

>>> import json
>>> import pickle
>>> name1 = [{'url':'url1','title':'title1'}, {'url':'url2','title':'title2'}]
>>> json.dumps(name1)
'[{"url": "url1", "title": "title1"}, {"url": "url2", "title": "title2"}]'
>>> pickle.dumps(name1)
b'\x80\x04\x95?\x00\x00\x00\x00\x00\x00\x00]\x94(}\x94(\x8c\x03url\x94\x8c\x04url1\x94\x8c\x05title\x94\x8c\x06title1\x94u}\x94(h\x02\x8c\x04url2\x94h\x04\x8c\x06title2\x94ue.'

Note both json.dump and pickle.dump have a version that dumps to string rather than to file, namely json.dumps and pickle.dumps, which may be convenient.

CodePudding user response:

To use JSON, the file needs to be in the proper format. So I came up with the following solution, mind the str(s).replace; JSON needs double quotes for the items.

with open(os.path.join(sys.path[0], "AntiTubePlaylists.txt"), "r") as csv_file:
    csv_reader = csv.reader(csv_file,  delimiter='§')
    for row in csv_reader:
        s = str(row[1][2:][:-1])
        s = str(s).replace("'",'"')
        playlist_import = json.loads(s)
        print(playlist_import[0])
        playlistmenu.add_command(label=row[0], command=(lambda playlistt = playlist_import:  ImportPlaylist(playlistt)))
  • Related