How can I merge unequal lists of dictionaries?-CodePudding

I have been struggling for days to merge several lists of dictionaries into dictionaries of lists grouped by shared index value, which I can use to perform nutritional conversions.

To be clear, there are two main issues:

Converting lists of dictionaries into dictionaries of lists, as shown below.
Exporting/importing them from either CSV or JSON files, so that Python recognizes the values as lists and not strings.

Spacy/Prodigy produces the lists of dictionaries as follows:

[{'index': 0, 'category': 'AMT', 'item': '4.25', 'ids': ''}, {'index': 0, 'category': 'UMS', 'item': 'tablespoons', 'ids': ''}, {'index': 0, 'category': 'ING', 'item': 'olive oil', 'ids': '171413'}]    
        
[{'index': 1, 'category': 'AMT', 'item': '1', 'ids': ''}, {'index': 1, 'category': 'MOD', 'item': 'medium', 'ids': ''}, {'index': 1, 'category': 'ING', 'item': 'yellow onion', 'ids': '170008'}]    
    
[{'index': 2, 'category': 'AMT', 'item': '2', 'ids': ''}, {'index': 2, 'category': 'MOD', 'item': 'medium', 'ids': ''}, {'index': 2, 'category': 'ING', 'item': 'carrots', 'ids': '170393'}]

If I can get those IDs to 'remember' their index values, I should be able to finish the task at hand. The "ids" are USDA IDs linking each ingredient to its nutritional profile.

I would love to convert them into something like the bolded example here:

recipe_dict = {'ids': idslist, 'amount':amtlist, 'units':umslist, 'ingredients':inglist, 'modifiers': modlist}

I have succeeded in exporting these dictionaries of lists in the desired format to a CSV file, with less success exporting/importing them to JSON, the preferred format.

In the CSV file, they look like this:

Unfortunately, I have (again) struggled to get Python to recognize the values as lists. I have tried everything I know; but it splits the lists into individual characters.

If anyone has any (sincere) feedback, I would be eternally grateful. This is a passion project. I have worked on it for months. But this problem has become a real sticking point.

CodePudding user response：

you're just reading strings

my_data='[5]'
print(type(my_data))
>>> str
for x in my_data:
    print(x)
>>> "'",'[','5',']',"'"

use ast

import ast
my_data=ast.literal_eval('[5]')
print(type(my_data))
>>> list

for x in my_data:
    print(x)
>>> 5

CodePudding user response：

Quick code using dictionary and list comprehensions:

list_of_dicts = [{'index': 0, 'category': 'AMT', 'item': '4.25', 'ids': ''}, {'index': 0, 'category': 'UMS', 'item': 'tablespoons', 'ids': ''}, {'index': 0, 'category': 'ING', 'item': 'olive oil', 'ids': '171413'}]

dict_of_lists = { key: [ d[key] for d in list_of_dicts] for key in list_of_dicts[0].keys()} 

print(dict_of_lists)

output

{'index': [0, 0, 0],
 'category': ['AMT', 'UMS', 'ING'],
 'item': ['4.25', 'tablespoons', 'olive oil'],
 'ids': ['', '', '171413']}

Not sure if that was what you wanted, the question is a bit unclear