I have some data that is contained in a list of tuples in Python, as shown below.

data = [('Test-1', 1, 0.203194), ('Test-1', 2, 0.0143804), ('Test-1', 3, 0.0769853), ('Test-2', 1, 0.00173769), ('Test-3', 1, 0.00842112), ('Test-3', 2, 0.128969), ('Test-4', 1, 0.0481806)]

Each tuple contains a value for test_name, session number, and percentile score (In that exact order). I need to reshape this data into a list of dictionaries, where each unique test_name is grouped like so:

[
    {
        "test_name": "Test-1",
        "session": [
            {"submission": 1, "percentile": 0.203194},
            {"submission": 2, "percentile": 0.0143804},
            {"submission": 3, "percentile": 0.0769853}
        ]
    },
    {
        "test_name": "Test-2",
        "session": [
            {"submission": 1, "percentile": 0.0}
        ]
    },
    {
        "test_name": "Test-3",
        "session": [
            {"submission": 1, "percentile": 0.0},
            {"submission": 2, "percentile": 0.0}
        ]
    },
    {
        "test_name": "Test-4",
        "session": [
            {"submission": 1, "percentile": 0.0}
        ]
    }
]

How could I do this in Python?

CodePudding user response：

Here I assume that the same test_name in data is continuous, so that I can write it in one line:

>>> from itertools import groupby
>>> from operator import itemgetter
>>> [{'test_name': test_name,
...   'session': [{'submission': submission, 'percentile': percentile}
...               for _, submission, percentile in group]}
...  for test_name, group in groupby(data, key=itemgetter(0))]
[{'test_name': 'Test-1',
  'session': [{'submission': 1, 'percentile': 0.203194},
              {'submission': 2, 'percentile': 0.0143804},
              {'submission': 3, 'percentile': 0.0769853}]},
 {'test_name': 'Test-2',
  'session': [{'submission': 1, 'percentile': 0.00173769}]},
 {'test_name': 'Test-3',
  'session': [{'submission': 1, 'percentile': 0.00842112},
              {'submission': 2, 'percentile': 0.128969}]},
 {'test_name': 'Test-4',
  'session': [{'submission': 1, 'percentile': 0.0481806}]}]

If the assumption does not hold:

>>> mp = {}
>>> for test_name, submission, percentile in data:
...    mp.setdefault(test_name, []).append({'submission': submission, 'percentile': percentile})
...
>>> [{'test_name': test_name, 'session': session}
...  for test_name, session in mp.items()]
[{'test_name': 'Test-1',
  'session': [{'submission': 1, 'percentile': 0.203194},
              {'submission': 2, 'percentile': 0.0143804},
              {'submission': 3, 'percentile': 0.0769853}]},
 {'test_name': 'Test-2',
  'session': [{'submission': 1, 'percentile': 0.00173769}]},
 {'test_name': 'Test-3',
  'session': [{'submission': 1, 'percentile': 0.00842112},
              {'submission': 2, 'percentile': 0.128969}]},
 {'test_name': 'Test-4',
  'session': [{'submission': 1, 'percentile': 0.0481806}]}]

CodePudding user response：

Let's break the problem into steps

This is how i would do it using simple primitives (loops, lists, dictionaries)

1. For each tuple identify the "key"
2. Prepare a dictonary for a key that you haven't seen before
3. If you have seen the key before, update the dictionary
4. collect all the dicts into a list

intermediate_dict = {} # dictionaries of dictionaries

# 1. For each tuple identify the "key"
for tup in data:
    key, submission, percentile = tup
    if key in intermediate_dict:
        # 3. If you have seen the key before, update the dictionary
        intermediate_dict[key]["session"].append({"submission": submission, "percentile": percentile})

    else:
        # 2. Prepare a dictonary for a key that you haven't seen before
        intermediate_dict[key] = {"session": [{"submission": submission, "percentile": percentile}]}

Your intermediate dict would probably look like this

{'Test-1': {'session': [{'submission': 1, 'percentile': 0.203194},
   {'submission': 1, 'percentile': 0.203194},
   {'submission': 2, 'percentile': 0.0143804},
   {'submission': 3, 'percentile': 0.0769853}]},
 'Test-2': {'session': [{'submission': 1, 'percentile': 0.00173769}]},
 'Test-3': {'session': [{'submission': 1, 'percentile': 0.00842112},
   {'submission': 2, 'percentile': 0.128969}]},
 'Test-4': {'session': [{'submission': 1, 'percentile': 0.0481806}]}}

now the next step is to simply transform the dictionary to a list

return [{"test_name": key, "session": value["session"]} for key, value in intermediate_dict.items()]