Python3: Merging multiple dictionaries into a new dictionary-CodePudding

I apologize if this is a duplicate. I'm sure it is, I think I just don't know how to frame the question I'm trying to ask to be able to search for the correct answers.

I have a function that returns a dictionary per line, like so:

{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '1m', 'scenario': 1, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '3m', 'scenario': 2, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '5m', 'scenario': 1, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '15m', 'scenario': 1, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '30m', 'scenario': 2, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '1h', 'scenario': 2, 'continuity': 1}
...
{'symbol_name': 'TRBUSDT', 'timeframe_name': '1m', 'scenario': -2, 'continuity': 0}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '3m', 'scenario': 2, 'continuity': 1}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '5m', 'scenario': 1, 'continuity': -1}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '15m', 'scenario': 1, 'continuity': -1}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '30m', 'scenario': 2, 'continuity': 1}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '1h', 'scenario': 2, 'continuity': 1}

I want to create a new dictionary line that merges the data for the lines that have a common symbol_name, like so:

{'symbol_name': 'AUDIOUSDT', '1m_scenario': 1, '1m_continuity': 1, '3m_scenario': 2, '3m_continuity': 1, '5m_scenario': 1, '5m_continuity': 1, '15m_scenario': 1, '15m_continuity': 1, '30m_scenario': 2, '3m_continuity': 1, '1h_scenario': 2, '1h_continuity': 1 }
...
{'symbol_name': 'TRBUSDT', '1m_scenario': -2, '1m_continuity': 0, '3m_scenario': 2, '3m_continuity': 1, '5m_scenario': 1, '5m_continuity': -1, '15m_scenario': 1, '15m_continuity': -1, '30m_scenario': 2, '3m_continuity': 1, '1h_scenario': 2, '1h_continuity': 1 }

I'm having a hard time figuring out how to do this. I've tried to break this down into small bite-size chunks so I can get the hang of it, starting with just getting each timeframe into a new dictionary based on symbols_name:

data = {}
for item in query:
    symbol = item['symbol_name']
    timeframe = item['timeframe_name']    
    data[symbol] = {}
    data[symbol][timeframe] = {}

print(data)

# returns:
{'AUDIOUSDT': {'1h': {}}, ... 'TRBUSDT': {'1h': {}}}

In the above example, I thought I would see the other timeframe keys in the new dictionary line I'm trying there other than just 1h, because each time the loop runs, timeframe_name is different, but I just see the last timeframe_name.

I tried a different variation, but it just got me back to where I had started:

for item in query:
    symbol = item['symbol_name']
    timeframe = item['timeframe_name']
    data = {}
    data[symbol] = {}
    data[symbol][timeframe] = {}
    print(data)

# returns:
{'AUDIOUSDT': {'1m': {}}}
{'AUDIOUSDT': {'3m': {}}}
{'AUDIOUSDT': {'5m': {}}}
{'AUDIOUSDT': {'15m': {}}}
{'AUDIOUSDT': {'30m': {}}}
{'AUDIOUSDT': {'1h': {}}}
...
{'TRBUSDT': {'1m': {}}}
{'TRBUSDT': {'3m': {}}}
{'TRBUSDT': {'5m': {}}}
{'TRBUSDT': {'15m': {}}}
{'TRBUSDT': {'30m': {}}}
{'TRBUSDT': {'1h': {}}}

Any pointers?

CodePudding user response：

Assuming you have a list of dicts as shown above...

You can do this with collections.defaultdict to group the dicts by symbol_name, and create the expected keys, and update the dict with them.

Try this -

from collections import defaultdict

l= [{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '1m', 'scenario': 1, 'continuity': 1},
    {'symbol_name': 'AUDIOUSDT', 'timeframe_name': '3m', 'scenario': 2, 'continuity': 1},
    {'symbol_name': 'AUDIOUSDT', 'timeframe_name': '5m', 'scenario': 1, 'continuity': 1},
    {'symbol_name': 'AUDIOUSDT', 'timeframe_name': '15m', 'scenario': 1, 'continuity': 1},
    {'symbol_name': 'AUDIOUSDT', 'timeframe_name': '30m', 'scenario': 2, 'continuity': 1},
    {'symbol_name': 'AUDIOUSDT', 'timeframe_name': '1h', 'scenario': 2, 'continuity': 1},
    {'symbol_name': 'TRBUSDT', 'timeframe_name': '1m', 'scenario': -2, 'continuity': 0},
    {'symbol_name': 'TRBUSDT', 'timeframe_name': '3m', 'scenario': 2, 'continuity': 1},
    {'symbol_name': 'TRBUSDT', 'timeframe_name': '5m', 'scenario': 1, 'continuity': -1},
    {'symbol_name': 'TRBUSDT', 'timeframe_name': '15m', 'scenario': 1, 'continuity': -1},
    {'symbol_name': 'TRBUSDT', 'timeframe_name': '30m', 'scenario': 2, 'continuity': 1},
    {'symbol_name': 'TRBUSDT', 'timeframe_name': '1h', 'scenario': 2, 'continuity': 1}]


d = defaultdict(dict)

for i in l:
    k = {'symbol_name':i['symbol_name'],
         i['timeframe_name'] '_scenario':i['scenario'], 
         i['timeframe_name'] '_continuity':i['continuity']}
    d[i['symbol_name']].update(k)
    
output = list(d.values())
print(output)

[{'symbol_name': 'AUDIOUSDT', '1m_scenario': 1, '1m_continuity': 1, '3m_scenario': 2, '3m_continuity': 1, '5m_scenario': 1, '5m_continuity': 1, '15m_scenario': 1, '15m_continuity': 1, '30m_scenario': 2, '30m_continuity': 1, '1h_scenario': 2, '1h_continuity': 1}, 
 {'symbol_name': 'TRBUSDT', '1m_scenario': -2, '1m_continuity': 0, '3m_scenario': 2, '3m_continuity': 1, '5m_scenario': 1, '5m_continuity': -1, '15m_scenario': 1, '15m_continuity': -1, '30m_scenario': 2, '30m_continuity': 1, '1h_scenario': 2, '1h_continuity': 1}]

CodePudding user response：

You can use setdefault and update methods of dict:

data = {}

for item in query:
    symbol = item['symbol_name']
    timeframe = item['timeframe_name']
    d = data.setdefault(symbol, {})
    d.update({'symbol_name': symbol,
              f'{timeframe}_scenario': item['scenario'],
              f'{timeframe}_continuity': item['continuity']})

>>> list(data.values())
[{'symbol_name': 'AUDIOUSDT',
  '1m_scenario': 1,
  '1m_continuity': 1,
  '3m_scenario': 2,
  '3m_continuity': 1,
  '5m_scenario': 1,
  '5m_continuity': 1,
  '15m_scenario': 1,
  '15m_continuity': 1,
  '30m_scenario': 2,
  '30m_continuity': 1,
  '1h_scenario': 2,
  '1h_continuity': 1},
 {'symbol_name': 'TRBUSDT',
  '1m_scenario': -2,
  '1m_continuity': 0,
  '3m_scenario': 2,
  '3m_continuity': 1,
  '5m_scenario': 1,
  '5m_continuity': -1,
  '15m_scenario': 1,
  '15m_continuity': -1,
  '30m_scenario': 2,
  '30m_continuity': 1,
  '1h_scenario': 2,
  '1h_continuity': 1}]

Old answer (to avoid duplicate answer with @AkshaySehgal)

Use defaultdict from collections module

from collections import defaultdict

data = collections.defaultdict(dict)

for item in query:
    symbol = item['symbol_name']
    timeframe = item['timeframe_name']
    data[symbol].update({'symbol_name': symbol,
                      f'{timeframe}_scenario': item['scenario'],
                      f'{timeframe}_continuity': item['continuity']})

Output:

>>> list(data.values())
[{'symbol_name': 'AUDIOUSDT',
  '1m_scenario': 1,
  '1m_continuity': 1,
  '3m_scenario': 2,
  '3m_continuity': 1,
  '5m_scenario': 1,
  '5m_continuity': 1,
  '15m_scenario': 1,
  '15m_continuity': 1,
  '30m_scenario': 2,
  '30m_continuity': 1,
  '1h_scenario': 2,
  '1h_continuity': 1},
 {'symbol_name': 'TRBUSDT',
  '1m_scenario': -2,
  '1m_continuity': 0,
  '3m_scenario': 2,
  '3m_continuity': 1,
  '5m_scenario': 1,
  '5m_continuity': -1,
  '15m_scenario': 1,
  '15m_continuity': -1,
  '30m_scenario': 2,
  '30m_continuity': 1,
  '1h_scenario': 2,
  '1h_continuity': 1}]

CodePudding user response：

Here is my suggestion:

query = [{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '1m', 'scenario': 1, 'continuity': 1},
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '3m', 'scenario': 2, 'continuity': 1},
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '5m', 'scenario': 1, 'continuity': 1},
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '15m', 'scenario': 1, 'continuity': 1},
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '30m', 'scenario': 2, 'continuity': 1},
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '1h', 'scenario': 2, 'continuity': 1},
{'symbol_name': 'TRBUSDT', 'timeframe_name': '1m', 'scenario': -2, 'continuity': 0},
{'symbol_name': 'TRBUSDT', 'timeframe_name': '3m', 'scenario': 2, 'continuity': 1},
{'symbol_name': 'TRBUSDT', 'timeframe_name': '5m', 'scenario': 1, 'continuity': -1},
{'symbol_name': 'TRBUSDT', 'timeframe_name': '15m', 'scenario': 1, 'continuity': -1},
{'symbol_name': 'TRBUSDT', 'timeframe_name': '30m', 'scenario': 2, 'continuity': 1},
{'symbol_name': 'TRBUSDT', 'timeframe_name': '1h', 'scenario': 2, 'continuity': 1}]

res = {}

for i in query:
    res[i['symbol_name']] = {'symbol_name':i['symbol_name']}

for i in query:
    x = i['timeframe_name']
    scen_key = x   '_'   'scenario'
    con_key = x   '_'   'continuity'
    if scen_key in res[i['symbol_name']]:
        res[i['symbol_name']][scen_key]  = i['scenario']
    else:
        res[i['symbol_name']][scen_key] = i['scenario']
    if con_key in res[i['symbol_name']]:
        res[i['symbol_name']][con_key]  = i['continuity']
    else:
        res[i['symbol_name']][con_key] = i['continuity']

print(res)

Output:

{'AUDIOUSDT': {'symbol_name': 'AUDIOUSDT', '1m_scenario': 1, '1m_continuity': 1, '3m_scenario': 2, '3m_continuity': 1, '5m_scenario': 1, '5m_continuity': 1, '15m_scenario': 1, '15m_continuity': 1, '30m_scenario': 2, '30m_continuity': 1, '1h_scenario': 2, '1h_continuity': 1}, 'TRBUSDT': {'symbol_name': 'TRBUSDT', '1m_scenario': -2, '1m_continuity': 0, '3m_scenario': 2, '3m_continuity': 1, '5m_scenario': 1, '5m_continuity': -1, '15m_scenario': 1, '15m_continuity': -1, '30m_scenario': 2, '30m_continuity': 1, '1h_scenario': 2, '1h_continuity': 1}}

if you want result as list, you can get it with res.values()