Python: convert column containing string to column containing json dictionary-CodePudding

I have a Dataframe with columns that look like this:

df=pd.DataFrame()
df['symbol'] = ['A','B','C']
df['json_list'] = ['[{name:S&P500, perc:25, ticker:SPY, weight:1}]',
          '[{name:S&P500, perc:25, ticker:SPY, weight:0.5}, {name:NASDAQ, perc:26, ticker:NASDAQ, weight:0.5}]',
          '[{name:S&P500, perc:25, ticker:SPY, weight:1}]']
df['date'] = ['2022-01-01', '2022-01-02', '2022-01-02']
df:
    symbol  json_list                                         date
0   A       [{name:S&P500, perc:25, ticker:SPY, weight:1}]    2022-01-01
1   B       [{name:S&P500, perc:25, ticker:SPY, weight:0.5... 2022-01-02
2   C       [{name:S&P500, perc:25, ticker:SPY, weight:1}]    2022-01-02

The values in the json_list column are of <class 'str'>.

How can I convert the json_list column items to dicts so I can access them based on key:value pairs?

Thank you in advance.

CodePudding user response：

UPDATED to reflect the fact that the json strings in the question are not singleton lists, but can contain multiple dict-like elements.

This will put a list of dict object in a new column of your dataframe:

def foo(x):
    src = x['json_list']
    rawList = src[1:-1].split('{')[1:]
    rawDictList = [x.split('}')[0] for x in rawList]
    dictList = [dict(x.strip().split(':') for x in y.split(',')) for y in rawDictList]
    for dct in dictList:
        for k in dct:
            try:
                dct[k] = int(dct[k])
            except ValueError:
                try:
                    dct[k] = float(dct[k])
                except ValueError:
                    pass
    return dictList
df['list_of_dict_object'] = df.apply(foo, axis = 1)

Original answer:

This will put a dict in a new column of your dataframe that should give you something close to what you want, except for numeric typing:

df['dict_object'] = df.apply(lambda x: dict(x.strip().split(':') for x in x['json_list'][2:-2].split(',')), axis = 1)

To get float or int where string values are convertible, you can do this:

def foo(x):
    d = dict(x.strip().split(':') for x in x['json_list'][2:-2].split(','))
    for k in d:
        try:
            d[k] = int(d[k])
        except ValueError:
            try:
                d[k] = float(d[k])
            except ValueError:
                pass
    return d
df['dict_object'] = df.apply(foo, axis = 1)

CodePudding user response：

The "json" is almost valid yaml. If you add a space after the colons, you can parse it using pyyaml.

df.json_list.apply(lambda data: yaml.safe_load(data.replace(':', ': ')))