nNumbers = [1,2,3]
baseVariables = ['a','b','c','d','e']

arr = np.empty(0)
    
    for i in nNumbers:
        x = np.empty(0)
        for v in baseVariables:
            x = np.append(x, y['result'][i][v])
            print(x)
        arr = np.concatenate((arr, x))

I have one Json input stored in y. need to filter some variables out of that json format. the above code works in that it gives me the output in an array, but it is only in a one dimensional array. I want the output in a two dimensional array like:

[['q','qr','qe','qw','etc']['','','','','']['','','','','']]

I have tried various different ways but am not able to figure it out. Any feedback on how to get it to the desired output format would be greatly appreciated.

CodePudding user response：

A correct basic Python way of making a nested list of strings:

In [57]: nNumbers = [1,2,3]
    ...: baseVariables = ['a','b','c','d','e']

In [58]: alist = []
    ...: for i in nNumbers:
    ...:     blist = []
    ...:     for v in baseVariables:
    ...:         blist.append(v str(i))
    ...:     alist.append(blist)
    ...:     

In [59]: alist
Out[59]: 
[['a1', 'b1', 'c1', 'd1', 'e1'],
 ['a2', 'b2', 'c2', 'd2', 'e2'],
 ['a3', 'b3', 'c3', 'd3', 'e3']]

That can be turned into an array if necessary - though numpy doesn't provide much added utility for strings:

In [60]: np.array(alist)
Out[60]: 
array([['a1', 'b1', 'c1', 'd1', 'e1'],
       ['a2', 'b2', 'c2', 'd2', 'e2'],
       ['a3', 'b3', 'c3', 'd3', 'e3']], dtype='<U2')

Or in a compact list comprehension form:

In [61]: [[v str(i) for v in baseVariables] for i in nNumbers]
Out[61]: 
[['a1', 'b1', 'c1', 'd1', 'e1'],
 ['a2', 'b2', 'c2', 'd2', 'e2'],
 ['a3', 'b3', 'c3', 'd3', 'e3']]

You are starting with lists! And making strings! And selecting items from a JSON, with y['result'][i][v]. None of that benefits from using numpy, especially not the repeated use of np.append and np.concatenate.

CodePudding user response：

Could you provide an example of JSON? It sounds like you basically want to

1. Filter the JSON
2. Flatten the JSON Depending on what your output example means, you might want to not filter, but replace certain values with empty values, is that correct?

Please note that Pandas has very powerfull out-of-the-box options to handle, and in particular, flatten JSONs. https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#io-json-reader. An approach could be to first load in Pandas and filter it from there. Flattening a JSON can also be done by iterating over it like so:

def flatten_json(y):
    out = {}

    def flatten(x, name=''):
        if type(x) is dict:
            for a in x:
                flatten(x[a], name   a   '_')
        elif type(x) is list:
            i = 0
            for a in x:
                flatten(a, name   str(i)   '_')
                i  = 1
        else:
            out[name[:-1]] = x

    flatten(y)
    return out

I got this code from: https://towardsdatascience.com/flattening-json-objects-in-python-f5343c794b10. The author explains some challenges of flattening JSON. Of course, you can put some if statement into the function for your filtering need. I hope this can get you started at least!