drop a dictionary with nan value-CodePudding

I have the following dictionary:

my_dict = {'fields': ['id': 1.0,
  'name': 'aaa',
  'type': 'string'},
 {'id': 3.0,
  'name': 'eee',
  'type': 'string'},

 {'id': nan,
  'name': 'bbb',
  'type': 'string'},

 {'id': 4.0,
  'name': 'ccc',
  'type': 'string'},

 {'id': nan,
  'name': 'ddd',
  'type': 'string'}],

'type': 'struct'
}

From this dictionary, I would like to drop the dictionary with the id value nan value and would like to get the following.

my_updated_dict = {'fields': ['id': 1.0,
  'name': 'aaa',
  'type': 'string'},

 {'id': 3.0,
  'name': 'eee',
  'type': 'string'},

 {'id': 4.0,
  'name': 'ccc',
  'type': 'string'}],

'type': 'struct'
}

I was trying changing to data frame and dropping the id value with the nan value and changing to dictionary back but couldn't get the intended result.

 my_updated_dict = pd.DataFrame(my_dict ).dropna().to_dict('list')

CodePudding user response：

I do not know why would you need pandas for that if u can simply do:

my_dict["fields"] = [i for i in my_dict["fields"] if not np.isnan(i["id"])]

** UPDATE **

if you really do need for some reason to use pandas, you may try this constructiion:

my_dict["fields"] = pd.Series(my_dict["fields"]).apply(pd.Series).dropna().to_dict(orient="records")

though I do not see any advantages over simple list comprehension, except may be on big volume of information.

CodePudding user response：

You can use update() to overwrite the value of the key, then you can try:

my_dict.update({'fields':[x for x in my_dict['fields'] if np.nan not in x.values()]})

Returning:

{'fields': [{'id': 1.0, 'name': 'aaa', 'type': 'string'},
  {'id': 3.0, 'name': 'eee', 'type': 'string'},
  {'id': 4.0, 'name': 'ccc', 'type': 'string'}],
 'type': 'struct'}