Python throws KeyError: 1 when using a Dictionary restored from a file-CodePudding

In my program, I have certain settings that can be modified by the user, saved on the disk, and then loaded when application is restarted. Some these settings are stored as dictionaries. While trying to implement this, I noticed that after a dictionary is restored, it's values cannot be used to access values of another dictionary, because it throws a KeyError: 1 exception.

This is a minimal code example that ilustrates the issue:

import json

motorRemap = {
    1: 3,
    2: 1,
    3: 6,
    4: 4,
    5: 5,
    6: 2,
}

motorPins = {
    1: 6,
    2: 9,
    3: 10,
    4: 11,
    5: 13,
    6: 22
}

print(motorPins[motorRemap[1]]); #works correctly

with open('motorRemap.json', 'w') as fp:
    json.dump(motorRemap, fp)

with open('motorRemap.json', 'r') as fp:
    motorRemap = json.load(fp)

print(motorPins[motorRemap[1]]); #throws KeyError: 1

You can run this code as it is. First print statement works fine, but after the first dictionary is saved and restored, it doesn't work anymore. Apparently, saving/restoring somehow breaks that dictionary.

I have tried saving and restoring with json and pickle libraries, and both produce in the same error. I tried printing values of the first dictionary after it is restored directly ( print(motorRemap[1]), and it prints out correct values without any added spaces or anything. KeyError usually means that the specified key doesn't exist in the dictionary, but in this instance print statement shows that it does exist - unless some underlying data types have changed or something. So I am really puzzled as to why this is happening.

Can anyone help me understand what is causing this issue, and how to solve it?

CodePudding user response：

What happens becomes clear when you look at what json.dump wrote into motorRemap.json:

{"1": 3, "2": 1, "3": 6, "4": 4, "5": 5, "6": 2}

Unlike Python, json can only use strings as keys. Python, on the other hand, allows many different types for dictionary keys, including booleans, floats and even tuples:

my_dict = {False: 1,
           3.14: 2,
           (1, 2): 3}

print(my_dict[False], my_dict[3.14], my_dict[(1, 2)])
# Outputs '1 2 3'

The json.dump function automatically converts some of these types to string when you try to save the dictionary to a json file. False becomes "false", 3.14 becomes "3.14" and, in your example, 1 becomes "1". (This doesn't work for the more complex types such as a tuple. You will get a TypeError if you try to json.dump the above dictionary where one of the keys is (1, 2).)

Note how the keys change when you dump and load a dictionary with some of the Python-specific keys:

import json

my_dict = {False: 1,
           3.14: 2}

print(my_dict[False], my_dict[3.14])

with open('my_dict.json', 'w') as fp:
    json.dump(my_dict, fp)
    # Writes {"false": 1, "3.14": 2} into the json file

with open('my_dict.json', 'r') as fp:
    my_dict = json.load(fp)
    
print(my_dict["false"], my_dict["3.14"])
# And not my_dict[False] or my_dict[3.14] which raise a KeyError

Thus, the solution to your issue is to access the values using strings rather than integers after you load the dictionary from the json file. print(motorPins[motorRemap["1"]]) instead of your last line will fix your code.

From a more general perspective, it might be worth considering keeping the keys as strings from the beginning if you know you will be saving the dictionary into a json file. You could also convert the values back to integers after loading as discussed here; however, that can lead to bugs if not all the keys are integers and is not a very good idea in bigger scale.

Checkout pickle if you want to save the dictionary keeping the Python format. It is, however, not human-readable unlike json and it's also Python-specific so it cannot be used to transfer data to other languages, missing virtually all the main benefits of json.

If you want to save and load the dictionary using pickle, this is how you would do it:

# import pickle
...
with open('motorRemap.b', 'wb') as fp:
    pickle.dump(motorRemap, fp)

with open('motorRemap.b', 'rb') as fp:
    motorRemap = pickle.load(fp)
...

CodePudding user response：

since the keys (integers) from a dict will be written to the json file as strings, we can modify the reading of the json file. using a dict comprehension restores the original dict values:

...
with open('motorRemap.json', 'r') as fp:
    motorRemap = {int(item[0]):item[1] for item in json.load(fp).items()}
...