Delete unused key:value properties in JSON file-CodePudding

I have key:value JSON object that is used in my JavaScript project. Value is a string and this object looks like this

{
  key1:{
    someKey: "Some text",
    someKey2: "Some text2"
  },
  key2:{
    someKey3:{
      someKey4: "Some text3",
      someKey5: "Some text4"
    }
  }
}

I use it in the project like this: key1.someKey and key2.someKey3.someKey4. Do you have idea how to delete unused properties? Let's say we don't use key2.someKey3.someKey5 in any file in a project, so i want it to be deleted from a JSON file. To people in the comments. I did't say i want to use JavaScript for this. I don't want to use it in browser or server. I just want the script that can do that on my local computer.

CodePudding user response：

If you live within javascript and node, you can use something like this to get all the paths:

Using some modified code from here: https://stackoverflow.com/a/70763473/999943

var lodash=require('lodash')  // use this if calling from the node REPL
// import lodash from 'lodash';  // use this if calling from a script
const allPaths = (o, prefix = '', out = []) => {
        if (lodash.isObject(o) || lodash.isArray(o)) Object.entries(o).forEach(([k, v]) => allPaths(v, prefix === '' ? k : `${prefix}.${k}`, out));
        else out.push(prefix);
        return out;
    };
let j = {
  key1: { someKey: 'Some text', someKey2: 'Some text2' },
  key2: { someKey3: { someKey4: 'Some text3', someKey5: 'Some text4' } }
}
allPaths(j)
[
  'key1.someKey',
  'key1.someKey2',
  'key2.someKey3.someKey4',
  'key2.someKey3.someKey5'
]

That's all well and good, but now you want to take that list and look through your codebase for usage.

The main choices available are text searching with grep or awk or ag, or parse the language and look through the symbolic representation of the language after it's loaded into your project. Tree-shaking can do this for libraries... I haven't looked into how to do tree-shaking for dictionary keys, or some other undefined reference check like a linter may do for a language.

Then once you have all the instances found, then you either manually modify your list or use a json library to modify it.

My weapons of choice in this instance are: jq and bash and grep

It's not infallible. But it's a start. (use with caution).

setup_test.sh

#!/usr/bin/env bash

mkdir src
echo "key2.someKey3.someKey4" > src/a.js
echo "key1.someKey2" > src/b.js
echo "key3.otherKey" > src/c.js

test.json

{
  "key1":{
    "someKey": "Some text",
    "someKey2": "Some text2"
  },
  "key2":{
    "someKey3":{
      "someKey4": "Some text3",
      "someKey5": "Some text4"
    }
  }
}

check_for_dict_references.sh

#!/usr/bin/env bash

json_input=$1
code_path=$2

cat << HEREDOC
  json_input=$json_input
  code_path=$code_path
HEREDOC

echo "Paths found in json"
paths="$(cat "$json_input" | jq -r 'paths | join(".")')"

no_refs=
for path in $paths; do
  escaped_path=$(echo "$path" | sed -e "s|\.|\\\\.|g")
  if ! grep -r "$escaped_path" "$code_path" ; then
    no_refs="$no_refs $path"
  fi
done
echo "Missing paths..."
echo "$no_refs"

echo "Creating a new json file without the unused paths"
del_paths_list=
for path in $no_refs; do
  del_paths_list =".$path, "
done
del_paths_list=${del_paths_list:0:-2} # remove trailing comma space

cat "$json_input" | jq -r 'del('$del_paths_list')' > ${json_input}.new.json

Running the setup_test.sh, then we can test the jq grep solution

$ ./check_for_dict_references.sh test.json src
  json_input=test.json
  code_path=src
Paths found in json
src/b.js:key1.someKey2
src/b.js:key1.someKey2
src/b.js:key1.someKey2
src/a.js:key2.someKey3.someKey4
src/a.js:key2.someKey3.someKey4
src/a.js:key2.someKey3.someKey4
Missing paths...
 key2.someKey3.someKey5
Creating a new json file without the unused paths

If you look closely you would want it to also print key1.someKey, but this got "found" in the middle of the name key1.someKey2. There are some more fancy regex things you can do, but for the purpose of this script it may be enough.

Now look in your directory for the new json file:

$ cat test.json.new.json
{
  "key1": {
    "someKey": "Some text",
    "someKey2": "Some text2"
  },
  "key2": {
    "someKey3": {
      "someKey4": "Some text3"
    }
  }
}

Hope that helps.