Home > Software engineering >  How to get output with unique values but only with last instance of a duplicate key - unique_by
How to get output with unique values but only with last instance of a duplicate key - unique_by

Time:10-08

I am currently working with jq to parse through some json. I would like to retrieve unique values based on a certain key. I came across unique_by. It does just that of getting unique values for key name but I am still not getting my desired output. From my understanding, unique_by looks at key name value an uses the first instance and then removes the duplicates that follow in the final output. However, I would like to grab the last duplicate key name value and display that in the final output.

Below is an example of my desired output. Is it possible to do this with unique_by or what would be the best approach?

cat file.json

Original json:

[
    {
      "name": "app-fastly",
      "tag": "20210825-95-448f024",
      "image": "docker.io/repoxy/app-fastly:20210825-95-448f024"
    },
    {
      "name": "app-lovely",
      "tag": "20211004-2101-b6a256c",
      "image": "ghcr.io/repox/app-lovely:20211004-2101-b6a256c"
    },
    {
      "name": "app-lovely",
      "tag": "20211007-6622-b3fooba",
      "image": "ghcr.io/repoxy/app-lovely:20211007-6622-b3fooba"
    },
    {
      "name": "app-dogwood",
      "tag": "20210325-36-2a349e9",
      "image": "docker.io/repoxy/app-dogwood:20210325-36-2a349e9"
    }
]

Jq Command:

cat file.json | jq 'unique_by( {name} )'

Current Output:

[
  {
    "name": "app-dogwood",
    "tag": "20210325-36-2a349e9",
    "image": "docker.io/repoxy/app-dogwood:20210325-36-2a349e9"
  },
  {
    "name": "app-fastly",
    "tag": "20210825-95-448f024",
    "image": "docker.io/repoxy/app-fastly:20210825-95-448f024"
  },
  {
    "name": "app-lovely",
    "tag": "20211004-2101-b6a256c",
    "image": "ghcr.io/repox/app-lovely:20211004-2101-b6a256c"
  }
]

Desired Output:

[
  {
    "name": "app-dogwood",
    "tag": "20210325-36-2a349e9",
    "image": "docker.io/repoxy/app-dogwood:20210325-36-2a349e9"
  },
  {
    "name": "app-fastly",
    "tag": "20210825-95-448f024",
    "image": "docker.io/repoxy/app-fastly:20210825-95-448f024"
  },
  {
    "name": "app-lovely",
    "tag": "20211007-6622-b3fooba",
    "image": "ghcr.io/repoxy/app-lovely:20211007-6622-b3fooba"
  }
]

CodePudding user response:

If you want the last unique item, simply reverse the array first

jq 'reverse | unique_by( {name} )'

And if you want to retain the original order, reverse back again afterwards

jq 'reverse | unique_by( {name} ) | reverse'
  • Related