yq (GO/Mike Farah) uniquify all arrays recursively-CodePudding

This is probably related to another question I posted: yq (GO/Mike Farah) sort all arrays recursively?

Mike Farah's yq provides documentation for making arrays unique but I'm having trouble figuring how to apply that to lists that are nested deeper

Input

classes:
  driver:
    fields:
      - height
      - age
      - age
  vehicle:
    fields:
      - model
      - model
      - color
      - year

classes:
  driver:
    fields:
      - age
      - height
  vehicle:
    fields:
      - color
      - model
      - year

Naively trying to globally uniqify

cat to_sort.yaml | yq 'unique'

Error: Only arrays are supported for unique

And if it takes arguments, I don't know what to provide. I don't want to just sort one explicit path, but I did try this:

 cat to_sort.yaml | yq 'unique(.classes.driver.fields)'

Error: Bad expression, please check expression syntax

I have seen some yq examples where one has to do a select operation first, but I don't know what to try in this case.

CodePudding user response：

 yq e '(... | select(type == "!!seq")) |= unique' input

Will recursively loop over all the items, and select() those of type !!seq

Then update (|=) those with unique:

Result from provided input:

classes:
  driver:
    fields:
      - height
      - age
  vehicle:
    fields:
      - model
      - color
      - year

CodePudding user response：

You have to first traverse there, then update the array (here, using the update |= operator).

Either one after another:

yq '
  .classes.driver.fields |= unique
  | .classes.vehicle.fields |= unique
' to_sort.yaml

Or both at once:

yq '
  (.classes.driver.fields, .classes.vehicle.fields) |= unique
' to_sort.yaml

Both output

classes:
  driver:
    fields:
      - height
      - age
  vehicle:
    fields:
      - model
      - color
      - year