How to skip or avoid duplicate keys from json when using for

I am trying to generate Terraform resources from a JSON file that has duplicate keys. Is there any way to avoid the duplicate?

Here is an example JSON

{
  "my-ptr-zone": {
    "21-10": {
      "zone"  : "21.10.in-addr.arpa"
    },

    "21-10": {
      "zone"  : "21.10.in-addr.arpa"
    }
  }
}

Both the keys are the same here.

And here is the resource

resource "aws_route53_zone" "my-ptr-zone" {
  for_each = var.my-ptr-zone
  name     = each.value.zone
}

Is there any way to skip the duplicates when looping?

I have tried to avoid generating duplicate values in the JSON file from the source, but that is another challenge.

CodePudding user response：

If you want to generate only one resource per unique key in a map, ask Terraform to generate a set of the keys, then loop over those.

Consider the following example using the hashicorp/random provider.

locals {
  my-ptr-zone = {
    "21-10" = {
      zone = "21.10.in-addr.arpa"
    }
    "21-10" = {
      zone = "21.10.in-addr.arpa"
    }
  }
}

resource "random_id" "example" {
  for_each = toset(keys(local.my-ptr-zone))
  byte_length = 8
}

Planning that with terraform plan, you'll see that terraform only wants to generate one resource.

# random_id.example["21-10"] will be created
  resource "random_id" "example" {
    b64_std     = (known after apply)
    b64_url     = (known after apply)
    byte_length = 8
    dec         = (known after apply)
    hex         = (known after apply)
    id          = (known after apply)
}

Plan: 1 to add, 0 to change, 0 to destroy.

Change one of the keys to something different, and terraform will want to create two resources.

# random_id.example["21-10"] will be created
...

# random_id.example["21-11"] will be created
...

Plan: 2 to add, 0 to change, 0 to destroy.

If you instead are interested in some other uniqueness (e.g. only the unique values of zone), the same approach could be applied ... generate a set of the things you want to be unique, then iterate over that set.

CodePudding user response：

A simple way to approach this would be to run a merge on the map twice.

JSON File:

{
  "my-ptr-zone": {
    "21-10": {
      "zone"  : "21.10.in-addr.arpa"
    },

    "21-10": {
      "zone"  : "21.10.in-addr.arpa"
    }
  }
}

TF Code:

locals {
  data  = jsondecode(file("data.json"))
  zones = merge(local.data.my-ptr-zone, local.data.my-ptr-zone)
}

output "zones" {
  value = local.zones
}

# This will now be a unique resource
resource "aws_route53_zone" "my-ptr-zone" {
  for_each = local.zones
  name     = each.value.zone
}

Output:

Outputs:

zones = {
  "21-10" = {
    "zone" = "21.10.in-addr.arpa"
  }
}