I create few resources in the terraform -
resource "aws_dms_replication_instance" "foobar_instance_1" {
}
resource "aws_dms_replication_instance" "foobar_instance_2" {
}
resource "aws_dms_endpoint" "foobar_source_1" {
}
resource "aws_dms_endpoint" "foobar_source_2" {
}
Then define a replication task that's dependent on above two resources -
resource "aws_dms_replication_task" "foobar_task_1" {
replication_instance_arn = aws_dms_replication_instance.foobar_instance_1.replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.foobar_source_1.endpoint_arn
.
.
.
}
I want to use a tf variable to automate a similar aws_dms_replication_task
creation using for_each. I create a Map variable in variables.tf
that
looks like this -
variable "task_map" {
type = map(object({
source_endpoint = string
repl_instance = string
}))
default = {
"dms_attr_map" = {
source_ep = "foobar_source_1"
repl_instance = "foobar_instance_1"
},
"dms_attr_map2" = {
source_ep = "foobar_source_2"
repl_instance = "foobar_instance_2"
}
}
}
Now, I go on to create a resource block to loop over task_map and create replication_task
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.${each.value["repl_instance"]}.replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
.
.
.
}
As I execute my plan, terrraform plan throws an error that referring to replication_instance using ${each.value["repl_instance"]}
is wrong.
Error message -
│ Error: Invalid character │ │ On ../modules/cdc/main.tf line 236: This character is not used within the │ language. ╵
╷ │ Error: Invalid attribute name │ │ On ../modules/cdc/main.tf line 233: An attribute name is required after a │ dot. ╵
Error messages point to the specific line where I use for_each to refer to replication_instance and source_endpoint
aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
aws_dms_replication_instance.${each.value["repl_instance"]}.replication_instance_arn
How do I refer to the resources created using their names using a for_each
.
Thank you.
CodePudding user response:
How do I refer to the resources created using their names using a for_each.
You can't do this. In other words, you can't create dynamic references to resources in the form of:
aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
Instead you have to use maps or lists to create your aws_dms_replication_instance
and aws_dms_endpoint
. For example:
resource "aws_dms_replication_instance" "foobar_instance" {
for_each = toset(["foobar_instance_1", "foobar_instance_2"])
}
resource "aws_dms_endpoint" "foobar_source" {
for_each = toset(["foobar_source_1", "foobar_source_2"])
}
end then refer to them as follows:
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.foobar_instance[each.value["repl_instance"]].replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.foobar_source[each.value["source_ep"]].endpoint_arn
.
.
.
}
CodePudding user response:
When thinking about problems like this it's important to recognize that a reference expression like aws_dms_replication_instance.foobar_instance_1
is an indivisible unit: aws_dms_replication_instance
alone doesn't exist as a separate data structure that you can query dynamically, because Terraform needs to be able to determine exactly which resources a particular expression depends on in order to produce the dependency graph, before evaluating any expressions.
However, you can construct your own mapping data structure which incorporates the set of resources you are interested in:
locals {
source_endpoints = {
"foobar_1" = aws_dms_endpoint.foobar_source_1
"foobar_2" = aws_dms_endpoint.foobar_source_2
}
replication_instances = {
"foobar_1" = aws_dms_replication_instance.foobar_instance_1
"foobar_2" = aws_dms_replication_instance.foobar_instance_2
}
}
You can then use local.source_endpoints
and local.replication_instances
as mappings to look up the keys specified by your module's caller:
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = local.replication_instances[each.value.repl_instance].replication_instance_arn
source_endpoint_arn = local.source_endpoints[each.value.source_ep].endpoint_arn
# ...
}
This can work because a local value is also an object that participates in the dependency graph. Terraform can see that local.source_endpoints
depends on both aws_dms_endpoint.foobar_source_1
and aws_dms_endpoint.foobar_source_2
, and so therefore indirectly anything which depends on local.source_endpoints
must effectively depend on those resources.
Although it's not crucial to your question, I want to note that this design means that your module will declare the full set of aws_dms_endpoint
and aws_dms_replication_instance
objects, even if some of them don't have any var.task_map
elements referring to them.
I wouldn't worry about that if the only practical use of the module involves referring to all of them, but if that isn't true then a variant of this design is to allow the caller to also specify via input variables which endpoints and replication instances they need, and use for_each
on all three of these resources.
If you do that then you can avoid constructing the intermediate data structure, because for_each
resources are already naturally maps which support the same sort of dynamic lookup:
resource "aws_dms_replication_instance" "example" {
for_each = var.replication_instances
# ...
}
resource "aws_dms_endpoint" "example" {
for_each = var.source_endpoints
# ...
}
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.example[each.value.repl_instance].replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.example[each.value.source_ep].endpoint_arn
# ...
}
In this variant, the caller of the module controls both which replication instances and endpoints exist and which of those each of the tasks use, so the burden for the user of your module is greater but they also get the flexibility of not declaring objects they won't actually use, in case these objects have a significant cost.