Using outputs terraform files in other terraform files-CodePudding

I am new to terraform and have a few questions. Is there a way in terraform, to use the outputs of a module, in another tf file, WITHOUT having a "main" terraform file, or defining both modules in the same file?

For example, this is a project structure:

project/
├── networking
│   ├── init.tf
│   ├── terraform.tfvars
│   ├── variables.tf
│   └── vpc.tf
├── prd-project-1
│   ├── init.tf
│   ├── instances.tf
│   ├── terraform.tfvars
│   └── variables.tf
├── prd-project-2
│   ├── init.tf
│   ├── instances.tf
│   ├── terraform.tfvars
│   └── variables.tf
├── prd-project-3
│   ├── init.tf
│   ├── instances.tf
│   ├── terraform.tfvars
│   └── variables.tf
├── test
│   ├── init.tf
│   ├── instances.tf
│   ├── terraform.tfvars
│   └── variables.tf

vpc.tf would create a VPC and subnets. instances.tf would create EC2 instances, based on the vpc-id and subnets created previously in vpc.tf. I want to treat/manage these as two seperate actions. ie. First, I would run terraform to create the vpc and subnets, then later I might run terraform to create instances in prod/test for various projects. In this example, I would have only 1 VPC for test and 1 VPC for prod. I would not want a VPC per project (which I am again assuming TF might create if the vpc.tf module is referenced in one of the sub-projects).

Is it best practise to manage terraform code in this way? My concern is if eventually there are many projects needing EC2 instances, I don't want all of these in a single huge main.tf. I am thinking it is safer/cleaner to maintain these all seperately, as long as the vpc outputs can be referenced. I also don't want my terraform plan/applies being slow and checking each and every resource from every project if I am standing up a new project.

The second part of my question is - consider the following code snippet:

module "main-vpc" {
  source     = "../modules/vpc"
  ENV        = "prod"
  AWS_REGION = var.AWS_REGION
}

module "instances" {
  source         = "../modules/instances"
  ENV            = "prod"
  VPC_ID         = module.main-vpc.vpc_id
  PUBLIC_SUBNETS = module.main-vpc.public_subnets
}

Is terraform smart enough to determine that a VPC was already created with an ENV = prod and would not try to recreate it if I called the same "main-vpc" module in another tf file/project?

I've read a few other posts here on stackoverflow, but they all allude that outputs from a module can only be referenced in another module within the same tf file.

Some other posts I've reviewed:

CodePudding user response：

If you want to keep your current folder structure, its best to add main.tf to each project:

project/
├── networking
│   ├── init.tf
│   ├── terraform.tfvars
│   ├── variables.tf
│   └── vpc.tf
├── prd-project-1
│   ├── main.tf
│   ├── init.tf
│   ├── instances.tf
│   ├── terraform.tfvars
│   └── variables.tf
├── prd-project-2
│   ├── main.tf
│   ├── init.tf
│   ├── instances.tf
│   ├── terraform.tfvars
│   └── variables.tf
├── prd-project-3
│   ├── main.tf
│   ├── init.tf
│   ├── instances.tf
│   ├── terraform.tfvars
│   └── variables.tf
├── test
│   ├── init.tf
│   ├── instances.tf
│   ├── terraform.tfvars
│   └── variables.tf

Then each main.tf would be something as follow:

module "main-vpc" {
  source     = "../networking"
  ENV        = "prod"
  AWS_REGION = var.AWS_REGION
}

module "instances" {
  source         = "../instances"
  ENV            = "prod"
  VPC_ID         = module.main-vpc.vpc_id
  PUBLIC_SUBNETS = module.main-vpc.public_subnets
}

This way you keep your project separate and each project will have its own TF state file.

Is terraform smart enough to determine that a VPC was already created with an ENV

No its not. You would have to program your own logic for that. A common way is through a variable

variable "vpc_id" {
  default = ""
}

then if you do not explicitly specify the vpc_id for a given project, it would create new VPC. Otherwise, it would use var.vpc_id that you provide as an input parameter. Using data sources you would then get other information for that VPC, such as its subnets.

Having this setup, you would cd to each project folder first, before deploying it, e.g.

cd ./prd-project-1
terraform apply

CodePudding user response：

If I understand your question, you want to have independent projects that can still access outputs from one another. So, for example, a project that spins up EC2 instances can refer to the subnets defined in the project that creates a VPC.

There are two ways to do this. First, you can use data sources to query the current deployment environment. For example, the following will look for the VPC with the ENV tag having a value of "prod" (assuming that I haven't made any typos):

data "aws_vpc" "prod" {
  tags = { "ENV": "prod" }
}

Second, you can use remote state (which is a good idea anyway, for security). Each project would write its outputs to a data store (S3 is common, but Terraform offers its own store), and they would be referenced by the others.

And while SO doesn't like links, I can't explain it better than the Terraform docs: https://www.terraform.io/docs/language/state/backends.html