Home > Software design >  Terraform cannot init custom provider written in Go
Terraform cannot init custom provider written in Go

Time：11-13

I am trying to do this tutorial to create a custom terraform module using plugin framework.

Everything is fine up to the point where I need to use terraform to install & verify the provider. It fails with:

➜  examples git:(main) ✗ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of example.com/vladmiller/example...

│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│  - example.com/vladmiller/example in /Users/vladi/go/bin
│
│ Skip terraform init when using provider development overrides. It is not necessary and may
│ error unexpectedly.
╵

╷
│ Error: Invalid provider registry host
│
│ The host "example.com" given in in provider source address "example.com/vladmiller/example"
│ does not offer a Terraform provider registry.

I also tried:

  • cloning repo from tutorial, but it also fails
  • using locally installed Go instead of GoLand
  • go1.18.8 instead of go1.19.x
  • pointing folder in .terraformrc to a folder where I develop code (I thought that maybe terraform-registry-manifest.json needs to be in the same folder w/ binary)
  • cloning and trying this repo – no luck
  • I'm currently trying to comprehend terraform code around how they load plugins...

Wondering if anyone have any ideas how to troubleshoot. Thank you in advance.

Looking at the logs it's clear that terraform is trying to get registry details by making request to example.com. Which, I expect, it shouldn't do because there are dev_overrides.

...
Service discovery for example.com at https://example.com/.well-known/terraform.json
...

Below is are more logs and example of my setup.

➜  examples git:(main) ✗ TF_LOG=trace terraform init
2022-11-10T16:39:54.255 0200 [INFO]  Terraform version: 1.3.4
2022-11-10T16:39:54.255 0200 [DEBUG] using github.com/hashicorp/go-tfe v1.9.0
2022-11-10T16:39:54.255 0200 [DEBUG] using github.com/hashicorp/hcl/v2 v2.14.1
2022-11-10T16:39:54.255 0200 [DEBUG] using github.com/hashicorp/terraform-config-inspect v0.0.0-20210209133302-4fd17a0faac2
2022-11-10T16:39:54.255 0200 [DEBUG] using github.com/hashicorp/terraform-svchost v0.0.0-20200729002733-f050f53b9734
2022-11-10T16:39:54.255 0200 [DEBUG] using github.com/zclconf/go-cty v1.12.0
2022-11-10T16:39:54.255 0200 [INFO]  Go runtime version: go1.19.3
2022-11-10T16:39:54.255 0200 [INFO]  CLI args: []string{"terraform", "init"}
2022-11-10T16:39:54.255 0200 [TRACE] Stdout is a terminal of width 206
2022-11-10T16:39:54.255 0200 [TRACE] Stderr is a terminal of width 206
2022-11-10T16:39:54.255 0200 [TRACE] Stdin is a terminal
2022-11-10T16:39:54.255 0200 [DEBUG] Attempting to open CLI config file: /Users/vladi/.terraformrc
2022-11-10T16:39:54.255 0200 [INFO]  Loading CLI configuration from /Users/vladi/.terraformrc
2022-11-10T16:39:54.255 0200 [DEBUG] Explicit provider installation configuration is set
2022-11-10T16:39:54.255 0200 [TRACE] Selected provider installation method cliconfig.ProviderInstallationDirect with includes [] and excludes []
2022-11-10T16:39:54.255 0200 [INFO]  CLI command args: []string{"init"}

Initializing the backend...
2022-11-10T16:39:54.256 0200 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2022-11-10T16:39:54.256 0200 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2022-11-10T16:39:54.256 0200 [DEBUG] New state was assigned lineage "b3cc2d29-4dd2-a955-32f4-55d7481d5445"
2022-11-10T16:39:54.256 0200 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2022-11-10T16:39:54.256 0200 [TRACE] Meta.Backend: instantiated backend of type <nil>
2022-11-10T16:39:54.256 0200 [DEBUG] Provider example.com/vladmiller/example is overridden by dev_overrides
2022-11-10T16:39:54.256 0200 [DEBUG] Provider example.com/vladmiller/example is overridden to load from /Users/vladi/go/bin
2022-11-10T16:39:54.256 0200 [DEBUG] checking for provisioner in "."
2022-11-10T16:39:54.263 0200 [DEBUG] checking for provisioner in "/opt/homebrew/bin"
2022-11-10T16:39:54.263 0200 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2022-11-10T16:39:54.263 0200 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2022-11-10T16:39:54.263 0200 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2022-11-10T16:39:54.263 0200 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2022-11-10T16:39:54.263 0200 [TRACE] statemgr.Filesystem: read nil snapshot
2022-11-10T16:39:54.263 0200 [DEBUG] Provider example.com/vladmiller/example is overridden by dev_overrides

Initializing provider plugins...
- Finding latest version of example.com/vladmiller/example...
2022-11-10T16:39:54.263 0200 [DEBUG] Service discovery for example.com at https://example.com/.well-known/terraform.json
2022-11-10T16:39:54.263 0200 [TRACE] HTTP client GET request to https://example.com/.well-known/terraform.json
╷
│ Warning: Provider development overrides are in effect
│ 
│ The following provider development overrides are set in the CLI configuration:
│  - example.com/vladmiller/example in /Users/vladi/go/bin
│ 
│ Skip terraform init when using provider development overrides. It is not necessary and may error unexpectedly.
╵

╷
│ Error: Invalid provider registry host
│ 
│ The host "example.com" given in in provider source address "example.com/vladmiller/example" does not offer a Terraform provider registry.
╵

➜  examples git:(main) ✗ terraform version
Terraform v1.3.4
on darwin_arm64

➜  examples git:(main) ✗ cat ~/.terraformrc
provider_installation {

  dev_overrides {
      "example.com/vladmiller/example" = "/Users/vladi/go/bin/"
  }

  # For all other providers, install them directly from their origin provider
  # registries as normal. If you omit this, Terraform will _only_ use
  # the dev_overrides block, and so no other providers will be available.
  direct {}
}

➜  examples git:(main) ✗ tree /Users/vladi/go/bin
/Users/vladi/go/bin
└── terraform-provider-example

0 directories, 1 file

➜  bin ./terraform-provider-example 
This binary is a plugin. These are not meant to be executed directly.
Please execute the program that consumes these plugins, which will
load any plugins automatically

➜  terraform-provider-example git:(main) ✗ cat main.go
package main

import (
    "context"
    "flag"
    "log"

    "github.com/hashicorp/terraform-plugin-framework/providerserver"
    "terraform-provider-example/example"
)

var (
    // these will be set by the goreleaser configuration
    // to appropriate values for the compiled binary
    version string = "dev"

    // goreleaser can also pass the specific commit if you want
    // commit  string = ""
)

func main() {
    var debug bool

    flag.BoolVar(&debug, "debug", false, "set to true to run the provider with support for debuggers like delve")
    flag.Parse()

    err := providerserver.Serve(context.Background(), example.New(version), providerserver.ServeOpts{
        Address: "example.com/vladmiller/example",
        Debug:   debug,
    })
    if err != nil {
        log.Fatal(err.Error())
    }
}

➜  examples git:(main) ✗ cat provider-verification.tf 
terraform {
  required_providers {
    example = {
      source = "example.com/vladmiller/example"
    }
  }
}

provider "example" {}

➜  terraform-provider-example git:(main) ✗ cat go.mod 
module terraform-provider-example

go 1.18

require github.com/hashicorp/terraform-plugin-framework v0.15.0

require (
        github.com/fatih/color v1.13.0 // indirect
        github.com/golang/protobuf v1.5.2 // indirect
        // truncated
)

UPDATE 1: I got wondering that maybe there is an issue with my environment. I'm on MacOS Ventura 13.0 w/ M2.

I started a container from ubuntu/latest aarch64 and installed latest terraform in the container. Cloned provider branch of https://github.com/hashicorp/terraform-provider-hashicups-pf. And it's still the same issue.

I have also tried terraform 1.3.0, 1.2.0 and 0.15.5 just in case.

root@23de1551280a:~/terraform-provider-hashicups-pf/examples/provider-install-verification# TF_LOG=debug terraform init
2022-11-11T12:45:23.284Z [INFO]  Terraform version: 1.3.4
2022-11-11T12:45:23.284Z [DEBUG] using github.com/hashicorp/go-tfe v1.9.0
2022-11-11T12:45:23.284Z [DEBUG] using github.com/hashicorp/hcl/v2 v2.14.1
2022-11-11T12:45:23.284Z [DEBUG] using github.com/hashicorp/terraform-config-inspect v0.0.0-20210209133302-4fd17a0faac2
2022-11-11T12:45:23.284Z [DEBUG] using github.com/hashicorp/terraform-svchost v0.0.0-20200729002733-f050f53b9734
2022-11-11T12:45:23.284Z [DEBUG] using github.com/zclconf/go-cty v1.12.0
2022-11-11T12:45:23.284Z [INFO]  Go runtime version: go1.19.3
2022-11-11T12:45:23.284Z [INFO]  CLI args: []string{"terraform", "init"}
2022-11-11T12:45:23.284Z [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2022-11-11T12:45:23.284Z [INFO]  Loading CLI configuration from /root/.terraformrc
2022-11-11T12:45:23.285Z [DEBUG] Explicit provider installation configuration is set
2022-11-11T12:45:23.286Z [INFO]  CLI command args: []string{"init"}

Initializing the backend...
2022-11-11T12:45:23.287Z [DEBUG] New state was assigned lineage "91db86b6-1777-8a07-27fd-aac42c238c35"
2022-11-11T12:45:23.287Z [DEBUG] Provider hashicorp.com/edu/hashicups-pf is overridden by dev_overrides
2022-11-11T12:45:23.287Z [DEBUG] Provider hashicorp.com/edu/hashicups-pf is overridden to load from /root/go/bin
2022-11-11T12:45:23.287Z [DEBUG] checking for provisioner in "."
2022-11-11T12:45:23.287Z [DEBUG] checking for provisioner in "/root"
2022-11-11T12:45:23.287Z [DEBUG] Provider hashicorp.com/edu/hashicups-pf is overridden by dev_overrides

Initializing provider plugins...
- Finding latest version of hashicorp.com/edu/hashicups-pf...
2022-11-11T12:45:23.287Z [DEBUG] Service discovery for hashicorp.com at https://hashicorp.com/.well-known/terraform.json
2022-11-11T12:45:23.377Z [DEBUG] Service discovery redirected to https://www.hashicorp.com/.well-known/terraform.json
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│  - hashicorp.com/edu/hashicups-pf in /root/go/bin
│
│ Skip terraform init when using provider development overrides. It is not necessary and may error unexpectedly.
╵

╷
│ Error: Invalid provider registry host
│
│ The host "hashicorp.com" given in in provider source address "hashicorp.com/edu/hashicups-pf" does not offer a Terraform provider registry.
╵

Update 2: at this point I am inclined to believe that there is something wrong with the repo as if it's missing something.

Update 3: So, terraform init does not work, however, terraform plan works like expected. :thinking_face:

I guess I'm unblocked to continue with the development process. Reading the tutorial again the do not seem to do terraform init there.

CodePudding user response：

The dev_overrides argument you mentioned tells other Terraform commands like terraform apply to ignore the dependency lock file and provider cache directory populated by terraform init and to use your specified path instead. It doesn't affect the behavior of terraform init itself; the installer will always try to find a reasonable package to download and record in the dependency lock file, to avoid creating an incomplete lock file.

As the warning message mentions, you don't actually need to run terraform init if your goal is only to test your overridden provider. If you just run terraform apply directly then your configured override will prevent Terraform from looking into the dependency lock file to find that provider and it'll instead just use directly the directory you specified. Automatic installation isn't necessary in this case because the package is already available at a designated location on your computer.

Skip terraform init when using provider development overrides. It is not necessary and may error unexpectedly.

Page link：https//www.codepudding.com/Softwaredesign/607782.html

Prev:how to invoke a function from the result of another function
Next:Trying to parse xml in golang but getting empty struct
  • Related

About Us:  Contact Us      Terms of Service       Privacy Policy

Copyright © 2010-2023，Powered By CodePudding