JSONDecode json format error from terraform-CodePudding

Trying to add sql dbs created from azure portal to azure failover group.

Terraform block will call bash script :

data "external" "database_names" {
  program = ["sh", "${path.module}/scripts/fetch_db_id.sh"]
  query = {
    db_rg            = azurerm_resource_group.mssql.name
    server_name      = azurerm_mssql_server.mssqlserver.name
  }
}

fetch_db_id.sh bash script:

#!/usr/bin/env bash
# This script will get the database names at runtime.
eval "$(jq -r '@sh "export DB_RG=\(.db_rg) SERVER_NAME=\(.server_name)"')"

if [[ -z $DB_RG || -z $SERVER_NAME ]]; then
echo "Required variables DB_RG & SERVER_NAME not set" 1>&2
exit 1
fi

db_id=$(az sql db list --resource-group $DB_RG --server $SERVER_NAME --query [*].id | grep -v master 2>/dev/null)

jq -n --arg db_id "$db_id" '{"db_id":$db_id}'

unset DB_RG SERVER_NAME NODE_RG db_id

exit 0

Bash script output : Ran it locally on a linux VM without terraform :

"db_id": "[\n "/subscriptions/my_subscription_id/resourceGroups/sql_rg/providers/Microsoft.Sql/servers/sql_server_name/databases/databaseprd-db1",\n "/subscriptions/my_subscription_id/resourceGroups/sql_rg/providers/Microsoft.Sql/servers/sql_server_name/databases/databaseprd-db2",\n "/subscriptions/my_subscription_id/resourceGroups/sql_rg/providers/Microsoft.Sql/servers/sql_server_name/databases/databaseprd-db3"\n]" }

Terraform resource block to add database inside failover group:

resource "azurerm_sql_failover_group" "mssql_failover" {
  count                         = (var.enable_read_replica  && var.environment == "prd") ? 0 : 1
  name                          = var.mssql_failover_group
  resource_group_name           = azurerm_resource_group.mssql.name
  server_name                   = azurerm_mssql_server.mssqlserver.name
  databases                     = toset(jsondecode(data.external.database_names.result["db_id"]))
  partner_servers {
    id = azurerm_mssql_server.replica[0].id
  }

  read_write_endpoint_failover_policy {
    mode                        = "Automatic"
    grace_minutes               = 60
  }
  depends_on = [
    azurerm_mssql_server.replica
  ]
}

terraform error code: when executed via terraform jenkins pipeline

[1mdata.external.database_names.result["db_id"][0m is "[\n "/subscriptions/my_subscription_id/resourceGroups/sql_rg/providers/Microsoft.Sql/servers/sql_server_name/databases/databaseprd-db1",\n "/subscriptions/my_subscription_id/resourceGroups/sql_rg/providers/Microsoft.Sql/servers/sql_server_name/databases/databaseprd-db2",\n "/subscriptions/my_subscription_id/resourceGroups/sql_rg/providers/Microsoft.Sql/servers/sql_server_name/databases/databaseprd-db3",\n]" Call to function "jsondecode" failed: invalid character ']' looking for beginning of value

Note: it is introducing an extra "," when we run it with terraform jenkins pipeline, which could lead to json error.

CodePudding user response：

I don't think there's enough information here to be certain, but I can give a partial, speculative answer.

az sql db list --resource-group $DB_RG --server $SERVER_NAME --query [*].id | grep -v master 2>/dev/null

This looks suspicious: the az command outputs JSON, but you're filtering it with grep. What does the output from az look like here? Do you expect the result to be valid JSON?

You say that our output has a comma that you don't expect. This is what you'd expect to see if the az command spat out something like:

[
  "/blah/blah/databaseprd-db1",
  "/blah/blah/databaseprd-db2",
  "/blah/blah/databaseprd-db3",
  "/blah/blah/master"
]

The grep -v master would remove the line containing the term "master", leaving you with invalid JSON:

[
  "/blah/blah/databaseprd-db1",
  "/blah/blah/databaseprd-db2",
  "/blah/blah/databaseprd-db3",
]

If you want to use jq, you could replace the grep with something like

jq 'map(select(index("master")|not))'