Terraform external data source formatting issue-CodePudding

I want a list of my s3 buckets in TF.

With the AWS CLI I can get that list and format it into JSON:

my-mac$ eval aws s3 ls | awk '{print $3}' | jq -R -s -c 'split("\n")' | jq '.[:-1]'

[

  "xxx-bucket-1",

  "xxx-bucket-2"

]

But TF won't fetch it, and it's driving me mad.

Here's my data source config:

data "external" "bucket_objects" {

  program = ["bash", "cli.sh"]

}

The shell script file contents:

#!/bin/bash

set -e

eval aws s3 ls | awk '{print $3}' | jq -R -s -c 'split("\n")' | jq '.[:-1]'

And finally, the error message:

│ Error: Unexpected External Program Results

│ 

│   with data.external.bucket_objects,

│   on data.tf line 22, in data "external" "bucket_objects":

│   22:   program = ["bash", "cli.sh"]

│ 

│ The data source received unexpected results after executing the program.

│ 

│ Program output must be a JSON encoded map of string keys and string values.

│ 

│ If the error is unclear, the output can be viewed by enabling Terraform's

│ logging at TRACE level. Terraform documentation on logging:

│ https://www.terraform.io/internals/debugging

│ 

│ Program: /bin/bash

│ Result Error: json: cannot unmarshal array into Go value of type

│ map[string]string

My shell scripting game is weak and it's going to be something dumb, but I don't know what that is.

CodePudding user response：

Here is an example that worked in my case:

#!/bin/bash

set -e

BUCKET_NAMES=$(aws s3 ls | awk '{print $3}' | jq -R -s -c 'split("\n")' | jq '.[:-1]')

jq -n --arg bucket_names "$BUCKET_NAMES" '{"bucket_names":$bucket_names}'