Cloud Run Golang container issue/missunderstanding-CodePudding

I'm trying to do a report of all the objects in all the projects we have in Cloud Storage of our Org. I'm using this repo from the Google Professionnal Services as it's doing exactly what we want: https://github.com/GoogleCloudPlatform/professional-services/tree/main/tools/gcs2bq

We want to use containers instead of just the go code on a Cloud Function for portability mainly.

Locally everything is good and the program behave as expected but when I try in Cloud Run things get tricky. From what I understand, the go part needs to listen to a port, which I added at the beginning of the main so the container can be deployed, which it is:

// Determine port for HTTP service
port := os.Getenv("PORT")
if port == "" {
        port = "8080"
        log.Printf("defaulting to port %s", port)
}

Start HTTP server.
log.Printf("listening on port %s", port)
if err := http.ListenAndServe(":" port, nil); err != nil {
        log.Fatal(err)
}

But as you can see in the repo, the first file called is the run.sh one. Which set environment variables and then call the main.go. It sucessfully complete it's task, which is get all the size of the different files. But after that the run.sh doesnt "resume" and go to the part where it uploads the data in a BigQuery table, which locally work.

Here is the part in the run.sh file where I have a problem. Note : I don't have errors from executing the ./gcs2bq Note 2 : Every environment variable has a correct value

./gcs2bq $GCS2BQ_FLAGS || error "Export failed!" 2    <- doesnt get past this line


gsutil mb -p "${GCS2BQ_PROJECT}" -c standard -l "${GCS2BQ_LOCATION}" -b on "gs://${GCS2BQ_BUCKET}" || echo "Info: Storage bucket already exists: ${GCS2BQ_BUCKET}"

gsutil cp "${GCS2BQ_FILE}" "gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME}" || error "Failed copying ${GCS2BQ_FILE} to gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME}!" 3

bq mk --project_id="${GCS2BQ_PROJECT}" --location="${GCS2BQ_LOCATION}" "${GCS2BQ_DATASET}" || echo "Info: BigQuery dataset already exists: ${GCS2BQ_DATASET}"

bq load --project_id="${GCS2BQ_PROJECT}" --location="${GCS2BQ_LOCATION}" --schema bigquery.schema --source_format=AVRO --use_avro_logical_types --replace=true "${GCS2BQ_DATASET}.${GCS2BQ_TABLE}" "gs://${GCS2BQ_BUCKET}/${GCS2BQ_FIL$
  error "Failed to load gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME} to BigQuery table ${GCS2BQ_DATASET}.${GCS2BQ_TABLE}!" 4

gsutil rm "gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME}" || error "Failed deleting gs://${GCS2BQ_BUCKET}/${GCS2BQ_FILENAME}!" 5

rm -f "${GCS2BQ_FILE}"

I'm kinda new to containers and Cloud Run and even after reading projects and documentation I'm not sure what I'm doing wrong, Is it normal that the .sh is "stuck" when calling the main.go ? Any help is appreciated and I can provide more details/explaination if needed.

Thanks in advance.

CodePudding user response：

Okay so for anyone who encounter similar situation this is how I made it work for me.

The container isn't supposed to stop so no exit, it will just go back to the main function.

That means that when I called executable it just looped and never exited and completed the task. So the solution here is to "recode" everything past the call in golang directly into the main.go

Here the run.sh is then useless so I used another .go file that listen for http request and then call the code that gather data and send it to Bigquery.