How can i load my output generated from my bash script into a gcs location
Like my bash command is :
echo " hello world"
I want this output(hello world) to be shown in a location in gcs.
How to write a location command in bash.? Any updates would be appreciated? Thank You in Advance
CodePudding user response:
First, you should follow the install Cloud SDK instructions in order to use the cp command form gsutil tool on the machine you're running the script.
- Cloud SDK requires Python; supported versions are Python 3 (preferred, 3.5 to 3.8) and Python 2 (2.7.9 or higher)
- Run one of the following:
Linux 64-bit archive file from your command-line, run:
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-362.0.0-linux-x86_64.tar.gz
For the 32-bit archive file, run:
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-362.0.0-linux-x86.tar.gz
Depending on your setup, you can choose other installation methods
- Extract the contents of the file to any location on your file system (preferably your Home directory). If you would like to replace an existing installation, remove the existing
google-cloud-sdk
directory and extract the archive to the same location.- Run gcloud init to initialize the SDK:
./google-cloud-sdk/bin/gcloud init
After you have installed the Cloud SDK, you should create a bucket to uploadthe files that will contain the output generated by your script.
Use the
gsutil mb
command and a unique name to create a bucket:
gsutil mb -b on -l us-east1 gs://my-awesome-bucket/
This uses a bucket named "my-awesome-bucket". You must choose your own, globally-unique, bucket name.
Then you can redirect your output to a local file and upload to Google Cloud Storage like this:
#!/bin/bash
TIMESTAMP=$(date '%s')
BUCKET="my-awesome-bucket"
echo "Hello world!" > "logfile.$TIMESTAMP.log"
gsutil cp logfile.$TIMESTAMP.log gs://$BUCKET/logfile.$TIMESTAMP.log