Ruby test script upload file to gcs fails with curl command-CodePudding

I have problems with uploading to Google cloud storage via a signed url.

In order to find what is wrong a wrote a little test script in ruby.

#
#
#     Script to see how upload ror url generation works.
#
#

require "google/cloud/storage"
require 'digest'

project_id = "Your project_id"
bucket_name = "your_bucket"

# commandline...
unless ARGV.length == 1 or ARGV.length == 2
  puts "Usage: gcs_upload_file.rb file_to_upload content_type (default image/jpeg) \n"
  exit
end

@Content_type = "image/jpeg"
file_path = ARGV[0]

if !ARGV[1].nil?
  @Content_type = ARGV[1]
end

# File check
if !File.exist? file_path
  puts "Unable to open file #{ARGV[0]}\n"
  exit
end

def hex_to_base64_digest(hexdigest)
  [[hexdigest].pack("H*")].pack("m").strip
end

# make md5 and file size
file_size = File.size file_path
digest = Digest::MD5.file(file_path).hexdigest
Content_md5 = hex_to_base64_digest digest

p file_path
p "file_size = #{file_size}"
p "digest = #{digest}"
p "Content_md5 =  #{Content_md5}"

gcloud = Google::Cloud.new project_id, "./config/gcs.json"
storage = gcloud.storage
bucket = storage.bucket bucket_name

p "So far so good"
#p bucket.inspect
headers = Hash.new

# bug bug bug
# this should be changed into:
# sign = Google::Cloud::Storage::File::SignerV2.from_bucket bucket, file_path
sign = Google::Cloud::Storage::File::SignerV2.from_bucket bucket, nil
generated_url = sign.signed_url method: "PUT",
                                expires: 120,
                                headers: headers,
                                content_type: @Content_type,
                                content_md5: Content_md5,
                                issuer: nil,
                                client_email: nil,
                                signing_key: nil,
                                private_key: nil,
                                signer: nil,
                                query: nil

p "Generated url:"
p generated_url



commando = "curl -X PUT \""   generated_url   '"'
commando  = " -H \"Content-Type: #{@Content_type}\" "
#commando  = " -H Content-length:4254548 "
commando  = " -H Content-length:#{file_size} "
#commando  = " -H \"Content-MD5: 4IXaimkAX9KasFuXznB3jg==\" "
commando  = " -H \"Content-MD5: #{Content_md5}\" "
commando  = " --upload-file #{file_path} "
# p 'curl -X PUT "#{generated_url}" -H "Content-Type: image/jpeg" -H "Content-length: 100" --upload-file test.jpg '
#exec('curl -X PUT "#{generated_url}" -H "Content-Type: image/jpeg" -H "Content-length: 100" --upload-file test.jpg ')
p commando
exec(commando)

p "End of story !"

I get the impression the url is correct because in other situations i got a not correct signed message.

I send a text file hello_world.txt:

Start
Hello World
Hello World
Hello World
End

Commandline is:

ruby gcs_upload_file.rb hello_world.txt plain/text

But the upload fails, output of the script is:

"hello_world.txt"
"file_size = 51"
"digest = dcb2d4acdfc14f7fee21e64bff11892c"
"Content_md5 =  3LLUrN/BT3/uIeZL/xGJLA=="
"So far so good"
"Generated url:"
"https://storage.googleapis.com/my_bucket/[email protected]&Expires=1630665326&Signature=jboMRUyCXsMWvE5ctNRnP4l0lR521PTEP7l1bAJDqBapOPGDT7e1L
iIOWRBP5M4Q6y4K6CN98wAKMhPW7eXx/rHikuGsy20qQ8qsCEtyGvuS3Q2bizG32bw/6VVNSCARpok0hg+yr6cs2LC6CNUHjQ4mH5bFpxD1L6ZdXS2hsC1LNsIzuK+imS1RWo950aCBzKs4oPo1cw8lJoLSrg3Y9yUi3eytRr9zarpZECpqgmQynPeGKTS9aEqVlL
B8pLQ/wfBkhlKYC3AAXWcd0RIJjTBTHqVaL3hw/rVYWn/KmU6U3Q/rwRMm3HAvyGYYpn85LxpXXZYh1mfnjjXTSI6mvA=="
"curl -X PUT \"https://storage.googleapis.com/my_bucket/[email protected]&Expires=1630665326&Signature=jboMRUyCXsMWvE5ctNRnP4l0lR521PTEP7l1bAJ
DqBapOPGDT7e1LiIOWRBP5M4Q6y4K6CN98wAKMhPW7eXx/rHikuGsy20qQ8qsCEtyGvuS3Q2bizG32bw/6VVNSCARpok0hg+yr6cs2LC6CNUHjQ4mH5bFpxD1L6ZdXS2hsC1LNsIzuK+imS1RWo950aCBzKs4oPo1cw8lJoLSrg3Y9yUi3eytRr9zarpZECpqgmQy
nPeGKTS9aEqVlLB8pLQ/wfBkhlKYC3AAXWcd0RIJjTBTHqVaL3hw/rVYWn/KmU6U3Q/rwRMm3HAvyGYYpn85LxpXXZYh1mfnjjXTSI6mvA==\" -H \"Content-Type: plain/text\"  -H Content-length:51  -H \"Content-MD5: 3LLUrN/BT
3/uIeZL/xGJLA==\"  --upload-file hello_world.txt "
<?xml version='1.0' encoding='UTF-8'?><Error><Code>MalformedBucketConfiguration</Code><Message>The XML you provided was not well-formed or did not validate against our published schema.</Message><Details>F
ailed to parse bucket creation configuration Start
Hello World
Hello World
Hello World
End
</Details></Error>

Why is this not working ?

I working with ruby 2.7 on Windows.

CodePudding user response：

The signed URL you have generated https://storage.googleapis.com/my_bucket/?GoogleAccessId=... is not correct, as it is missing the path to the object. This is why the error message indicates GCS thinks you are creating a bucket rather than uploading a object.

This is happening because you are passing in nil as the path. Try specifying the path:

sign = Google::Cloud::Storage::File::SignerV2.from_bucket bucket, "my_object"