Home > Enterprise >  migrate multipart upload with presigned urls from aws javascript sdk v2 to v3
migrate multipart upload with presigned urls from aws javascript sdk v2 to v3

Time:05-07

I successfully have code that allows to support multipart uploads with presigned urls. I was successful converting the code for single uploads with presigned urls, but I cannot find out how to convert the creating of a presigned urls for a multipart upload from sdk v2 to sdk v3.

V2 - WORKING:

this is the working code with sdk v2

import * as AWS from 'aws-sdk'
const s3 = new AWS.S3()

async function getMultipartUploadId(bucket: string, key: string): Promise<any> {
  const params = {
    Bucket: ACCESS_POINT.concat(bucket),
    Key: key
  };
  return await new Promise((resolve, reject) => {
    s3.createMultipartUpload(params, (err, id) => {
      err ? reject(err) : resolve(id);
    });
  });
}

async function getPartUploadUrl(
  bucket: string,
  key: string,
  uploadId: string,
  partNumber: number,
  md5: string): Promise<string> {
  const params = {
    Bucket: bucket,
    Key: key,
    UploadId: uploadId,
    PartNumber: partNumber,
    Expires: 60 * 5,
    ContentMD5: md5
  };
  const url = await new Promise((resolve, reject) => {
    s3.getSignedUrl('uploadPart', params, (err, url) => {
      err ? reject(err) : resolve(url);
    });
  });
  return url as string;
}

async function completeMultiUpload ....

V3 - NOT WORKING:


import {
  S3Client,
  PutObjectCommand,
  CreateMultipartUploadCommand,
  CompleteMultipartUploadCommand
} from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner'

...

 this.client = new S3Client(
        {
          ...
        }
      )
...

/* this method produces successfully a upload id */
async getMultipartUploadId(bucket: string, key: string): Promise<string> {
    const input = {
      Bucket: this.accessPoint.concat(bucket),
      Key: key
    };
    const cmd = new CreateMultipartUploadCommand(input);
    const response = this.client.send(cmd);
    return response.UploadId;
  }


/**
   * Get an upload url for a part of a multi part upload.
   *
   * @param bucket The bucket name.
   * @param key The object key.
   * @param uploadId The multi part upload id.
   * @param partNumber The part number.
   * @param md5 The MD5 hash of the part.
   * @returns An upload url for a part of a multi part upload.
   */
  async getPartUploadUrl(
    bucket: string,
    key: string,
    uploadId: string,
    partNumber: number,
    md5: string): Promise<string> {

    const params = {
      Bucket: this.accessPoint.concat(bucket),
      Key: key,
      UploadId: uploadId,
      PartNumber: partNumber,
      ContentMD5: md5
    };

    const cmd = new PutObjectCommand(params);
    const signedUrl = await getSignedUrl(this.client, cmd, {
        expiresIn: 60 * 15,
      });
    return signedUrl as string;
  }

  async completeMultiUpload ....

The problem is that the presigned url is not for multipart but a complete file.

My question is how I produce a presigned url for a multipart upload. I cannot find any good documentation. I'm sure I'm using the getSignedUrl false, but I cannot find any information for this problem.

Thank you for helping me with this.

CodePudding user response:

There are 2 problems with the not working code:

1 - missing await statement

/* this method produces successfully a upload id */
async getMultipartUploadId(bucket: string, key: string): Promise<string> {
    const input = {
      Bucket: this.accessPoint.concat(bucket),
      Key: key
    };
    const cmd = new CreateMultipartUploadCommand(input);
    const response = await this.client.send(cmd);
    return response.UploadId;
  }

2 - use UploadPartCommand instead of PubObjectCommand


/**
   * Get an upload url for a part of a multi part upload.
   *
   * @param bucket The bucket name.
   * @param key The object key.
   * @param uploadId The multi part upload id.
   * @param partNumber The part number.
   * @param md5 The MD5 hash of the part.
   * @returns An upload url for a part of a multi part upload.
   */
  async getPartUploadUrl(
    bucket: string,
    key: string,
    uploadId: string,
    partNumber: number,
    md5: string): Promise<string> {

    const params = {
      Bucket: this.accessPoint.concat(bucket),
      Key: key,
      UploadId: uploadId,
      PartNumber: partNumber,
      ContentMD5: md5
    };

    const cmd = new UploadPartCommand(params);
    const signedUrl = await getSignedUrl(this.client, cmd, {
        expiresIn: 60 * 15,
      });
    return signedUrl as string;
  }


  • Related