I am using a Spring Boot app to expose an API tasked with uploading file to AWS S3. I am using a single instance of S3 client in the application:
@Bean
public AmazonS3 s3Client() {
BasicAWSCredentials credentials = new BasicAWSCredentials(awsAccessKey, awsSecretAccessKey);
AmazonS3ClientBuilder amazonS3ClientBuilder = AmazonS3ClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withEndpointConfiguration(
new AwsClientBuilder.EndpointConfiguration(
endpointUrl, Regions.EU_WEST_1.getName()));
return amazonS3ClientBuilder.build();
}
Can I create a bean for TransferManager in a similar fashion and reuse the same instance across requests or do I need to create a fresh instance for every API call to upload a file?
What is the recommended practice here?
CodePudding user response：
The recommended practice is to move away from AWS SDK for Java V1. Amazon recommends that you use AWS SDK for Java V2. Now to upload content to an Amazon S3 bucket, you can use the S3TransferManager.
If you are not familiar with using the V2 SDK, I recommend that you read the Developer Guide.
Here is a code example of how to use this to upload an object. And you can use the same instance to upload different objects. All you need to do is to set to pass the correct the values to transferManager.uploadFile().
package com.example.transfermanager;
import software.amazon.awssdk.auth.credentials.EnvironmentVariableCredentialsProvider;
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.transfer.s3.FileUpload;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import java.nio.file.Paths;
/**
* Before running this Java V2 code example, set up your development environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class UploadObject {
public static void main(String[] args) {
final String usage = "\n"
"Usage:\n"
" <bucketName> <objectKey> <objectPath> \n\n"
"Where:\n"
" bucketName - The Amazon S3 bucket to upload an object into.\n"
" objectKey - The object to upload (for example, book.pdf).\n"
" objectPath - The path where the file is located (for example, C:/AWS/book2.pdf). \n\n" ;
if (args.length != 3) {
System.out.println(usage);
System.exit(1);
}
long mb = 1024;
String bucketName = args[0];
String objectKey = args[1];
String objectPath = args[2];
System.out.println("Putting an object into bucket " bucketName " using the S3TransferManager");
ProfileCredentialsProvider credentialsProvider = ProfileCredentialsProvider.create();
Region region = Region.US_EAST_1;
S3TransferManager transferManager = S3TransferManager.builder()
.s3ClientConfiguration(cfg ->cfg.region(region)
.credentialsProvider(credentialsProvider)
.targetThroughputInGbps(20.0)
.minimumPartSizeInBytes(10 * mb))
.build();
uploadObjectTM(transferManager, bucketName, objectKey, objectPath);
System.out.println("Object was successfully uploaded using the Transfer Manager.");
transferManager.close();
}
public static void uploadObjectTM( S3TransferManager transferManager, String bucketName, String objectKey, String objectPath) {
FileUpload upload =
transferManager.uploadFile(u -> u.source(Paths.get(objectPath))
.putObjectRequest(p -> p.bucket(bucketName).key(objectKey)));
upload.completionFuture().join();
}
}
UPDATE
As the above API still appears to be in Preview Mode (judging from Maven repo):
https://mvnrepository.com/artifact/software.amazon.awssdk/s3-transfer-manager
You can use the Amazon S3 V2 Service Client to upload objects. V2 is still better practice then V1.
You can use this code:
package com.example.s3;
// snippet-start:[s3.java2.s3_object_upload.import]
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectResponse;
import software.amazon.awssdk.services.s3.model.S3Exception;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
// snippet-end:[s3.java2.s3_object_upload.import]
/**
* Before running this Java V2 code example, set up your development environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class PutObject {
public static void main(String[] args) {
final String usage = "\n"
"Usage:\n"
" <bucketName> <objectKey> <objectPath> \n\n"
"Where:\n"
" bucketName - The Amazon S3 bucket to upload an object into.\n"
" objectKey - The object to upload (for example, book.pdf).\n"
" objectPath - The path where the file is located (for example, C:/AWS/book2.pdf). \n\n" ;
if (args.length != 3) {
System.out.println(usage);
System.exit(1);
}
String bucketName =args[0];
String objectKey = args[1];
String objectPath = args[2];
System.out.println("Putting object " objectKey " into bucket " bucketName);
System.out.println(" in bucket: " bucketName);
ProfileCredentialsProvider credentialsProvider = ProfileCredentialsProvider.create();
Region region = Region.US_EAST_1;
S3Client s3 = S3Client.builder()
.region(region)
.credentialsProvider(credentialsProvider)
.build();
String result = putS3Object(s3, bucketName, objectKey, objectPath);
System.out.println("Tag information: " result);
s3.close();
}
// snippet-start:[s3.java2.s3_object_upload.main]
public static String putS3Object(S3Client s3,
String bucketName,
String objectKey,
String objectPath) {
try {
Map<String, String> metadata = new HashMap<>();
metadata.put("x-amz-meta-myVal", "test");
PutObjectRequest putOb = PutObjectRequest.builder()
.bucket(bucketName)
.key(objectKey)
.metadata(metadata)
.build();
PutObjectResponse response = s3.putObject(putOb,
RequestBody.fromBytes(getObjectFile(objectPath)));
return response.eTag();
} catch (S3Exception e) {
System.err.println(e.getMessage());
System.exit(1);
}
return "";
}
// Return a byte array.
private static byte[] getObjectFile(String filePath) {
FileInputStream fileInputStream = null;
byte[] bytesArray = null;
try {
File file = new File(filePath);
bytesArray = new byte[(int) file.length()];
fileInputStream = new FileInputStream(file);
fileInputStream.read(bytesArray);
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fileInputStream != null) {
try {
fileInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return bytesArray;
}
// snippet-end:[s3.java2.s3_object_upload.main]
}