Is there a way to read csv file from s3 using java without downloading it-CodePudding

i was able to connect java to aws s3, i was able to perform basic operations like listing buckets, so i just need a way to read csv file without downloading it. I am attaching my current code here.

import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.services.s3.model.CannedAccessControlList;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.S3ObjectSummary;

import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import java.util.Properties;

public class test {

    public static void main(String args[])throws IOException
    {

        AWSCredentials credentials =new BasicAWSCredentials("----","----");

        AmazonS3 s3client = AmazonS3ClientBuilder
                .standard()
                .withCredentials(new AWSStaticCredentialsProvider(credentials))
                .withRegion(Regions.US_EAST_2)
                .build();

        List<Bucket> buckets = s3client.listBuckets();
        for(Bucket bucket : buckets) {
            System.out.println(bucket.getName());
        }
    }

}

CodePudding user response：

For your code to read the file, it needs the contents -- and that means copying it to the local system.

However, you can use "range" (Java) to read just a part.

CodePudding user response：

There is a way with a code like this :

S3Object Obj = s3client.getObject("<Bucket Name>", "File Name");
BufferedReader reader = new BufferedReader(new InputStreamReader(Obj.getObjectContent()));
line = reader.readLine();
String row[] = line.split(",");
int length = row.length;
while((line=reader.readLine()) != null) {
    String value[] = line.split(",");
    for(int i=0;i<length;i  ) {
        System.out.print(value[i] "   ");
    }
    System.out.println();
}