How can I remove the 1st line of a CSV file which is in AWS S3 through-CodePudding

I am trying to remove the 1st line of a CSV file which is in S3 bucket. I am using the below, but I am getting "sed: no input files" and is creating empty file in the test directory.

aws s3 cp  s3://my-bucket/rough/test.csv  -| sed -i '1d' | aws s3 cp -  s3://my-bucket/test/final.csv

CodePudding user response：

If you run the steps one-by-one, you will see an error:

sed: -I or -i may not be used with stdin

So, use this:

aws s3 cp s3://my-bucket/rough/test.csv - \
    | sed '1d' \
    | aws s3 cp - s3://my-bucket/test/final.csv