how to create nested folders using aws cli in s3 bucket-CodePudding

In linux to create nested folders, irrespective of the intermediate folders exist or not can be done using the below command.

mkdir -p /home/user/some_non_existing_folder1/some_non_existing_folder2/somefolder

Similar to this i want to create a nested folder structure in S3 and place my files there later

how can i do this using aws cli

CodePudding user response：

Folders do not actually exist in Amazon S3.

For example, if you have an empty bucket you could upload a file to invoices/january.txt and the invoices folder will magically 'appear' without needing to be specifically created.

Then, if you were to delete the invoices/january.txt object, then the invoices folder will magically 'disappear' (because it never actually existed).

This works because the Key (filename) of an Amazon S3 object contains the full path of the object. The above object is not called january.txt -- rather, it is called invoices/january.txt. The Amazon S3 console will make it appear as if the folder exists, but it doesn't.

If you click the Create folder button in the S3 management console, then a zero-length object is created with the name of the folder. This causes the folder to 'appear' because it contains a file, but the file will not appear (well, it does appear, but humans see it as a folder). If this zero-length object is deleted, the folder will 'disappear' (because it never actually existed).

Therefore, if you wish to create a directory hierarchy before uploading files, you could upload zero-length objects with the same names as the folders you want to create. You can use the aws s3 cp command to upload such a file.

Or, just upload the files to where you want them to appear, and the folders will magically appear automatically.

CodePudding user response：

  # create bucket

  aws s3 mb s3://main_folder


  # created nested folder
  aws s3api put-object --bucket main_folder --key nested1/nested2/nested3/somefoldertosync

  # sync my local folder to s3

  aws s3 sync /home/ubuntu/somefoldertosync s3://main_folder/nested1/nested2/nested3/somefoldertosync

currently i am using the above way to carry on with my work