I have a simple flow consisting of

GenerateFlowFile ----> PutS3Object ----> Wait

And the generated flow files are getting stored in the bucket correctly. Now I want to add Metadata to my flow file.

If I add a property "Test1" to PutS3Object, it shows up as "X-Amz-Meta-Test1" in the metadata of the object.

But if I add a property "Test2" in GenerateFlowFile it doesn't show up in metadata. I tried adding "Test2" as s3.usermetadata.Test2 but it still didn't work.

Is there a way to pass all the flow files attributes as metadata without explicitly adding properties in the PutS3Object.

CodePudding user response：

PutS3Object only inserts metadata values that you have set as Dynamic Properties on the PutS3Object processor itself. Please see the docs link and look at the Dynamic Properties section.

PutS3Object does not just stick any Attribute you set as metadata, otherwise you would end up with potentially hundreds of metadata entries that you aren't interested in. The only Attribute it reads by default is filename - please see the Reads Attributes section of the docs.

If you have an existing Attibute, and you want to push the value of this Attribute into the metadata, you must add a Dynamic Property to PutS3Object and reference the value of the Attribute.

E.g. you have an Attribute called file_author with a value Steve and you want the S3 object to have the metadata field author with the value Steve:

You would add a Dynamic Property to PutS3Object with a name of author and a value of ${file_author}.

Edit: You could fork PutS3Object into a custom processor to add the dynamic functionality you want, but I would recommend just using the standard PutS3Object config and manually configuring the Attributes you want.