Amazon S3 - Pipe a string to upload multiple string on S3 Bucket in nodejs-CodePudding

Let's say I have a simple loop like this:


    for (const i=0;i<3;i  ) {
        to(`This counter is ${i}`)
    }

I want to have in my file at the end:

This counter is 0
This counter is 1
This counter is 2

I tried to do that by doing this :


    export class S3Output extends Output {
      #stream: Readable | null = null
      async to(s: string): Promise<void> {
        const params = {
          Bucket: config.aws.bucketName,
          Body: this.#stream,
          Key: 'test.json'
        }
        this.#stream?.pipe(process.stdout)
        this.#stream?.push(s)
        await S3.upload(params).promise()
        return
      }
    
      init(): void {
        this.#stream = new Readable()
        this.#stream._read = function () {};
      }
    
      finish(): void {
        this.#stream?.push(null)
        return
      }
    }

My init function is called at the beginning of the loop, my to function is called every time I want to push a string in the file and the finish function at the end of the loop. This code doesn't push any data, why?

CodePudding user response：

I actually found what was the problem. I had to send the stream once it was over and not while I was doing it. Also no need of pipe.

export class S3Output extends Output {
  #stream: Readable | null = null
  async to(s: string): Promise<void> {
    this.#stream?.push(s)
    return
  }

  init(): void {
    this.#stream = new Readable()
    this.#stream._read = function () {}
  }

  async finish(): Promise<void> {
    this.#stream?.push(null)
    const params = {
      Bucket: config.aws.bucketName,
      Body: this.#stream,
      Key: 'test.json'
    }
    await S3.upload(params).promise()
    return
  }
}