Store large amount of data in Javascript-CodePudding

Im receiving some file chunks in bytes format from my server and im collecting them into one variable in my frontend to download it later. And I can't change my server conception (receiving a file sliced into chunks).

My problem is that if the file is heavy (from 500MB), my variable length is starting to be very very big and im having an error :

RangeError: Invalid string length

This is because my variable has reach the limit of character (536 800 000).

Here's how im adding my data to my variable :

this.socket.on('new_file', (data: string) => {
  this.receivedFile  = data;
}

My download part :

public download(): void {
    const byteCharacters = this.receivedFile;
    const byteArrays = [];
    const sliceSize=512

    for (let offset = 0; offset < byteCharacters.length; offset  = 
     sliceSize) {
      const slice = byteCharacters.slice(offset, offset   sliceSize);

      const byteNumbers = new Array(slice.length);
      for (let i = 0; i < slice.length; i  ) {
        byteNumbers[i] = slice.charCodeAt(i);
      }

      const byteArray = new Uint8Array(byteNumbers);
      byteArrays.push(byteArray);
    }

    const blob = new Blob(byteArrays, {type: this.fileInfos.type});

    saveAs(blob, this.fileInfos.name);
  }

Which approach can I do ? Or does it exist some variable type in Javascript to accept more char ? Thanks

CodePudding user response：

Don't collect the chunks into a huge string. Instead, just convert each chunk into a bytearray immediately (that you'll need later anyway) and collect these:

this.socket.on('new_file', (data: string) => {
  const bytes = new Uint8Array(data.length);
  for (let i = 0; i < data.length; i  ) {
    bytes[i] = data.charCodeAt(i);
  }
  this.byteArrays.push(bytes);
}

then

public download(): void {
  const blob = new Blob(this.byteArrays, {type: this.fileInfos.type});
  saveAs(blob, this.fileInfos.name);
}

I don't think you need to make 512-byte-sized slices.

CodePudding user response：

Does stringifying one character at a time and then concatenating the result work?

e.g.

var out="[";
  for(var indx=0;indx<data.length-1;indx  ){
    out =JSON.stringify(data[indx],null,4) ",";
  }
  out =JSON.stringify(data[data.length-1],null,4) "]";

Credit: https://dev.to/madhunimmo/json-stringify-rangeerror-invalid-string-length-3977