I have a Spring & Angular application and I developed a service to upload and download files.

The files that will be downloaded are fetched from a folder on my server (so I will use FTP to put the files in the folder) then encoded in base64 and displayed to the user.

The user can download the file and I somehow managed to let the user download large files by dividing the long base64 string into small chuncks of 256 bytes (I tested it locally and it works even if it's still pretty slow).

The problem is in my Spring back-end, i can't compress a base64 string and the only way I have to display the files in my architecture is by returning base64 strings but when it comes to large files I get the following error:

    Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Java heap space

Here's the snippet of the service I'm using to return the base64 string:

    @Override
    public List<FileDto> loadAll() {
        List<FileDto> files = new ArrayList<FileDto>();
        
        try {
            File folder = new File("uploads"  File.separator  "adminfolder");
            File[] listOfFiles = folder.listFiles();
            
            for (int i = 0; i < listOfFiles.length; i  ) {
                String directory = folder.toString()   File.separator   listOfFiles[i].getName();

                String allegato = BufferedImageUtility.encodeFileToBase64Binary(directory);
                String tipoFile = BufferedImageUtility.getMimeType(directory);
                String estensione = "";
                
                files.add(new FileDto("", allegato, listOfFiles[i].getName(), tipoFile, estensione));
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

And here the encodeFileToBase64Binary() method:

    public static String encodeFileToBase64Binary(String fileName) {
            String encodedString = "";
            
            try {
                File file = new File(fileName);
                byte[] bytes = loadFile(file);
                byte[] encoded = Base64.getEncoder().encode(bytes);
                encodedString = new String(encoded,StandardCharsets.US_ASCII);
            } catch (IOException e) {
                logger.debug("errore encodeFileToBase64Binary, ", e);
                throw new EFWMException(e);
            }
            
            return encodedString;
    }

CodePudding user response：

Instead of loading the whole file into memory with loadFile(file), I would suggest you try to stream it into Base64.

Try this for encoding:

  public static String encodeFileToBase64Binary(final String fileName)
     {
        try(final ByteArrayOutputStream out = new ByteArrayOutputStream())
        {
            Files.copy(Path.of(fileName), Base64.getEncoder().wrap(out));
            return out.toString(StandardCharsets.US_ASCII);
        }
        catch (final IOException e)
        {
            logger.debug("errore encodeFileToBase64Binary, ", e);
            throw new EFWMException(e);
        }
     }

CodePudding user response：

As has already been pointed out in the comments, you must send the response in a streaming fashion. Here is a sample rest-controller, which you should adapt to fit your application:

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;

import java.io.InputStream;
import java.nio.file.Files;
import java.util.Base64;

@RestController
public class RestEndpoint {

    @GetMapping("/{file-id}")
    public ResponseEntity<StreamingResponseBody> downloadFile(@PathVariable("file-id") String fileId) {
        // StreamingResponseBody is response type, that allows you to generate the response on the fly
        StreamingResponseBody body = response -> {
            // The Base64 encoder from the standard library allows you to wrap an output stream in orther to do encoding on the fly
            var output = Base64.getEncoder().wrap(response);
            Files.copy(Paths.get("/path/to/file"), output);
            // do not forget to flush() the output
            output.flush();
        };
        return ResponseEntity.ok()
                .contentType("your-content-type")
                .body(body);

    }
}