TIdHTTP filestream incomplete downloads-CodePudding

I am using TIdHTTP.Get() to retrieve (potentially large) files into a TFileStream, basically;

filestream := TFileStream.Create(destination, fmCreate);
http.HandleRedirects := True;
try
  http.Get(url, filestream);
except
  on E: Exception do Memo.Lines.Add(E.Message);
end;
filestream.Free;

My problem is when/if the user closes/kills the program while the download is taking place. Rather than show the file size as smaller than what it should be, the file is the size of the completed download. Why is that?

I use the file size as a quick check to see if the download was successful, so having it the right size, but wrong contents leads to other errors. I don't want to have to hash checksum each file, as there are a bunch of them, and MD5 checking takes a while on larger multi-GB sized files.

Is there a way to make the destination file size smaller if the download is interrupted for any reason? Some way to not expand the stream to full size unless all of the download has completed OK?

CodePudding user response：

If the file size is known at the start of the download, ie in the HTTP Content-Length header, then TIdHTTP (more specifically, TIdIOHandler.ReadStream()) pre-allocates the stream to the full size to increase I/O performance during the download.

If the download (not the app) is terminated prematurely, ReadStream() has logic to reduce the stream's size to match the actual bytes downloaded. So, in that case, the file should NOT be left at the full size unless the download is complete.

However, if the user kills the app, then that code won't run, so yes, the file will be left at the pre-allocated size. There is currently no option in Indy to disable that pre-allocate logic. There is a TODO item in the code for that, actually.

In the meantime, there is a simple workaround - you can derive a custom class from TFileStream and overwrite its virtual SetSize() method (the Size property's setter) to do nothing. Indy doesn't validate that the requested size is actually applied. As long as the stream's Size and Position property getters work correctly, you should be fine.

type
  TFileStreamNoPreSize = class(TFileStream)
  protected
    procedure SetSize(const NewSize: Int64); override;
  end;
    
procedure TFileStreamNoPreSize.SetSize(const NewSize: Int64);
begin
  // do nothing...
end;

...

filestream := TFileStreamNoPreSize.Create(destination, fmCreate);