Using TIdHTTP to download file using multiple threads

Question

I'm trying to implement a multi-threaded download file procedure using TIdHTTP.

To achieve this, I'm using the TIdHTTP.Request.Range property to set the part of the file that will be downloaded by each thread. The download procedure is working, but I'm stuck because I don't know how to join the parts into a single file (and not corrupt it).

Below is my code. In this example, I'm downloading the file from the same server using 2 threads, but the idea is to download the same file from different servers, to increase overall download speed. When the threads end, I have two files, each with half the size of the original file. How do I transform these two files into one file?

unit princ;

interface

uses
  Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
  Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.StdCtrls, IdBaseComponent, IdComponent, IdTCPConnection, IdTCPClient, IdHTTP, IdIOHandler, IdIOHandlerSocket,
  IdIOHandlerStack, IdSSL, IdSSLOpenSSL, Vcl.ComCtrls;

type
  TDLThread = class(TThread)
  private
    procedure work(ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
  public
  var
    remoteFile, destination: string;
    currentPart, totalParts: integer;
  protected
    procedure Execute; override;
  end;

type
  TForm1 = class(TForm)
    Button1: TButton;
    IdHTTP1: TIdHTTP;
    IdSSLIOHandlerSocketOpenSSL1: TIdSSLIOHandlerSocketOpenSSL;
    ProgressBar1: TProgressBar;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  var
    processes: integer;    
  var
    dlthread: TDLThread;
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.Button1Click(Sender: TObject);
var
  i, totalParts: integer;
begin
  totalParts := 2;
  for i := 1 to totalParts do
  begin
    dlthread := TDLThread.Create(True);
    dlthread.remoteFile := 'https://download.jgsoft.com/editpad/SetupEditPadLite.exe';
    dlthread.destination :=  'c:\users\admin\desktop\';
    dlthread.totalParts := totalParts;
    dlthread.currentPart := i;
    dlthread.Resume;
  end;
end;

{ TDLThread }

procedure TDLThread.Execute;
var
  idh: TIdHTTP;
  ssl: TIdSSLIOHandlerSocketOpenSSL;
  f: TFileStream;
  remoteFileName, localFileName : string;
  i, fileSize, rangeLength, rangeStart, rangeEnd: integer;
begin
  inherited;
  for i := 1 to length(remoteFile) do
  begin
    if remoteFile[i] = '/' then
      remoteFileName := ''
    else
      remoteFileName := remoteFileName + remoteFile[i];
  end;
  idh := TIdHTTP.Create(Form1);
  idh.OnWork := work;
  ssl := TIdSSLIOHandlerSocketOpenSSL.create(Form1);
  ssl.SSLOptions.Method := sslvSSLv23;
  idh.IOHandler := ssl;
  localFileName := remoteFileName + '_' + IntToStr(currentPart);
  f := TFileStream.Create(destination + localFileName, fmCreate);
  idh.Head(remoteFile);
  fileSize := idh.Response.ContentLength;
  rangeLength := fileSize div totalParts;
  rangeStart := 1 + (rangeLength * (currentPart-1));
  rangeEnd := (rangeLength * currentPart);
  idh.Request.Range := 'bytes=' + IntToStr(rangeStart) + '-' + IntToStr(rangeEnd);
  idh.Get(remoteFile, f);
  idh.Free;
  f.Free;
end;

procedure TDLThread.work(ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
var
  Http: TIdHTTP;
  ContentLength: Int64;
  Percent: integer;
begin
  Http := TIdHTTP(ASender);
  ContentLength := Http.Response.ContentLength;

  if (Pos('chunked', LowerCase(Http.Response.TransferEncoding)) = 0) and (ContentLength > 0) then
  begin
    Percent := 100 * AWorkCount div ContentLength;
    Form1.ProgressBar1.Position := Percent;
  end;
end;

end.

You effectively ask how to copy 2 files into 1 file - are you sure you have no idea about that? See [How do I concatenate two files into one?](https://stackoverflow.com/q/16155038/4299358) — AmigoJack, Nov 30 '22 at 14:36
After i did this question i found this very topic you mentioned, thanks ! — delphirules, Nov 30 '22 at 16:23
On a side note: Your download thread should NOT be setting `form1` as the `Owner` of the `TIdHTTP` and `TIdSSLIOHandlerSocketOpenSSL` objects. Also, your `OnWork` event handler is not thread-safe, it needs to synchronize with the main thread when updating the UI (and, you have multiple threads updating the same progress bar with different statuses, which is just going to confuse users. Use a separate progress bar for each thread). Also, your thread code is not exception-safe, as it leaks memory/resources if an exception is raised (and is leaking `ssl` regardless). — Remy Lebeau, Nov 30 '22 at 18:16
@RemyLebeau Thanks for the info ! What should be the owner of TIdHTTP / TIdSSLIOHandlerSocketOpenSSL ? — delphirules, Nov 30 '22 at 18:34
@delphirules Don't specify any `Owner` for an object you intend to `Free` yourself. So, the `Owner` of the `TIdHTTP` should be `nil`, and the `Owner` of the `TIdSSLIOHandleSocketOpenSSL` should be either the `TIdHTTP` (in which case, you don't need to `Free` the IOHandler at all, let `TIdHTTP` do it) or `nil` (in which ase, you do need to `Free` the IOHandler). — Remy Lebeau, Nov 30 '22 at 19:07

Using TIdHTTP to download file using multiple threads

0 Answers0