This question is maybe more a tip for people to search a solution if they have the same problem (as I found the solution eventually).

I had an application that does some HTTP requests with a local server (a mix of GET/POST with JSON content in the request/response bodies). The server is a third-party application, and after I upgraded it to a recent version, my Delphi app was no longer working. It turned out that it was now hanging on the statement:

IdHTTP.Post("URL", "Payload", "BytesStreamResult");

As a manual POSTMAN request was still working, it had to be on the Delphi client side.

Further isolating the issue showed that the HTTP POST request did get an HTTP 200 response with valid HTTP response headers, but then was getting stuck reading the response body. It was hanging on:

IOHandler.ReadLn  

When I compared the headers with the POSTMAN response, I noticed that 'Transfer-Encoding: chunked' was missing in the Delphi response.

Finally, I noticed the code related to TIdHTTP's hoKeepOrigProtocol option, which is not set by default. So, my POST request was "downgraded" to an HTTP 1.0 request, and I guess this now made the (updated) server to respond differently (I'm not an RFC expert, but I guess 'chunked' is maybe an HTTP 1.1 option only).

After setting this option, everything worked like before (and indeed, the response was now read as "chunked" in Delphi).

Summary:

1. Shouldn't hoKeepOrigProtocol be the default option? (why punish good citizens for those that are not...)
2. Can we intercept this? Now my POST is assuming upfront a streamed response and thus it hangs because the server doesn't write anything to the buffer. What would that high-level code look like? As it seems a mix of interpreting the header response headers and then deciding if more response reading is required.
   (it didn't do anything specific regarding time-outs, either. I have the impression it hangs forever, or at least > 10 minutes...)

CodePudding user response：

TIdHTTP supports non-chunked responses (which yes, is an HTTP 1.1 feature), so the hanging would have to be caused by the server sending a malformed response (a bug that should be reported to the server author).

When reading a non-chunked and non-MIME response, TIdHTTP does not use IOHandler.ReadLn to read the response's body, as you claim. Only when reading the response's headers.

But, since you did not show what the response actually looks like, nobody can explain for sure exactly why the hang occurs.

Shouldn't hoKeepOrigProtocol be the default option?

At the time the option was first introduced, no. There were plenty of buggy HTTP 1.1 servers around that downgrading to HTTP 1.0 was warranted.

However, that was many years ago. Nowadays, HTTP 1.1 is much more mature, and such buggy servers are rare. So, feel free to submit a change/pull request to Indy's GitHub repo if you feel the default behavior should be changed.

Can we intercept this?

No. The behavior you describe is most likely caused by a bug in the HTTP server. Either it is not sending all of the data it should be, or else the response is likely malformed in a way that makes TIdHTTP expect more data than is actually being sent. Either way, all you can do is assign a non-infinite timeout to TIdHTTP.

it didn't do anything specific regarding time-outs, either. I have the impression it hangs forever, or at least > 10 minutes.

Indy is designed to use infinite timeouts by default. You can assign custom timeouts to TIdHTTP's ConnectTimeout and ReadTimeout properties.

CodePudding user response：

Setting this prevent the HTTP protocol downgrade:

IdHTTP.HTTPOptions := IdHTTP.HTTPOptions   [hoKeepOrigProtocol];

This is, of course, dependant upon how the server processes the protocol specification, and if it results in issues or not.