Indy 10 bug report: wrong ReadTimeOut for IdHTTP

I recently upgraded from indy 9 to 10 and found some differences that are a bit surprising:

In Indy 9 if you call TIdHTTP.Post, it will result in a call to TIdCustomHTTP.DoRequest. From this procedure three calls are made:

ReadLn;
FHTTPProto.RetrieveHeaders;
FHTTPProto.ProcessResponse;

All the three above converge eventually to TIdTCPConnection.ReadFromStack. If a ReadTimeOut occurs in ReadFromStack, an exception is raised and execution exists TIdCustomHTTP.DoRequest and TIdHTTP.Post. The result is that a call to TIdHTTP.Post blocks execution for a maximum time of ReadTimeOut.

In Indy 10 a call to TIdHTTP.Post still initiates TIdCustomHTTP.DoRequest. And within TIdCustomHTTP.DoRequest we still have three procedure calls

IOHandler.ReadLn;
FHTTPProto.RetrieveHeaders(MaxHeaderLines);
FHTTPProto.ProcessResponse(AIgnoreReplies);

All the three above end up calling TIdIOHandler.ReadFromSource. 


If the ReadLn fails, one of two things will happen:
Case A- An exception is raised saying: TimeOut. This exception is raised after 3 X ReadTimeOut.
Case B- An exception is raised saying nothing, i.e. the message of the exception is empty !! This exception is raised after 1-3 X TimeOut

Now, in the first case the behaviour is in principle acceptable, except that the timeout takes 3 times as long as in Indy 9. In the second case it is unacceptable that we get an exception that is empty.

I have traced the problem and found an explanation for the problem. In the next runthrough I will use as an example a ReadTimeOut value of 3000 ms:

1- ReadLn is run, from which ReadFromSource is called. 

1a- If ReadLn succeds then RetrieveHeaders and ProcessResponse are run, and since data was retrieved from peer, these succed too and the Post is suvccessful
1b- If ReadLn fails (data is unavailable from peer and ReadFromSource fails) a timeout occurs after 3000 ms. Instead of raising an exception here saying that the process timeouted, RetrieveHeaders is run, and this calls in turn ReadFromSource.

1b1- If data is now available from peer to ReadFromSource, an EIdHTTPProtocolException is raised (from inside RetrieveHeaders ?), but the exception message is empty, so no info about what has happened is presented. Time elapsed: 3000-5999 ms
1b2- If data is still unavailable (peer still not responding), A timeout occurs after 3000 ms. Once again instead of rasing an exception and exiting, ProcessResponse is called and from it again ReadFromSource

1b2a- If data is now available an EIdHTTPProtocolException is raised (from inside ProcessResponse ?), where the message is still empty. Time elapsed: 6000-8999 ms
1b2b- If Peer is still none-responsive, then an exception of type EIdReadTimeOut is raised saying: "Read Time Out" . Total time passed here: 9000 ms

To solve the problem of the strange empty exception of Case B, and the too long timeout interval of case A, we need to do only one thing: If readLn fails raise the EIdReadTimeOut exception immediatley and dont waste time callilng RetrieveHeaders and ProcessResponse. Cause when ReadLn fails the Post has failed anyway, so why waste time calling RetrieveHeaders and ProcessResponse and risk the peer responding to them so some strange EIdHTTPProtocolException with empty message is raised ?


PS: This post is also realted to an earlier post: http://delphi.newswhat.com/geoxml/forumhistorythread?groupname=borland.public.delphi.internet.winsock&messageid=44bea0a1$1@newsgroups.borland.com
0
Rozh
8/25/2008 12:07:18 PM
embarcadero.delphi.winsock 1874 articles. 2 followers. Follow

10 Replies
2327 Views

Similar Articles

[PageSpeed] 12

> {quote:title=Rozh Husain wrote:}{quote}
> To solve the problem of the strange empty exception
> of Case B, and the too long timeout interval of case A,
> we need to do only one thing: If readLn fails raise the
> EIdReadTimeOut exception immediatley and dont waste
> time callilng RetrieveHeaders and ProcessResponse. 

ReadLn() in Indy 9 called ReadFromStack() like this:

  FReadLnTimedOut := ReadFromStack(True, ATimeout, ATimeout = IdTimeoutDefault) = -1;

ReadLn() in Indy 10 calls ReadFromSource() like this:

  FReadLnTimedOut := ReadFromSource(True, ATimeout, False) = -1;

Notice the last parameter?  Would changing Indy 10 to behave like Indy 9 solve your issue?

  FReadLnTimedOut := ReadFromSource(True, ATimeout, ATimeout = IdTimeoutDefault) = -1;


--
Remy Lebeau (TeamB)
0
Remy
8/29/2008 8:28:30 AM
Dear Remy Lebeau (TeamB)

I am not sure about this. It depends of course on what the extra parameter is doing.
But I am thinking that the problem lies elsewhere. I will try to clarify:

As I wrote earlier, when ReadLn is called (and peer is none-responsive) a timeout will occur after ReadTimeOut, as it should. One would expect, when this timeout occurrs, that an exception would be raised right away saying: TimeOut and so forth. However, while this is what happens in Indy 9, this is not the case in Indy 10. When the mentioned timeout occurs, execution is continued with calls to RetrieveHeaders and ProcessResponse (which in turn both call ReadFromSource). Not until RetrieveHeaders and Proc
essResponse are finished (resulting in extra wait of 2 X ReadTimeOut), the exception is raised saying: TimeOut. So the result is that we have waited 3 X ReadTimeOut to get the exception, where the exception could be raised after 1 X ReadTimeOut. 

If Peer responds while RetrieveHeaders or ProcessResponse are running, an exception is raised right away saying nothing, i.e. an empty exception. I figured that the reason for this is that once ReadLn fails, RetrieveHeaders and ProcessResponse won’t know how to deal with the input and some unanticipated exception happens.

To solve all these problems, we have to do one thing: raise right away exception saying timeout, when ReadLn fails.

This is a summary of the problem. I hope it is not as cryptic as my last post, and easier to understand.

Kind Regards

Rozh
0
Rozh
8/31/2008 2:27:26 PM
<Rozh Husain> wrote in message news:8865@forums.codegear.com...

> I am not sure about this. It depends of course on what the extra parameter
> is doing.

It tells ReadFromStack/Source() whether to raise an exception if a timeout 
occurs.  In Indy 9, ReadLn() is telling ReadFromStack() not to raise a 
timeout exception only if the ATimeout parameter was set to 
IdTimeoutDefault.  In Indy 10, ReadLn() is telling ReadFromSource() never to 
raise a timeout exception.

> But I am thinking that the problem lies elsewhere.

That is not what you said earlier.  You narrowed it down to ReadFromStack() 
specifically, and I agreed why that is the case.

> As I wrote earlier, when ReadLn is called (and peer is none-responsive) a
> timeout will occur after ReadTimeOut, as it should. One would expect, when
> this timeout occurrs, that an exception would be raised right away saying:
> TimeOut and so forth.

Which is exactly what the code I showed you is doing in Indy 9, but not in 
Indy 10, because of the change in values being passed to ReadFromSource(). 
That is why I asked you to try changing those values back to the Indy 9 
behavior, recompile Indy 10, and see if that solves your problem.

> However, while this is what happens in Indy 9, this is not the case in 
> Indy 10.

Because ReadLn() in Indy 10 is specifically telling ReadFromSource() not to 
do it, whereas ReadLn() in Indy 9 did.


Gambit
0
Remy
9/2/2008 5:09:29 PM
Ok, so the problem seems to be with the call to ReadFromSource afterall. 

But is there any plans from the Indy team to fix this error ?
0
Rozh
9/3/2008 8:34:40 PM
<Rozh Husain> wrote in message news:10180@forums.codegear.com...

> Ok, so the problem seems to be with the call to ReadFromSource afterall.
>
> But is there any plans from the Indy team to fix this error ?

Well, have you tried making the change I suggested yet?  Did it solve the 
problem you are having?  I'm not going to check in a change to Indy's source 
code without confirmation first.

-- 
Remy Lebeau (TeamB)
0
Remy
9/3/2008 11:57:50 PM
I have now tried for quite some time, but with no luck. I cant make it work. I simply dont know how to use the extra parameter.

If I get lucky with some working code in the future, I will post it here. 

I however think that the Indy team should be notified about this error !
0
Rozh
9/15/2008 12:43:59 PM
<Rozh Husain> wrote in message news:15696@forums.codegear.com...

> I have now tried for quite some time, but with no luck. I cant
> make it work. I simply dont know how to use the extra parameter.

I already show you how.  Did you, or did you not, already try the exact code 
I gave you earlier?  If yes, then what was the exact result of doing so?

-- 
Remy Lebeau (TeamB)
0
Remy
9/15/2008 4:58:13 PM
Hello again

I am sorry for this late reply, but I abbandoned this thread at the time, cause a got stuck and was unable to move further. I have now finally understood the proposed solution.
So it is a bug in Indy10 that the ReadTimeout value is not affectuated, but in fact multiplied by 3. I have now revised what Remy wrote and inserted the proposed modification to the code, and now it works fine :-)

However I donot know wheather this will have negative sideeffects on the function of others parts of Indy. And besides would it not be better to write 
FReadLnTimedOut := ReadFromSource(True, ATimeout, True) = -1;
instead of
FReadLnTimedOut := ReadFromSource(True, ATimeout, ATimeout = IdTimeoutDefault) = -1;
Since the last parameter will always evealuate to true anyway ???

I hope the indy team will commit the fix into the sourcecode.
0
Rozh
8/5/2009 12:32:28 AM
> {quote:title=Rozh Husain wrote:}{quote}

> So it is a bug in Indy10 that the ReadTimeout value is not affectuated, but in fact multiplied by 3.

I do not understand what you are describing.  Please clearify.

> {quote:title=Rozh Husain wrote:}{quote}

> would it not be better to write 
> FReadLnTimedOut := ReadFromSource(True, ATimeout, True) = -1;
> instead of
> FReadLnTimedOut := ReadFromSource(True, ATimeout, ATimeout = IdTimeoutDefault) = -1;
> Since the last parameter will always evealuate to true anyway ???

No, it would not evaluate to True if ReadLn() were called with a non-infinite timeout.
0
Remy
8/10/2009 6:47:12 PM
>I do not understand what you are describing. Please clearify.

Sure.
In Indy 10 a call to TIdHTTP.Post Iinitiates TIdCustomHTTP.DoRequest. And within TIdCustomHTTP.DoRequest we have three procedure calls

IOHandler.ReadLn;
FHTTPProto.RetrieveHeaders(MaxHeaderLines);
FHTTPProto.ProcessResponse(AIgnoreReplies);

All the three above end up calling TIdIOHandler.ReadFromSource. 

If peer is noneresponsive, then a timeout occurs within ReadfromSource on first call, but no exception is raised (as it is now). So execution goes on to RetrieveHeaders and there is a second call to ReadfromSource and then a third call via ProcessResponse. Only then then the timeout exception is raised, so if you e.g. specify a timeout of 1 minut, it occurs infact after 3 minutes.

This is fixed if you change the call from within ReadLn to 
FReadLnTimedOut := ReadFromSource(True, ATimeout, ATimeout = IdTimeoutDefault) = -1;

I do however not know whether this will result i faulty behaviour elswhere...
0
Rozh
8/10/2009 9:27:51 PM
Reply:

Similar Artilces:

Converting Delphi 2007 Indy 10.2.3 to Delphi 2009 Indy 10.5.5 [Edit]
Hello, I am currently attempting to port over a Delphi 2007 project that uses Indy 10.2.3 (very successfully) to Delphi 2009 and Indy 10.5.5 (I just got the latest development build this morning). I think I am running into an encoding issue, but am not sure. Specifically, IDHTTP with SSL calls an old CGI and the CGI returns a .zip file and I then save it to the disk. In 2007 and before this worked perfectly. In 2009, it is not. Here is the examples of the 2 different results (though cut way short in the post) I am getting back: 2007: 'PK'#3#4#$14#0#0#0#8#0'rLQ9žrPb€'#0...

Indy 10 bug: ReadTimeOut
I first wrote about this bug a year ago. At the time I narrowed the problem down to a single procedure call, but was at the time unable to provide a solution The original thread is here https://forums.codegear.com/thread.jspa?threadID=1532 The bug was never fixed. I have recently updated the above thread with a possible solution. I hope that one of the Indy team members would commit the fix to the source code, so everyone can benefit from it :-) Kond regards ...

migrating from Delphi 6 With Indy 10 to XE7 with Indy 10
I updated the original Indy in D6 to version 10 several years ago. Now I want to migrate my application from D6 to XE7 and would like some feedback on the best route to take. I usually send data using readln and writeln statements. The data is typically XML format. Since migrating to XE7 will include potential unicode data what is the best approach to take when reading and writing data? Will writeln and readln work in these cases or should I be using a different strategy to send unicode data between the tidtcpclient and tidtcpserver applications? al wrote: > I usually send data ...

Indy bug report: wrong declarations
Hi, strictly Indy related, I tried registering on the official forums but apparently failed all attempts to answer the magic questions... perhaps I *am* a bot? Anyway, this is a bug report. The following declarations are wrong: First, in IdWinsock2.pas, "TAddrInfoExW = ADDRINFOEXA;". Should be ADDRINFOEXW of course. This is especially annoying since "PADDRINFOEXW = ^TAddrInfoEXW;", which then affects the ai_next member. Second, in IdWship6.pas, {code} {$IFDEF UNICODE} FreeAddrInfoEx : LPFN_FREEADDRINFOEX = nil; {$ELSE} FreeAddrInfoEx : LPFN_FREEADDR...

delphi 7 Indy 9 and Indy 10
Hi, I can upgrade to indy 10 in delphi 7. But I have discover that indy 9 and Indy 10 have some different properties, so i had to change my old programs that were made in indy 9 to upgrade it to new version, but it is a long work. So I want to know if it is possible to install in the same delphi 7 both versions indy 10 and indy 9. I have tried to do it but i received a error message: Cannot load package 'IndySystem70'. It contains unit 'IdWinSock2', which is also contained in package 'Indy70'. Some can give me ideas or a link to read how to install both versi...

Delphi 2009
Hello, I am using Delphi 2009 with a recent Indy 10.5.5 development snapshot. In my code I am using a TIDHttp with TIDSSLIOHandlerSocketOpenSSL. The Open SLL .dlls I am using is from the the indy website and are openssl-0.9.8k-i386-win32. The issue is, randomly, I am getting a socket error # 0 error. I never got this error with Delphi 2007 and Indy 10.2.3. Any help would be GREATLY appreciated! Thank you. The code looks something like this: var sHttpResult, sVar, sParam: string; ssHttpResult : TStringStream; slPostData : TStringList; begin IdSSL.CheckForDi...

A bug report on the bug reporting process.
Name: David B. Hedrick Email: davidhedrickatearthlinkdotnet Product: Other (please state) Summary: A bug report on the bug reporting process. Comments: Dear Mozillians, This is a bug report on the bug reporting process. The hope is that my feedback can make the process smoother. I am a long-time computer user, not a programmer. This bug occurred on a Win XP SP 2 system. My bug is that the contents of folders are partially illegible. It appears that the even-numbered entries are each over-written by the one above leaving the text over-printed. Stepping down thr...

Fails to send bug report through bug report system and mIRC bug
Name: Justin A Email: thisisnotgeorgebush_at_gmail.com Product: Deer Park Summary: Fails to send bug report through bug report system and mIRC bug Comments: Error I got: Error Details url must use a valid URL syntax http://domain.tld/foo Also when clicking a link through mIRC it adds a file///C:\Program Files\mIRC\ to the begining of the URL. So if im clicking www.redbull.com through mIRC, it would try to load file///C:\Program Files\mIRC\www.redbull.com Good product so far :] Browser Details: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b3) Gecko/20050...

Indy E-Mail Problem (Indy 10, Delphi 2007)
Hello, we use Indy 10 with Delphi 2007 to get E-Mail using IMAP. We recieved an E-Mail which is not handled correctly. The E-Mail looks like this {code} Return-Path: <test@mail.com>; From: "Test Tester" <test@mail.com>; To: <test@mail.com>; Subject: Order Date: Wed, 7 Mar 2012 16:50:40 +0100 Message-ID: <15D526BEB8091D43859549D9E16E370FBA3123> MIME-Version: 1.0 Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQFk6K6fcF3...

Delphi 2009 automatic bug reports appear as Delphi.NET issues
For automatic bug reports, Delphi 2009 seems to put 'Delphi.NET' into the field for 'Project'. I will try to verify it and enter it in QC. Unfortunately this means I have to use the QC database for 'testing', but will delete my test entries if I have verified the problem. Michael Justin Michael Justin wrote: > For automatic bug reports, Delphi 2009 seems to put 'Delphi.NET' into > the field for 'Project'. AFAIK this is as expected. There is no project "RAD Studio" and so "Delphi.NET" has been choosen as proj...

comparing Cystal Report 10.2.3(provided by VS 2005) and Crystal Report 10.0.3(provdied by Crystal reports 10
 hi all, i was just wondering what u guys think about the two version (10.2.3 - by VS2005) and (10.0.3 - by Crystal reports 10).it seems like the Coding syntax is abit diff for the two version. (10.2.3 dont even need to mappath) i was wondering if i write the web application using 10.2.3, would i encounter problems when displaying report written by 10.0.3. opinions need, thanks! regards,martin ...

Bug Report #10
Name: Zach Email: ZachSka87atgmaildotcom Product: Minefield Summary: Bug Report Comments: After using Minefield for a few hours, all of my toolbar buttons have simply disappeared. Browser Details: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9b2pre) Gecko/2007112105 Minefield/3.0b2pre ...

Bugs in Indy 10?
I updated my Indy 10 components a few weeks ago, and without making any source code changes, recent users that have loaded the new build are experiencing random hangs. I think it may be occurring during a synchonize in my thread. this thread uses Indy for uploading files to an FTP server, and also accesses a web server for something else. The Synchronize runs a procedure in my main thread to output a log file entry saying an FTP upload error has occurred. It seems to only coincide when items are being automatically loaded into a listview at :45 every hour in my main thread, using a Timer...

Bug Report #10
Name: Renata Almeida Email: rdotvilarinhoatyahoodotcomdotbr Product: Firefox Summary: Bug Report Comments: Whenever Firefox is closed, I lose all my favorite links stored in the favorites bar... It is like it can't retain the information once the pc is turned off. Browser Details: Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9b5) Gecko/2008032620 Firefox/2.0.0.14;MEGAUPLOAD 1.0 From URL: http://hendrix.mozilla.org/ ...

Web resources about - Indy 10 bug report: wrong ReadTimeOut for IdHTTP - embarcadero.delphi.winsock

Issues - gdata-java-client - Google Data Java Client Library - Google Project Hosting
My favorites ▼ - Sign in gdata-java-client Google Data Java Client Library Project Home Downloads Wiki Issues Source New issue Search Search ...

A first chance exception of type 'System.IO.IOException' occurred in System.dll - Pastebin.com
PASTEBIN - #1 paste tool since 2002 create new paste tools api archive faq PASTEBIN create new paste trending pastes sign up login my alerts ...

Mads Klinkby’s home
Life in general and technology in particular. (by Mads Klinkby)

[ruby-core:43751] [ruby-trunk - Feature #6088] Add Net::ReadTimeout to distinguish which operation failed ...
... by naruse (Yui NARUSE).I don't object the direction but it should have more document about the new Exception.Feature #6088: Add Net::ReadTimeout ...

Resources last updated: 12/11/2015 2:54:47 AM