TIdHTTP.Head --> "Connection Closed Gracefully"?

Thanks to help from this group, I'm making progress on using the 
TIdHTTP.Head call to determine if a file exists, but it's slow 
sloggin'!!!


I've found that some (but not all) servers politely give you you a "404" 
response code when testing for a file known not to exist.  Google is one 
one the good guys:  Calling TIdHTTP.Head for this address
          http://www.google.com/goofypage.html
produces an exception, with this message:
          HTTP/1.1 404 Not Found

That makes Google a good place to see if my methods are working.  So, I 
make three calls to TIdHTTP.Head for these three addresses:

  http://www.google.com/goofypage.html
  https://www.domainname.com/download?filename=WANTEDFILE.ZIP
     (I have already logged into this password-protected
      site using my browser)
  http://www.google.com/goofypage.html   (same as before)

Here's what happens:

-- The first request to Google returns an exception message of 404 (as 
desired)

-- The second request, using an "https" address to check a file that I 
know is good, produces an exception with only the message, "Connection 
Closed Gracefully"

       Aaaccckk!  How can I determine whether a specified file
       exists or not?

-- The call to Google no longer works!  Calling again with the Google 
address no longer gives 404, but instead produces an exception with the 
message "Socket Error # 10038"

       What's that about?  It worked the first time,
       but not after that second call returned the "Gracefully"
       message.

HELP!!!!

The relevant code is below.

                   **** MANY THANKS!!! ****

Kevin Killion

-----------------------------

The "IOHandler" property of IdHTTP1 is set to 
IdSSLIOHandlerSocketOpenSSL1.


type
  TForm1 = class(TForm)
    ...
    IdHTTP1: TIdHTTP;
    IdSSLIOHandlerSocketOpenSSL1: TIdSSLIOHandlerSocketOpenSSL;
    ...
  public
    procedure DoesFileExist(url: string);
  end;


procedure TForm1.DoesFileExist(url: string);
var
   msg: string;
begin
   msg := 'NO MESSAGE';

   try
      IdHTTP1.HandleRedirects := TRUE;

      IdHTTP1.Head(url);

      msg := IntToStr(IdHTTP1.ResponseCode)
             + ': ' + IdHTTP1.ResponseText;

   except

      on E: Exception do   
         msg := 'EXCEPTION! ' + E.Message;
   end;

   ShowMessage(msg);
end;
0
Kevin
11/20/2008 8:37:24 PM
embarcadero.delphi.winsock 1874 articles. 2 followers. Follow

7 Replies
1918 Views

Similar Articles

[PageSpeed] 51

"Kevin Killion" <kevin@shsmedia.com> wrote in message 
news:46279@forums.codegear.com...

> -- The second request, using an "https" address to check a file
> that I know is good, produces an exception with only the
> message, "Connection Closed Gracefully"
>
>       Aaaccckk!  How can I determine whether a specified file
>       exists or not?

That sounds like a bug.  TIdHTTP is probably trying to read from the socket 
after the server has disconnected it.

Is the exception actually reaching your code?  Or is Indy silently handling 
it internally, and you are just seeing it inside the debugger?

> -- The call to Google no longer works!  Calling again with the
> Google address no longer gives 404, but instead produces an
> exception with the message "Socket Error # 10038"

TIdHTTP got into a bad state, where it doesn't know the socket is no longer 
valid.  Disconnect it, or destroy and recreate the TIdHTTP object, and start 
over.

-- 
Remy Lebeau (TeamB)
0
Remy
11/21/2008 2:13:58 AM
"Kevin Killion" <kevin@shsmedia.com> wrote ...

> > -- The second request, using an "https" address to check a file
> > that I know is good, produces an exception with only the
> > message, "Connection Closed Gracefully"
> >
> >       Aaaccckk!  How can I determine whether a specified file
> >       exists or not?

Remy Lebeau (TeamB) <no.spam@no.spam.com> wrote:
 
> That sounds like a bug.  TIdHTTP is probably trying to read from the socket 
> after the server has disconnected it.
> 
> Is the exception actually reaching your code?  Or is Indy silently handling 
> it internally, and you are just seeing it inside the debugger?


Thanks, Remy.

Yes, it's reaching my code:  I have the call to Head inside a 
TRY...EXCEPT, and the message is coming from the handler in the EXCEPT 
section.

This has me at a dead stop.  What else I can I try/test/attempt to 
resolve this?  Help/thanks!

(Also:  This seems like it should be a straightforward task, yet Indy is 
proving cranky at each and every step.  Is there any alternative to Indy 
that might be more manageable?)

THANKS!!!!!
Kevin

--------------------------------------------------

URL:    https://www.domainname.com/download?filename=WANTEDFILE.ZIP
Call:   IdHTTP1.Head(url);
Result: Exception message is "Connection Closed Gracefully"

--------------------------------------------------

The "IOHandler" property of IdHTTP1 is set to 
IdSSLIOHandlerSocketOpenSSL1.


type
  TForm1 = class(TForm)
    ...
    IdHTTP1: TIdHTTP;
    IdSSLIOHandlerSocketOpenSSL1: TIdSSLIOHandlerSocketOpenSSL;
    ...
  public
    procedure DoesFileExist(url: string);
  end;


procedure TForm1.DoesFileExist(url: string);
var
   msg: string;
begin
   msg := 'NO MESSAGE';

   try
      IdHTTP1.HandleRedirects := TRUE;

      IdHTTP1.Head(url);

      msg := IntToStr(IdHTTP1.ResponseCode)
             + ': ' + IdHTTP1.ResponseText;

   except

      on E: Exception do   
         msg := 'EXCEPTION! ' + E.Message;
   end;

   ShowMessage(msg);
end;
0
Kevin
11/21/2008 2:21:49 PM
"Kevin Killion" <kevin@shsmedia.com> wrote in message 
news:46536@forums.codegear.com...

> This has me at a dead stop.  What else I can I try/test/attempt to
> resolve this?  Help/thanks!

Unless the exception is happening early in the request process, then the 
server's response data still should be getting downloaded before the 
exception occurs.  You can try accessing the TIdHTTP.Response property 
inside your exception handler.  You can check if the exception class type is 
EIdConnClosedGracefully, and if so then ignore it.

Also, do you have the same problem for non-HTTPS URLs? 
TIdSSLIOHandlerSocketOpenSSL has been known to report false timeouts and 
false disconnects, mainly because internally it can't tell whether 0 bytes 
received by OpenSSL means that 0 bytes were decrypted but the connection is 
still alive, or because the other party actually disconnected.  We don't 
know how to get that information from OpenSSL properly yet.  Recent 
experiments with various API functions that OpenSSL exposes have not worked 
correctly yet.

If the problem only occurs when SSL is being used, then you could try 
switching to Eldos SecureBlackBox (http://www.eldos.com), or other 
third-party Indy-compatible SSL implementation, and see if that helps you.

-- 
Remy Lebeau (TeamB)
0
Remy
11/21/2008 6:19:58 PM
Short summary of the problem:

  URL:    https://www.domainname.com/download?filename=GOODFILE.ZIP
  Call:   IdHTTP1.Head(url);
  Result: Exception message is "Connection Closed Gracefully"


Remy Lebeau (TeamB) <no.spam@no.spam.com> wrote:

> You can try accessing the TIdHTTP.Response property 
> inside your exception handler.  You can check if the
> exception class type is EIdConnClosedGracefully,

OK, good suggestion:  I'm now checking E.ClassName, and indeed it *IS* 
EIdConnClosedGracefully!

> and if so then ignore it.



Remy, by "ignore it", do you mean that I should just go ahead and use 
the IdHTTP1.ResponseCode anyway, even though there is this exception?


Because if that is so, then we have another problem:  The call to 
IdHTTP1.Head returns a response code of 200 for that file (which sounds 
good!).  BUT ... when I exit the program and launch it again (to clear 
anything that got smashed) and give it a URL to a file that does NOT 
exist, then I STILL get a ResponseCode of 200!!!

  URL:    https://www.domainname.com/download?filename=NOSUCHFILE.ZIP
  Call:   IdHTTP1.Head(url);
  Result: Exception message is "Connection Closed Gracefully"
          IdHTTP1.ResponseCode = 200 (should be 404!)

So, testing good file throws exception, returns 200,
    testing bad file throws exception, returns 200.
Not much discriminatory power there!

Thanks for your help, Remy.  I know this has to be frustrating for you 
as well in dealing with these Indy quirks.  I'm on deadline to finish a 
task, and it's discouraging to find it so difficult in Delphi to 
determine whether a URL points to an existing file or not.

THANK YOU!!!!!

Kevin
0
Kevin
11/21/2008 8:10:25 PM
"Kevin Killion" <kevin@shsmedia.com> wrote in message 
news:46719@forums.codegear.com...

> I'm now checking E.ClassName, and indeed it *IS* EIdConnClosedGracefully!

Well, that was obvious given the error message that the exception contains. 
What I meant was for your code to check the class type so it can react 
differently if it encounters EIdConnClosedGracefully versus another 
Exception type.  For example:

{code}
try
....
except
    on E: Exception do
    begin
        if not (E is EIdConnClosedGracefully) thn
            msg := 'EXCEPTION! ' + E.Message;
    end;
end;
{code}

Alternatively, you can use a separate 'on' statement to handle 
EIdConnClosedGracefully by itself:

{code}
try
....
except
    on E: EIdConnClosedGracefully do begin end;
    on E: Exception do
        msg := 'EXCEPTION! ' + E.Message;
end;
{code}

> Remy, by "ignore it", do you mean that I should just go ahead
> and use the IdHTTP1.ResponseCode anyway, even though there
> is this exception?

If it has a valid value, then yes.  It is not a fatal error in general.

> Because if that is so, then we have another problem:  The call to
> IdHTTP1.Head returns a response code of 200 for that file (which sounds
> good!).  BUT ... when I exit the program and launch it again (to clear
> anything that got smashed) and give it a URL to a file that does NOT
> exist, then I STILL get a ResponseCode of 200!!!

Did you try clearing the Response values before calling Head()? 
Alternatively, you could just instantiate a new TIdHTTP object for each 
request, and then free it when done.

Have you tried grabbing a stack trace when the exception occurs?  Then we 
can see exactly what code logic is leading up to it.

-- 
Remy Lebeau (TeamB)
0
Remy
11/21/2008 9:38:28 PM
> Well, that was obvious

Well, maybe to some people, but alas, not to me! :)



>> Remy, by "ignore it", do you mean that I should just go ahead
>> and use the IdHTTP1.ResponseCode anyway, even though there
>> is this exception?

> If it has a valid value, then yes.

So, my exception code would be something like:

   except
      on E: EIdConnClosedGracefully do 
          msg := 'Result code is ' + IntToStr(IdHTTP1.ResponseCode);
      on E: Exception do
          msg := 'EXCEPTION! ' + E.Message;
   end;


> Did you try clearing the Response values before calling Head()? 

Do you mean,

        IdHTTP1.Response.Clear;

If so, I tried it and it had no effect.  I still get a "200" return code 
for a nonexistent file.


> Alternatively, you could just instantiate a new TIdHTTP object for each 
> request, and then free it when done.

OK, I've now tried that as well (revised code below).  A test for a 
known good filename and a bad filename both result in the same 200 
return code.  Bummer.


> Have you tried grabbing a stack trace when the exception occurs?  Then we 
> can see exactly what code logic is leading up to it.

I would look at the call stack if I could, but I can't because I'm not 
running under the IDE.  I build the app, then double-click it outside 
the IDE.  I can't run under the IDE because the IDE complains, "Could 
not load SSL library", and I've run out of imagination on where the heck 
the thing is looking:  I've copied ssleay32.dll and libeay32.dll to the 
directory containing the project files, to the target directory for the 
app, and to my System32 folder.  No dice.


-- Kevin


--------------------------------------------------

The problem in a nutshell:

Tests for a known good file and a nonexistent file return same code:

URL:    https://www.domainname.com/download?filename=GOODFILE.ZIP
Call:   IdHTTP1.Head(url);
Result: EXCEPTION (EIdConnClosedGracefully)!
           Connection ClosedGracefully., 200 =  HTTP/1.1 200 OK

URL:    https://www.domainname.com/download?filename=NOSUCHFILE.ZIP
Call:   IdHTTP1.Head(url);
Result: EXCEPTION (EIdConnClosedGracefully)!
           Connection ClosedGracefully., 200 =  HTTP/1.1 200 OK

--------------------------------------------------

procedure TForm1.DoesFileExist(url: string);
var
   msg: string;
   http: TIdHTTP;
   handler: TIdSSLIOHandlerSocketOpenSSL;
begin
   msg := 'No message';

   try
      http := TIdHTTP.Create(NIL);
      handler := TIdSSLIOHandlerSocketOpenSSL.Create(NIL);
      http.IOHandler := handler;
      http.Response.Clear;
      http.HandleRedirects := TRUE;

      try
         http.Head(url);

         msg := IntToStr(http.ResponseCode)
                + ': ' + http.ResponseText;

      except

         on E: EIdConnClosedGracefully do
            BEGIN
               msg := 'EXCEPTION (' + E.ClassName + ')! ' + E.Message
                  + ', ' + IntToStr(http.ResponseCode)
                  + ' =  '   + http.ResponseText;
            END;
         on E: Exception do
            BEGIN
               msg := 'EXCEPTION (' + E.ClassName + ')! ' + E.Message;
            END;
      end;

   finally
      http.Free;
   end;

   ShowMessage(msg);
end;
0
Kevin
11/21/2008 11:54:48 PM
"Kevin Killion" <kevin@shsmedia.com> wrote in message 
news:46781@forums.codegear.com...

> Do you mean,
>
>        IdHTTP1.Response.Clear;

Clear() does not clear the ResponseCode.  You have to do that separately.

> I still get a "200" return code for a nonexistent file.

Something else to keep in mind - not all servers report 404 for non-existing 
files.  Some servers report 200 and send an error page back.  In order to 
handle that, you have to look at the ContentType of the reply.  If you are 
expecting a Zip file, and get an HTML page instead, it is pretty safe to 
assume that the Zip file did not exist, or for whatever other reason had an 
error.

> OK, I've now tried that as well (revised code below).  A test for
> a known good filename and a bad filename both result in the same
> 200 return code.  Bummer.

Since both requests are being performed by completely separate TIdHTTP 
objects, then you are falling into the situation I describe above.  You can 
use a packet sniffer to validate that, though.

> I would look at the call stack if I could, but I can't because I'm
> not running under the IDE.

You can use MadExcept or similar third-party error reporting tool to catch 
Exceptions and generate stack traces.

> I build the app, then double-click it outside the IDE.

If it is on the same machine, then why are you not running it inside the IDE 
debugger?

> I can't run under the IDE because the IDE complains, "Could not
> load SSL library"

Then you probably don't have your OS search paths set up properly, or you 
don't have the OpenSSL DLLs in the right folder, or you have the wrong 
version of the DLLs.

-- 
Remy Lebeau (TeamB)
0
Remy
11/22/2008 12:15:20 AM
Reply:

Similar Artilces:

"Connection closed gracefully" on computers where Delphi is not installed
Hello dear programmers, I am experiencing a problem with an email client application I made using indy TIdSMTP: I didn't experience problems on development computers, where there's Delphi installed. On other computers, while sending mail I get this exception: "Connection closed gracefully" and email is not sent. Maybe I should include some package? Thanks in advance. <Flavio Foglia> wrote in message news:297116@forums.embarcadero.com... > I am experiencing a problem with an email client application I made using > indy TIdSMTP: Which versions of...

When is "close" not really "close"
I've had a strange thing occur a couple of times. When I leave at night, I leave my computer on, but close Thunderbird. That way, I can VPN in from home and view the mail store, but still access e-mail through POP3. One night, a week or so ago, I checked POP3 from home and noticed the mailbox was empty. That's impossible; I get about 500 messages a day. So, I took control of my desktop session and looked around. Thunderbird was not visible, but it was still running in the background. I have since determined that if I close it with the "X" icon at upper ...

"Connection closed gracefully" errors
I am using the TIdHTTP component from Indy 10.5.1 with Delphi 2007. Occasionally my app will be working away and I will get a "Connection closed gracefully" exception during the Post method for no apparent reason. That is, there are no other errors reported. Once, this error is reported, all subsequent attempts to Post will return the same error. The only way to get the application running again is to shut it down and restart it. BTW, I always call the Close method after each Post, successful or not. How can I recover from this error condition? "Stephen B...

"Connection closed gracefully" problem
Hi, I have two Delphi applications, which communicate with Indy / Http. The client runs in a loop and sends several hundred times a request to the server. Requests are sent one by one (all from the same thread). Sometimes the server app needs up to one minute to prepare the response (time in OnCommandGet event). In this case (at least I think so), the client gets a "Connection closed gracefully" exception. If I run the server app in the debugger and stop it in the OnCommandGet event, I can stop it for 2-3 minutes. In this case, the client app is just blocked and...

"submit" to Access DB returns "connection closed"error
The error when I submit data  is as follows: Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.InvalidOperationException: ExecuteNonQuery requires an open and available Connection. The connection's current state is closed.Source Error: Line 17: Dim comDatabase As New System.Data.OleDb.OleDbCommand(strSQL, conDatabase) Line 18: Line 19: comDatabase.ExecuteNonQuery() Line 20:...

Precedence of "where" ("of", "is", "will")?
Nobody on #perl6 today could answer this one. Is: Str | Int where { $_ } the same as: (Str | Int) where { $_ } or: Str | (Int where { $_ }) ? Followup questions, Mr. President: What kind of operators are "where", "of", "is", and "will"? Is there a reason that S03 doesn't list them? What are their precedence(s)? -- Chip Salzenberg - a.k.a. - <chip@pobox.com> Open Source is not an excuse to write fun code then leave the actual work to others. Chip Salzenberg writes: &...

.ALLCOL("%COLUMN%", " ", ", ", ", ")
Do you know anyway for me to exclude a subset of columns returned by this function. We have two columns (rec_user and rec_datetime) which are in all of our tables, but when generating triggers I want automatically generate a script which does not include those two columns but does include all other columns in that table. Bruce I should add that I am using PD 9.0.0.580. Bruce "Bruce Lamb" <lamb.bruce@mayo.edu> wrote in message news:6HgI315nCHA.155@forums.sybase.com... > Do you know anyway for me to exclude a subset of columns returned by this > function. ...

WISH: Undo for "Close All Tabs" or "Close Other Tabs"
It seems that I often mess up and click "close other tabs" or "close all tabs" when I mean to click "Close this tab" Is there anyway we could have a fast undo for this mistake? Thanks On 24/07/2003 12:10, Brian King wrote: > It seems that I often mess up and click "close other tabs" or "close all > tabs" when I mean to click "Close this tab" > > Is there anyway we could have a fast undo for this mistake? Thanks http://multizilla.mozdev.org has this capability. If you right-click on the bar that the ...

Why "Stealth" is better than "Closed"...
Your comments, suggestions are very welcome to strengthen the explanation.. Why "Stealth" is better than "Closed"... There has been an on-going thread in grc.shieldsup about the relative merits of "Stealthing" ports. Some folks here argue the point that a "Closed" port is just as secure as a "Stealthed" port. I've been arguing that "Stealth" is better than "Closed", and I've now put together what I think is a comprehensive explanation as to why that is. I argue that the vast majority of visitors t...

"Close Other Tabs" "bug"
[from: http://forums.mozillazine.org/viewtopic.php?t=69338] Ugh!!!, fricken, it just happened to me *again* writing this damn post!!! ARGH!!!! ok, now i'm rewriting this post for the exact reason i'm posting. the singular most frustrating and repeated experience i have with firefox/mozilla is accidentally selecting the "Close Other Tabs" when i intended to select "Close Tab". i would like a confirmation dialog to appear just as it does when i attempt to close a browser window with multiple tabs. i've searched many posts on the forums and have...

quotes, quotes, quotes...
I am getting this error and I know what is causing it, but I have no idea how to fix it, any help would be great. The script steps through the /var/log/messages file on a linux server and puts The entries into a mysql database. However when it gets to the 'hlt' line in the messages file it just barfs. The single quotes are freaking it out. I know about quotes but not how to use in this situation. Thanks, Paul Error: May 27 17:53:00 localhost kernel: Checking 'hlt' instruction... OK. <----- doesn't like this in the messages file DBD::mysql::st exec...

suggestion: "Close Tabs to the Left", "Close Tabs to the Right"
I would find it useful if in addition to the "Close Other Tabs" command, there were a way to "Close Tabs to the Left" and "Close Tabs to the Right" when you right-click on a particular tab. Particularly when I've exhausted a train of thought that might consist of a number of tabs, it'd be nice to be able to just close all tabs to the right of the last tab before that train of thought. This may be peculiar to the way I abuse tabs but I'm just throwing it out there in case I'm not alone. ...

Using "+" or "||"
Using SQLAnywhere 5.5.04, I've gotten into the habit of using "||" in ISQL to indicate a string concatenation. I needed to paste my SQL statement into the PowerBuilder script painter for some embedded SQL, and PB didn't like the "||" very much at all. I changed it to "+" and it seems to be ok. Do these two operators indicate ~exactly~ the same thing? moin, afaik these two's are not the same! if you're using "||" and any term is NULL then in the resultstring the term will be ignored if you use "+" then the resu...

"-" not "_"
I wrote a SQL statement in the data tab. I wrote a bunch of alaises as example ' word-type ' but when I hit the layout tab it converts the "-" to "_". So now my field name is ' word_type '. Is there any way to prevent this? CardGunner Don' use a hypen ( - ).  It isn't a valid character for column names.   See http://searchsqlserver.techtarget.com/expert/KnowledgebaseAnswer/0,289625,sid87_gci1188931,00.html   Here's an excerpt about column names: Letters as defined in the Unicode Standard 2.0 Decimal numbers from either B...

"Me" is better than "You"
Yes I know, strings are frozen. But let me talk about it, I really can't get through the idea of a PC talkin to me. I consider my PC as an extension of myself, not a dumb companion who addresses Me as You. Yes there are times when I get angry with Him while I work and get wrong calculations etc.., but it really is my fault, Me using wrong istructions and eventually wanting to find someone else to blame, but it's Me. And yes, I consider Thunderbird my mail program, reading my mail on my PC as Me. So I personally like to have Me in the header bar as a compact address ...

Web resources about - TIdHTTP.Head --> "Connection Closed Gracefully"? - embarcadero.delphi.winsock

Resources last updated: 12/2/2015 10:46:36 AM