indy 9 ---> indy 10 cpu

I use delphi 5, indy 9.0.50 and indy 10_4806

I have used indy 10 in my some projects before.
And I decided to covert my indy 9 server program to indy 10.

Same code and same system but indy 10 uses alot of cpu if the connection count is greater than 200

indy 9 Server : 900 connections - cpu usage: %0
indy 10 server: 500 connections - cpu usage: %30


I only change below codes:

- Server.ThreadClass := TMypeerthread ---> Server.ContextClass := TMypeerthread

- TMyPeerThread = class(TIdPeerThread) ----> TMyPeerThread = class(TIdServerContext)

- constructor TMyPeerThread.Create(ACreateSuspended: Boolean = True);override; ----> 
 
     constructor TMyPeerThread.Create( AConnection: TIdTCPConnection; AYarn: TIdYarn; AList: TThreadList = nil); override;

- Server.Threads.LockList --->Server.Contexts.LockList 

- Athread.Connection.ReadTimeout := 25000; ---> AContext.Connection.IOHandler.ReadTimeout := 25000

- Athread.Connection.InputBuffer.Clear; ----> AContext.Connection.iohandler.InputBuffer.Clear;

  bb: byte;
- Athread.Connection.ReadBuffer( bb, sizeof( bb ) ) ----> bb := AContext.Connection.IOHandler.ReadByte;

  myint:integer;
- Athread.Connection.ReadBuffer( myint, sizeof( myint ) ); ----> myint := AContext.Connection.IOHandler.ReadLongInt( False );

- Athread.Connection.ReadStream( Stream, myint ); ---> AContext.Connection.IOHandler.ReadStream( stream, myint );


- TMypeerthread: Connection.Writestream( Stream ); ---> Connection.iohandler.Write( stream );

  (tidtcpserver onexecute)
- if not Athread.Connection.IOHandler.Readable( 160 ) then Exit; ---->

    if AContext.Connection.IOHandler.InputBufferIsEmpty then
     begin
       if not AContext.Connection.IOHandler.CheckForDataOnSource( 160 ) then Exit;
     end;
0
mustafa
10/22/2012 7:04:07 PM
embarcadero.delphi.winsock 1874 articles. 2 followers. Follow

8 Replies
2409 Views

Similar Articles

[PageSpeed] 10
Get it on Google Play
Get it on Apple App Store

mustafa wrote:

> I have used indy 10 in my some projects before.
> And I decided to covert my indy 9 server program to indy 10.
> Same code and same system but indy 10 uses alot of cpu if the
> connection count is greater than 200

Without seeing your actual code, it is hard to diagnose issues like this. 
 I strongly suggest you use a profiler, such as AQTime, to track exactly 
where your code is spending its time and CPU usage.

--
Remy Lebeau (TeamB)
0
Remy
10/22/2012 9:00:07 PM
procedure ServerExecute(AContext: TIdContext);
........
Ath := TMypeerthread( AContext );

........

Ath.SendQueue;

if AContext.Connection.IOHandler.InputBufferIsEmpty then
begin
  if not AContext.Connection.IOHandler.CheckForDataOnSource( 160 ) then Exit;
end;

fP := AContext.Connection.IOHandler.ReadByte;
.......


For above structure:
Can we use only "if AContext.Connection.IOHandler.Readable( 160 ) = false then exit;"  instead of this:
if AContext.Connection.IOHandler.InputBufferIsEmpty then
begin
  if not AContext.Connection.IOHandler.CheckForDataOnSource( 160 ) then Exit;
end;

Becase I want to call SendQueue if there is no data else read the data.
But if there is any data: CheckForDataOnSource()---->ReadFromSource()--->
.....
if SourceIsAvailable then begin
......// ---->I dont need these lines for my structure because already I will call this function: fP := AContext.Connection.IOHandler.ReadByte;
end;
0
mustafa
11/2/2012 12:36:57 AM
mustafa wrote:

> Can we use only "if AContext.Connection.IOHandler.Readable( 160 ) =
> false then exit;"  instead of this:
> if AContext.Connection.IOHandler.InputBufferIsEmpty then
> begin
> if not AContext.Connection.IOHandler.CheckForDataOnSource( 160 )
> then Exit;
> end;

I would not suggest that.  Readable() only tells you if the underlying socket 
has new data waiting to be read.  It does not tell you if the InputBuffer 
already has pending unread data from an earlier read operation.  When Indy 
reads data from the socket into the InputBuffer, it reads all available data 
and buffers everything it reads.  All reading operations in the IOHandler 
first check the InputBuffer for available data before resorting to accessing 
the underlying socket to read new data.

> Becase I want to call SendQueue if there is no data else read the
> data.

I would stick with the CheckForDataOnSource() approach.  Not only does it 
call Readable() for you, but it also reads new data into the InputBuffer 
if it arrives.

> But if there is any data:
> CheckForDataOnSource()---->ReadFromSource()--->
> 
> ....
> 
> if SourceIsAvailable then begin
> 
> .....// ---->I dont need these lines for my structure because already
> I will call this function: fP :=
> AContext.Connection.IOHandler.ReadByte;

ReadByte() blocks the calling thread until a Byte becomes available in the 
InputBuffer, reading new data from the socket if needed.  If you use Indy's 
default ReadTimeout of IdTimeoutInfinite, ReadByte() will prevent you from 
calling SendQueue() in a timely manner if the client is not sending bytes 
at regular intervals.  By using the CheckForDataOnSource() approach, you 
only call ReadByte() when you know for sure that there is at least 1 byte 
waiting in the InputBuffer, while still allowing you timely intervals to 
call SendQueue().

--
Remy Lebeau (TeamB)
0
Remy
11/2/2012 1:34:35 AM
I think readable() of indy 9 works different than indy 10.

I need readable() of indy 9 :)
Because CheckForDataOnSource increase cpu usage if alot of connection receives data at the same time.
Because if any data available ReadFromSource() function every time use this line: SetLength(LBuffer, RecvBufferSize);

indy 9 cpu usage: %0
indy 10 cpu usage: %30   for 600 connection.

For my structure Alternatively, what I can use instead of below code:

if AContext.Connection.IOHandler.InputBufferIsEmpty then
begin
  if not AContext.Connection.IOHandler.CheckForDataOnSource( 160 ) then Exit;
end;
0
mustafa
11/2/2012 1:57:14 AM
mustafa wrote:

> I think readable() of indy 9 works different than indy 10.

No, it does not.  Readable() works exactly the same way in both versions, 
in fact their implementations are almost identical code.  In Indy 9, IOHandler.Readable() 
simply calls TIdSocketHandle.Readable(), which calls TIdStack.WSSelect(), 
which calls the socket API select() function.  In Indy 10, IOHandler.Readable() 
calls TIdSocketHandle.Readable(), which calls TIdSocketHandle.Select(), which 
calls TIdSocketList.SelectRead(), which calls the socket API select() function. 
 I assure you, the end result is exactly the same - Readable() only checks 
the underlying socket and reports if the socket has new data waiting to be 
read from its internal buffer.  Nothing more.

> I need readable() of indy 9 :)

No, you don't, actually.  You should not be using Readable() directly to 
begin with.  You should be using TIdTCPConnection.InputBuffer.Size and TIdTCPConnection.ReadFromStack() 
in Indy 9, and TIdIOHandler.InputBufferIsEmpty() and TIdIOHandler.CheckForDataOnSource() 
in Indy 10.  Calling Readable() directly can lead to false negatives in both 
versions if data has already been read into the InputBuffer and the socket 
is empty, causing you to ignore data that you could be acting on.  If you 
are relying on Readable() alone in Indy 9, then your Indy 9 code is buggy. 
 But if you absolutely must have the same behavior in Indy 10 as you do in 
Indy 9, then go back to using Readable() directly (even though it is wrong):

{code:delphi}
- if not Athread.Connection.IOHandler.Readable( 160 ) then Exit; ----> if 
not AContext.Connection.IOHandler.Readable( 160 ) then Exit; ---->
{code}

> Because CheckForDataOnSource increase cpu usage if alot of connection
> receives data at the same time.

If lots of connections are receiving data at the same time, then CheckForDataOnSource() 
is going to exit very quickly and allow the TIdContext threads to perform 
a lot of socket reading operations at the same time.  That is fine.  Your 
misuse of Readable() was likely throttling that behavior, actually making 
your code run slower than it needed to be.  There is nothing wrong with having 
30% CPU utilization.  It means your app is actually doing something.  If 
you don't like it, you could always just put some calls to Sleep() in your 
reading code to slow it down again.

> Because if any data available ReadFromSource() function every time use
> this line: SetLength(LBuffer, RecvBufferSize);

So what?  It should not be taking 30% CPU utilization just to allocate a 
few memory buffers.  Before you start placing blame, please use a profiler, 
like AQTime, to actually measure where your app is actually spending its 
time and CPU usage.

--
Remy Lebeau (TeamB)
0
Remy
11/2/2012 6:11:11 AM
Remy wrote:

> Calling Readable() directly can lead to false negatives in both versions if
> data has already been read into the InputBuffer and the socket is empty,
> causing you to ignore data that you could be acting on.

If you are going to use Readable() directly, you should at least be checking 
if the InputBuffer is empty first, in both versions, eg:

Indy 9:
{code:delphi}
Ath.SendQueue;

if AThread.Connection.InputBuffer.Size = 0 then begin
  if not Athread.Connection.IOHandler.Readable( 160 ) then Exit;
end;

AThread.Connection.ReadBuffer(fP, sizeof(fP));
{code}

Indy 10:
{code:delphi}
Ath.SendQueue;

if AContext.Connection.IOHandler.InputBufferIsEmpty then begin
  if not AContext.Connection.IOHandler.Readable( 160 ) then Exit;
end;

fP := AContext.Connection.IOHandler.ReadByte;
{code}

--
Remy Lebeau (TeamB)
0
Remy
11/2/2012 6:18:00 AM
I use indy 9.0.50

I use directly for indy 9: if not Athread.Connection.IOHandler.Readable( 160 ) then Exit;

And has not been a problem for years.

And 5 years ago you has advised me this using  :)
https://groups.google.com/group/borland.public.delphi.internet.winsock/browse_frm/month/2007-4?scoring=d&
this is your code code: :)
>Then use the connection's IOHandler.Readable() method, ie: 
 >   procedure TForm1.ServerExecute(AThread: TIdPeerThread); 
 >   var 
 >       b: Byte; 
 >       AThread2: TMyPeerThread; 
 >       Stream: TMemoryStream; 
 >   begin 
 >       TMyPeerThread(AThread).SendQueue; 
 >       if not AThread.Connection.IOHandler.Readable(10) then Exit; 
 >       AThread.Connection.ReadBuffer(b, SizeOf(b)); 
 >       ... 
 >   end; 



I tried to use directly Readable with indy 10 but all clients could not work properly and crashed.



>if AContext.Connection.IOHandler.InputBufferIsEmpty then begin
>  if not AContext.Connection.IOHandler.Readable( 160 ) then Exit;
>end;

I will try this.


Ok i will try profiler if there is free one.

%30 cpu usage very important for me. 
Because:
1- When cpu usage of the server increase client programs start to working very slow.
2- 7 programs working on the same server.And each program has 1000 connections. :)  7 * %30
0
mustafa
11/2/2012 7:57:06 AM
Hello mustafa,

> I use indy 9.0.50
> 
> I use directly for indy 9: if not
> Athread.Connection.IOHandler.Readable( 160 ) then Exit;
> 
> And has not been a problem for years.

Then you have been using it the wrong way for years.  Here is a simply example 
of how it can fail:

Say the client sends 2 messages consisting of 1 integer each, for a total 
of 8 bytes.  All of the bytes (or at least more than 4) arrive on the server's 
TCP stack before the server code decides to read the first message.  OnExecute 
calls Readable(), which returns True since there is data waiting on the socket, 
so OnExecute calls ReadBuffer(4) to read the first message and then exits. 
 ReadBuffer() would have read all available bytes into the InputBuffer and 
then returned just the first 4 bytes, leaving the remaining bytes in the 
InputBuffer for a future read.  The next time OnExecute is called, it calls 
Readable(), but this time it returns False because there is no more data 
on the socket, but the InputBuffer still has data in it.  Until the client 
sends another message, those bytes just sit in the InputBuffer.  Eventually, 
the client sends a third message, at which time Readable() returns True, 
and OnExecute calls ReadBuffer(4), which returns the second message, not 
the third message.

Now compound that if the client is sending multiple messages in a short amount 
of time.  If you call Readable() without checking the InputBuffer first, 
you can easily fall way behind in your message processing.  You are relying 
on the wrote state machine to tell you when data is available to process.

> And 5 years ago you has advised me this using  :)
> 
> https://groups.google.com/group/borland.public.delphi.internet.winsock/browse_frm/month/2007-4?scoring=d&
> 
> this is your code code: :)

That was a long time ago, and that advice was wrong.  It should be more like 
this instead:

{code:delphi}
procedure TForm1.ServerExecute(AThread: TIdPeerThread);
var
  b: Byte;
  ...
begin
  TMyPeerThread(AThread).SendQueue;
  if AThread.Connection.InputBuffer.Size = 0 then begin
    if AThread.Connection.ReadFromStack(True, 160, False) <= 0 then Exit;
  end;
  AThread.Connection.ReadBuffer(b, SizeOf(b));
  ...
end;
{code}

--
Remy Lebeau (TeamB)
0
Remy
11/2/2012 5:16:30 PM
Reply:

Similar Artilces:

delphi 7 Indy 9 and Indy 10
Hi, I can upgrade to indy 10 in delphi 7. But I have discover that indy 9 and Indy 10 have some different properties, so i had to change my old programs that were made in indy 9 to upgrade it to new version, but it is a long work. So I want to know if it is possible to install in the same delphi 7 both versions indy 10 and indy 9. I have tried to do it but i received a error message: Cannot load package 'IndySystem70'. It contains unit 'IdWinSock2', which is also contained in package 'Indy70'. Some can give me ideas or a link to read how to install both versi...

Converting Delphi 2007 Indy 10.2.3 to Delphi 2009 Indy 10.5.5 [Edit]
Hello, I am currently attempting to port over a Delphi 2007 project that uses Indy 10.2.3 (very successfully) to Delphi 2009 and Indy 10.5.5 (I just got the latest development build this morning). I think I am running into an encoding issue, but am not sure. Specifically, IDHTTP with SSL calls an old CGI and the CGI returns a .zip file and I then save it to the disk. In 2007 and before this worked perfectly. In 2009, it is not. Here is the examples of the 2 different results (though cut way short in the post) I am getting back: 2007: 'PK'#3#4#$14#0#0#0#8#0'rLQ9žrPb€'#0...

migrating from Delphi 6 With Indy 10 to XE7 with Indy 10
I updated the original Indy in D6 to version 10 several years ago. Now I want to migrate my application from D6 to XE7 and would like some feedback on the best route to take. I usually send data using readln and writeln statements. The data is typically XML format. Since migrating to XE7 will include potential unicode data what is the best approach to take when reading and writing data? Will writeln and readln work in these cases or should I be using a different strategy to send unicode data between the tidtcpclient and tidtcpserver applications? al wrote: > I usually send data ...

indy 10 back to indy 9
Can anyone point me in the right direction on how to uninstall Indy 10 and go back to Indy 9? I have a Delphi 2007 project that was apparently done with Indy 9 ...

Is Indy 9 more estable than Indy 10?
Using Delphi 2009 update 3 and the latest snapshot indy(tiburon). I am porting app from Indy 9 to Indy 10, the app use a TIdMappedPortTCP component with SSL. This is my app configuration using webservices. Client -> TIdMappedPortTCP and SSL -> IIS Server with webservice app. The client app uses a method on the webservice that returns a TByteDynArray that contains a zipped file. This works with no problem with Indy 9, but when ported to Indy 10,the data transfer between the webservice and the client application with some zip files does no get completed, the client a...

Migrating from Indy 9 to Indy 10
Hi... At work I had to update an old project and it is socket related. It was written with indy 9 and now all I have is Indy 10. I could install indy 9 but I'd rather make it work with I10. The problem is: In the server side there's a lot of: AThread.Connection.WriteInteger(1); AThread.Connection.WriteBuffer(XX, XX); AThread.Connection.ReadStream(XX, XX, XX); idTCPClient is widely used like this: TCPClient.WriteBuffer(Msg, sizeof(TMensaje)); TCPClient.ReadBuffer(Rta, sizeof(TRespuesta)); TCPClient.WriteStream(fStream); TCPClie...

Is it possible to install indy 9 and indy 10 at the same time?
Is it possible to install indy 9 and indy 10 at the same time with a Delphi compiler? Thanks. <joshua lim> wrote in message news:226196@forums.embarcadero.com... > Is it possible to install indy 9 and indy 10 at the same time with a > Delphi compiler? It is technically possible, but it is not easy to set up, and requires manual work to switch between them when needed. -- Remy Lebeau (TeamB) Thanks Remy, are you referring to this? http://www.indyproject.org/KB/howdoiinstallindyd.htm I'm sure many of us face the issue of having some projects in Indy 9 ...

How to convert app from Indy 9 to Indy 10 using Delphi7
I have a small app I have been using a lot, which is a mailchecker where I have used Indy components to handle the POP3 access to mailservers. It was last built in 2004. Now I need to add the possibility to change the POP port for certain non-standard servers. So I opened the project in Delphi7 and immediately received component exceptions.... After ignoring these and changing my package load info for the project to use the Indy that was delivered with Delphi7 I thought all was well since the app now built without errors, but at runtime it caused an exception inside the Forms unit! ...

Indy E-Mail Problem (Indy 10, Delphi 2007)
Hello, we use Indy 10 with Delphi 2007 to get E-Mail using IMAP. We recieved an E-Mail which is not handled correctly. The E-Mail looks like this {code} Return-Path: <test@mail.com>; From: "Test Tester" <test@mail.com>; To: <test@mail.com>; Subject: Order Date: Wed, 7 Mar 2012 16:50:40 +0100 Message-ID: <15D526BEB8091D43859549D9E16E370FBA3123> MIME-Version: 1.0 Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQFk6K6fcF3...

Moving from Indy 9 to Indy 10
I tried posting this to the Atozed forums, but the post isn't going thru. So trying here... In our D7 mail app, this was our code when using Indy 9 to send emails (FEmail is TIdSMTP): FMail.AuthenticationType := atNone; FMail.Connect; try if FMail.AuthSchemesSupported.IndexOf('LOGIN') > -1 then begin FMail.AuthenticationType := atLogin; try FMail.Authenticate; except on E: Exception do begin try FMail.AuthenticationType := atNone; FMail.Authenticate; excep...

Indy 9 to Indy 10 Conversion
Good morning, I have a Client / Server in Indy 9. On Client connect event, I have the next code: ----------------------------------------------------------------------------------- 1. Connection.Write('1'); 2. c := Connection.ReadChar; 3. Connection.Write('3'); 4. c := Connection.ReadChar; 5. Connection.Write('5'); -------------------------------------------------------------------------------- On Server connect event, I have the next code: ----------------------------------------------------------------------------------- 1. c := Connect...

Indy 9 TIdTCPClient and Indy 10.5 TIdTCPServer
Hello, My application is multitiered and has a frontend that installs on a customer's machine and backends which run on our servers. There is a requirement that the legacy frontends compiled with Indy 9 be able to communicate with the new backends compiled with Indy 10.5. Are there any known problems with having Indy 9 TIdTCPClients communicate with Indy 10.5 TIdTCPServers? If so, are there any known workarounds? In particular, is there some limitation in passing TMemoryStreams between two different Indy vers ions? Thanks, Erzsebet Edited by: Erzsebet Carmean on Feb 22, 201...

Indy 9/10
I have been using Indy 9 and have been saving the message to a database using: IdMsgSend.SaveToStream(MyStream); Will Indy 10 read this? Brent brent wrote: > I have been using Indy 9 and have been saving the message to a > database using: > > IdMsgSend.SaveToStream(MyStream); > > Will Indy 10 read this? Yes. -- Remy Lebeau (TeamB) ...

Indy 9/10 Support
[I've already posted this to indy's newsgroup but was told to try elsewhere as nobody monitors the group] I'm trying to support Indy 9/10 for my users in BCB 5, 6, 2006 and 2007. I've downloaded the latest versions of Indy 9 and 10 but am not having any luck compiling them in each of the above compilers. For Indy 9, I don't have any project files for BCB 2006 and 2007. If I'm meant to use the Delphi projects, which ones do I use and how to I compile them ? Will they produce BPL's compatible with BCB ? For Indy 10, I have not managed past BCB5 - th...

Indy 9 and 10 are killing me
I have a FTP client that was written in BCB6 with Indy 8.0.25. The client asked if I could update it if possible and add a few things. I thought that this was going to be a easy project until I tried to update my Indy8 as I also use BCB6 (which I thought was a good thing). So I went to indy.fulgan.com/zip and downloaded the latest snapshot. Using the bat it generated C6 folder with all the files I needed. Following all the directions I modified my BCB installation by installing Indy 8 (Control Panel/Add or remove/c Builder/modify). I then copied the 5 .BPL files to the BIN folder and t...

Web resources about - indy 9 ---> indy 10 cpu - embarcadero.delphi.winsock

Resources last updated: 1/13/2016 7:35:55 AM