reading from (TCP) sockets

Hi,

when reading data from a TCP socket, how do I tell when I have received
all the data I should have received?

Particularly, I=C2=B4m trying to read data sent by a web server, so I=C2=B4m
opening a socket, send an HTTP request and receive an answer.

Apparently I need to read from the socket in an endless loop.  I really
don=C2=B4t like that because it blocks the program and creates 100% CPU loa=
d,
and I can=C2=B4t tell if there is more data to be read or not.  It doesn=C2=
=B4t
seem wise to rely on the Content-Length: header to figure out whether I
received all data or not.

I guess I=C2=B4m doing it all wrong because I couldn=C2=B4t find a document=
ation
about using sockets which isn=C2=B4t anything but confusing and leaves lots
of questions.  I also guess I need something event-driven, i. e. send a
request, and in the event that the server sends data, somehow receive it
in the background.

The application would need to do a few things in the background because
there would be several types of events.  Forking a process for every
event doesn=C2=B4t seem right, and it provides me with the problem of
exchanging data between the various processes (at least unless I let
every process do its job and exit and fork another one to do basically
the same thing again).

Is perl the wrong language to do this?  What would you suggest?
0
hw
8/12/2017 8:45:38 AM
perl.beginners 29244 articles. 3 followers. Follow

1 Replies
11 Views

Similar Articles

[PageSpeed] 26

--f403043893487833f905569cb43b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Sockets cannot tell you how much data will come in any language.  You HAVE
to rely on Content-Length, that is what it is for.  Why do you think "It
doesn=C2=B4t
seem wise to rely [it]"?

It is not possible for a loop to both be blocking (which means it is using
no CPU until signaled there is data) and use 100% of CPU.

Why are you reading data directly from a socket yourself?  Especially if it
is HTTP data?  There are modules for this that handle all of the
intricacies for you.  For example, if you want to fetch a webpage, you
should say

use strict;
use LWP::UserAgent;
use warnings;

my $ua =3D LWP::UserAgent->new;

my $response =3D $ua->get("http://ifconfig.io/ip");

unless ($response->is_success) {
    die "could not fetch IP address from ifconfig.io: ",
$response->status_line;
}

print $response->decoded_content, "\n";



On Sat, Aug 12, 2017 at 11:20 PM hw <hw@adminart.net> wrote:

> Hi,
>
> when reading data from a TCP socket, how do I tell when I have received
> all the data I should have received?
>
> Particularly, I=C2=B4m trying to read data sent by a web server, so I=C2=
=B4m
> opening a socket, send an HTTP request and receive an answer.
>
> Apparently I need to read from the socket in an endless loop.  I really
> don=C2=B4t like that because it blocks the program and creates 100% CPU l=
oad,
> and I can=C2=B4t tell if there is more data to be read or not.  It doesn=
=C2=B4t
> seem wise to rely on the Content-Length: header to figure out whether I
> received all data or not.
>
> I guess I=C2=B4m doing it all wrong because I couldn=C2=B4t find a docume=
ntation
> about using sockets which isn=C2=B4t anything but confusing and leaves lo=
ts
> of questions.  I also guess I need something event-driven, i. e. send a
> request, and in the event that the server sends data, somehow receive it
> in the background.
>
> The application would need to do a few things in the background because
> there would be several types of events.  Forking a process for every
> event doesn=C2=B4t seem right, and it provides me with the problem of
> exchanging data between the various processes (at least unless I let
> every process do its job and exit and fork another one to do basically
> the same thing again).
>
> Is perl the wrong language to do this?  What would you suggest?
>
> --
> To unsubscribe, e-mail: beginners-unsubscribe@perl.org
> For additional commands, e-mail: beginners-help@perl.org
> http://learn.perl.org/
>
>
>

--f403043893487833f905569cb43b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Sockets cannot tell you how much data will come in any lan=
guage.=C2=A0 You HAVE to rely on=C2=A0<span style=3D"color:rgb(33,33,33)">C=
ontent-Length, that is what it is for.=C2=A0 Why do you think &quot;</span>=
<span style=3D"color:rgb(33,33,33)">It doesn=C2=B4t</span><br style=3D"colo=
r:rgb(33,33,33)"><span style=3D"color:rgb(33,33,33)">seem wise to rely [it]=
&quot;?</span><div><span style=3D"color:rgb(33,33,33)"><br></span></div><di=
v><span style=3D"color:rgb(33,33,33)">It is not possible for a loop to both=
 be blocking (which means it is using no CPU until signaled there is data) =
and use 100% of CPU.</span></div><div><span style=3D"color:rgb(33,33,33)"><=
br></span></div><div><span style=3D"color:rgb(33,33,33)">Why are you readin=
g data directly from a socket yourself?=C2=A0 Especially if it is HTTP data=
?=C2=A0 There are modules for this that handle all of the intricacies for y=
ou.=C2=A0 For example, if you want to fetch a webpage, you should say</span=
></div><div><span style=3D"color:rgb(33,33,33)"><br></span></div><div><font=
 color=3D"#212121">use strict;</font></div><div><font color=3D"#212121">use=
 LWP::UserAgent;</font></div><div><font color=3D"#212121">use warnings;</fo=
nt></div><div><font color=3D"#212121"><br></font></div><div><font color=3D"=
#212121">my $ua =3D LWP::UserAgent-&gt;new;</font></div><div><font color=3D=
"#212121"><br></font></div><div><font color=3D"#212121">my $response =3D $u=
a-&gt;get(&quot;<a href=3D"http://ifconfig.io/ip">http://ifconfig.io/ip</a>=
&quot;);</font></div><div><font color=3D"#212121"><br></font></div><div><fo=
nt color=3D"#212121">unless ($response-&gt;is_success) {</font></div><div><=
font color=3D"#212121">=C2=A0 =C2=A0 die &quot;could not fetch IP address f=
rom <a href=3D"http://ifconfig.io">ifconfig.io</a>: &quot;, $response-&gt;s=
tatus_line;</font></div><div><font color=3D"#212121">}</font></div><div><fo=
nt color=3D"#212121"><br></font></div><div><font color=3D"#212121">print $r=
esponse-&gt;decoded_content, &quot;\n&quot;;</font></div><div><font color=
=3D"#212121"><br></font></div><div><font color=3D"#212121"><br></font></div=
></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Sat, Aug 12, 2017=
 at 11:20 PM hw &lt;<a href=3D"mailto:hw@adminart.net">hw@adminart.net</a>&=
gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=
 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
when reading data from a TCP socket, how do I tell when I have received<br>
all the data I should have received?<br>
<br>
Particularly, I=C2=B4m trying to read data sent by a web server, so I=C2=B4=
m<br>
opening a socket, send an HTTP request and receive an answer.<br>
<br>
Apparently I need to read from the socket in an endless loop.=C2=A0 I reall=
y<br>
don=C2=B4t like that because it blocks the program and creates 100% CPU loa=
d,<br>
and I can=C2=B4t tell if there is more data to be read or not.=C2=A0 It doe=
sn=C2=B4t<br>
seem wise to rely on the Content-Length: header to figure out whether I<br>
received all data or not.<br>
<br>
I guess I=C2=B4m doing it all wrong because I couldn=C2=B4t find a document=
ation<br>
about using sockets which isn=C2=B4t anything but confusing and leaves lots=
<br>
of questions.=C2=A0 I also guess I need something event-driven, i. e. send =
a<br>
request, and in the event that the server sends data, somehow receive it<br=
>
in the background.<br>
<br>
The application would need to do a few things in the background because<br>
there would be several types of events.=C2=A0 Forking a process for every<b=
r>
event doesn=C2=B4t seem right, and it provides me with the problem of<br>
exchanging data between the various processes (at least unless I let<br>
every process do its job and exit and fork another one to do basically<br>
the same thing again).<br>
<br>
Is perl the wrong language to do this?=C2=A0 What would you suggest?<br>
<br>
--<br>
To unsubscribe, e-mail: <a href=3D"mailto:beginners-unsubscribe@perl.org" t=
arget=3D"_blank">beginners-unsubscribe@perl.org</a><br>
For additional commands, e-mail: <a href=3D"mailto:beginners-help@perl.org"=
 target=3D"_blank">beginners-help@perl.org</a><br>
<a href=3D"http://learn.perl.org/" rel=3D"noreferrer" target=3D"_blank">htt=
p://learn.perl.org/</a><br>
<br>
<br>
</blockquote></div>

--f403043893487833f905569cb43b--
0
chas
8/13/2017 6:31:55 AM
Reply: