IO::Socket client

G

George Mpouras

I have a small problem receiving data using the IO::Socket


if the message is bigger than the buffer (512) I loose data with the
simple statement

$socket->recv($_, 512)

if I do this

while (1)
{
$socket->recv($_, 512);
s/[\r\n]*$//;
last if length $_ == 0;
$msg .= $_;
}

it reads the big messages but is blocked for messages smaller than 512.
Any idea ?
 
R

Rainer Weikusat

George Mpouras said:
if the message is bigger than the buffer (512) I loose data with the
simple statement

$socket->recv($_, 512)

if I do this

while (1)
{
$socket->recv($_, 512);
s/[\r\n]*$//;
last if length $_ == 0;
$msg .= $_;
}

it reads the big messages but is blocked for messages smaller than
512. Any idea ?

If you're using a record protocol layered above a bytestream transport,
you'll need to ;'frame' your records/ messages with sufficient metadata
so that the application can detect message boundaries.
 
G

George Mpouras

Στις 28/4/2014 18:07, ο/η Rainer Weikusat έγÏαψε:
George Mpouras said:
if the message is bigger than the buffer (512) I loose data with the
simple statement

$socket->recv($_, 512)

if I do this

while (1)
{
$socket->recv($_, 512);
s/[\r\n]*$//;
last if length $_ == 0;
$msg .= $_;
}

it reads the big messages but is blocked for messages smaller than
512. Any idea ?

If you're using a record protocol layered above a bytestream transport,
you'll need to ;'frame' your records/ messages with sufficient metadata
so that the application can detect message boundaries.


yes I know this, I was hoping for an "easier" answer !
 
R

Rainer Weikusat

George Mpouras said:
Στις 28/4/2014 18:07, ο/η Rainer Weikusat έγÏαψε:
George Mpouras said:
if the message is bigger than the buffer (512) I loose data with the
simple statement

$socket->recv($_, 512)

if I do this

while (1)
{
$socket->recv($_, 512);
s/[\r\n]*$//;
last if length $_ == 0;
$msg .= $_;
}

it reads the big messages but is blocked for messages smaller than
512. Any idea ?

If you're using a record protocol layered above a bytestream transport,
you'll need to ;'frame' your records/ messages with sufficient metadata
so that the application can detect message boundaries.

yes I know this, I was hoping for an "easier" answer !

Easier method which works (but is inefficient): Do it "HTTP/1.0-style",
open a new connection for each request/reply exchange. Whoever is
currently sending a message can then do a 'write shutdown' afterwards
and the receiver sees this as 'EOF indicator' but it can still reply.

Easier method which doesn't work (but you might get away with it until
someone runs half of your code in a nuclear reactor close to your home
and the other in a remote-control station located in Anarctica to have a
good chance to be unaffected by 'funny networking issues'): Use a
receive timeout (setsockopt/ SO_RCVTIMEO or via select) you believe to
be 'large enough'.
 
$

$Bill

I have a small problem receiving data using the IO::Socket


if the message is bigger than the buffer (512) I loose data with the simple statement

$socket->recv($_, 512)

if I do this

while (1)
{
$socket->recv($_, 512);
s/[\r\n]*$//;
last if length $_ == 0;
$msg .= $_;
}

it reads the big messages but is blocked for messages smaller than 512. Any idea ?

Are you on NIX or Doze ? Are you using UDP/TCP/UNIX sockets ?

Have you tried using sysread instead of recv ?

If that doesn't help, you could try setting the socket to
non-blocking OR use FIONBIO ioctl to set non-blocking and
test for EWOULDBLOCK return OR test the amount of data
avail using FIONREAD ioctl before reading (I'd try the
latter which gives you the info you really need).

It's harder on Doze to get any of these to work properly
than it is on NIX.
 
X

Xho Jingleheimerschmidt

Στις 28/4/2014 18:07, ο/η Rainer Weikusat έγÏαψε:
George Mpouras said:
if the message is bigger than the buffer (512) I loose data with the
simple statement

$socket->recv($_, 512)

if I do this

while (1)
{
$socket->recv($_, 512);
s/[\r\n]*$//;
last if length $_ == 0;
$msg .= $_;
}

it reads the big messages but is blocked for messages smaller than
512. Any idea ?

If you're using a record protocol layered above a bytestream transport,
you'll need to ;'frame' your records/ messages with sufficient metadata
so that the application can detect message boundaries.


yes I know this, I was hoping for an "easier" answer !

Consult a psychic.

It is easy, every time.

Xho
 
G

George Mpouras

Στις 28/4/2014 18:07, ο/η Rainer Weikusat έγÏαψε:
George Mpouras said:
if the message is bigger than the buffer (512) I loose data with the
simple statement

$socket->recv($_, 512)

if I do this

while (1)
{
$socket->recv($_, 512);
s/[\r\n]*$//;
last if length $_ == 0;
$msg .= $_;
}

it reads the big messages but is blocked for messages smaller than
512. Any idea ?

If you're using a record protocol layered above a bytestream transport,
you'll need to ;'frame' your records/ messages with sufficient metadata
so that the application can detect message boundaries.


after many tests I think that there is no easy workaround, so I will
implement a simple protocol, thanks
 
$

$Bill

after many tests I think that there is no easy workaround, so I will implement a simple protocol, thanks

What have you tried? I gave you several suggestions - at least
one of them should work for you depending somewhat on platform.
 
$

$Bill

What have you tried? I gave you several suggestions - at least
one of them should work for you depending somewhat on platform.

A really small test script showing the problem would probably get
you a more usable solution by those of us that have the time to
test/try it.
 
G

George Mpouras

Στις 30/4/2014 09:52, ο/η $Bill έγÏαψε:
A really small test script showing the problem would probably get
you a more usable solution by those of us that have the time to
test/try it.


I test everything you mention Bill. Actually it was not really a
problem but an understanding that you can not get away using simple approach
while ($socket) ... or ... $socket->recv($var, 1024) ...

when you want complex things like.

-- hey client, I am going to send you a file so big
++ ok server
-- now lets exchange some small control messages
++ ok server
-- now get a big serialized data structure
++ not everything is defined
-- ops client I did not like your answer, wait for three messages for me

Thanks
 
$

$Bill

I test everything you mention Bill. Actually it was not really a problem but an understanding that you can not get away using simple approach
while ($socket) ... or ... $socket->recv($var, 1024) ...

when you want complex things like.

-- hey client, I am going to send you a file so big
++ ok server

You should be able to just send the file with a proper header saying what it is.
-- now lets exchange some small control messages
++ ok server

Again, you don't have to tell the server what you're going to do,
just prepend a header saying what it is.
-- now get a big serialized data structure
++ not everything is defined

Asking for something back - that needs a request. You have to be prepared
to then receive the data back.
-- ops client I did not like your answer, wait for three messages for me

That's not something I would do. Normally, you would just re-request the
data, but you can implement whatever you need obviously.

Obviously you have to have a control structure of some kind if you're doing
more than one thing in the server, but the code to read the socket can still
be simple before farming the record off to the proper sub to handle it.

I've done similar logic for a PTP app where two or more users can share a
directory on their disk with others. Basically I implemented logic to connect
to a master server to find other sharers; logic to connect to a share server
and get a list of files being shared; logic to register with the master server;
logic to retrieve a file from the sharer; a GUI to show sharers directories
and active xfers, etc. in basically two tasks (a client and a server).

I wrote a layer on top of TCP/IP Sockets in my case to create a record structure
to pass between them with a common record header structure. I used select
and sysread/syswrite for my I/O on Doze (harder than UNIX).

Good luck doing what you're doing - feel free to ask for any further assistance.
 
P

Peter J. Holzer

You should be able to just send the file with a proper header saying what it is.


Again, you don't have to tell the server what you're going to do,
just prepend a header saying what it is.

What's the difference between "telling the server what you are going to
do" and "prepending a header saying what it is"?

hp
 
$

$Bill

One msg instead of 2 or 3 ?

Plus maybe some extra unnecessary state information that the
server has to keep track of. Instead of sending control msgs,
you just send the data or a request for data - much simpler if
the scenario allows for it.
 
P

Peter J. Holzer

One msg instead of 2 or 3 ?

Nobody said that you have to "tell the server what you are going to do"
in a separate message. And the concept of "message" in a stream-based
protocol is a bit hazy, anyway.

If an HTTP server sends something like this

200 OK
Content-Type: text/html
Size: 1234

<html>...

is this one message (a file with a prepended header) or two (the first
one tells the client "I'm going to send to you an HTML file with 1234
bytes", the second contains the file)? I don't see the difference. In
any case, after reading the header the client knows what the server is
going to do (or at least promised to do).

hp
 
$

$Bill

Nobody said that you have to "tell the server what you are going to do"
in a separate message. And the concept of "message" in a stream-based
protocol is a bit hazy, anyway.

If an HTTP server sends something like this

200 OK
Content-Type: text/html
Size: 1234

<html>...

is this one message (a file with a prepended header) or two (the first
one tells the client "I'm going to send to you an HTML file with 1234
bytes", the second contains the file)? I don't see the difference. In
any case, after reading the header the client knows what the server is
going to do (or at least promised to do).

If you do it with a single msg, it's easier than with 2 or 3 msgs.
You don't have to keep state information on the socket (or keep the same
socket open for that matter).

All I'm saying is it's cleaner to keep the msgs to a minimum and eliminate
most control msgs. If you feel differently - fine - we all have opinions.
 
P

Peter J. Holzer

If you do it with a single msg, it's easier than with 2 or 3 msgs.

You haven't answered any of my points. Neither have you given a reason
why "telling the server what you are going to do" would have to be in a
separate message, nor have you answered whether the above is one message
or two (or five). And for the latter question I would like to see a
better reason than "because rfc 2616 says so".
You don't have to keep state information on the socket (or keep the same
socket open for that matter).

How does the state information needed to read the HTTP response above
change whether you view it as one or two (or five) messages?
All I'm saying is it's cleaner to keep the msgs to a minimum and eliminate
most control msgs. If you feel differently - fine - we all have opinions.

No, that's not my point at all. My point is that you are making
assumptions which aren't founded in anything written before in this
thread.

hp
 
$

$Bill

You haven't answered any of my points. Neither have you given a reason
why "telling the server what you are going to do" would have to be in a
separate message, nor have you answered whether the above is one message
or two (or five). And for the latter question I would like to see a
better reason than "because rfc 2616 says so".

Sure I did. Using more than 1 msg necessitates state info and probably
maintaining an open socket.
How does the state information needed to read the HTTP response above
change whether you view it as one or two (or five) messages?

You don't need state info if there's only 1 msg.
No, that's not my point at all. My point is that you are making
assumptions which aren't founded in anything written before in this
thread.

I think you just want to argue. I was responding to George's scenario:

when you want complex things like.

-- hey client, I am going to send you a file so big
++ ok server
-- now lets exchange some small control messages
++ ok server
-- now get a big serialized data structure
++ not everything is defined
-- ops client I did not like your answer, wait for three messages for me

Like I said earlier - if you want to interpret that differently than I - go ahead,
but it seemed overly complex to me. OAO
 
P

Peter J. Holzer

Sure I did. Using more than 1 msg necessitates state info and probably
maintaining an open socket.

Ok, again:

Question 1: Why does "telling the server what you are going to do"
have to be in a separate message?

"Using more than 1 msg necessitates state info and probably
maintaining an open socket." doesn't answer the question. On the
contrary, it argues that you should tell the server in the same
message, not in a separate message. It doesn't even try to
argue why this would be impossible.

Question 2: Is the above one message or two or five?

"Using more than 1 msg necessitates state info and probably
maintaining an open socket." doesn't answer the question. The
answer would have to be include one of the numbers I suggested
(or possibly a different number).

So. No, you haven't.
You don't need state info if there's only 1 msg.

You keep repeating this but you don't give any reason for it. The
sequence of bytes on the wire doesn't change. Why would the amount of
state info change? In any case you need two pieces of state info:

* Whether you are in the header/message 1 or the body/message 2
* If you are in the latter, how much you have already read.

hp
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,820
Messages
2,569,721
Members
45,509
Latest member
Danish324

Latest Threads

Top