IO::Socket client

Discussion in 'Perl Misc' started by George Mpouras, Apr 28, 2014.

  1. I have a small problem receiving data using the IO::Socket


    if the message is bigger than the buffer (512) I loose data with the
    simple statement

    $socket->recv($_, 512)

    if I do this

    while (1)
    {
    $socket->recv($_, 512);
    s/[\r\n]*$//;
    last if length $_ == 0;
    $msg .= $_;
    }

    it reads the big messages but is blocked for messages smaller than 512.
    Any idea ?
    George Mpouras, Apr 28, 2014
    #1
    1. Advertising

  2. George Mpouras <> writes:
    > if the message is bigger than the buffer (512) I loose data with the
    > simple statement
    >
    > $socket->recv($_, 512)
    >
    > if I do this
    >
    > while (1)
    > {
    > $socket->recv($_, 512);
    > s/[\r\n]*$//;
    > last if length $_ == 0;
    > $msg .= $_;
    > }
    >
    > it reads the big messages but is blocked for messages smaller than
    > 512. Any idea ?


    If you're using a record protocol layered above a bytestream transport,
    you'll need to ;'frame' your records/ messages with sufficient metadata
    so that the application can detect message boundaries.
    Rainer Weikusat, Apr 28, 2014
    #2
    1. Advertising

  3. Στις 28/4/2014 18:07, ο/η Rainer Weikusat έγÏαψε:
    > George Mpouras <> writes:
    >> if the message is bigger than the buffer (512) I loose data with the
    >> simple statement
    >>
    >> $socket->recv($_, 512)
    >>
    >> if I do this
    >>
    >> while (1)
    >> {
    >> $socket->recv($_, 512);
    >> s/[\r\n]*$//;
    >> last if length $_ == 0;
    >> $msg .= $_;
    >> }
    >>
    >> it reads the big messages but is blocked for messages smaller than
    >> 512. Any idea ?

    >
    > If you're using a record protocol layered above a bytestream transport,
    > you'll need to ;'frame' your records/ messages with sufficient metadata
    > so that the application can detect message boundaries.
    >



    yes I know this, I was hoping for an "easier" answer !
    George Mpouras, Apr 28, 2014
    #3
  4. George Mpouras <> writes:
    > Στις 28/4/2014 18:07, ο/η Rainer Weikusat έγÏαψε:
    >> George Mpouras <> writes:
    >>> if the message is bigger than the buffer (512) I loose data with the
    >>> simple statement
    >>>
    >>> $socket->recv($_, 512)
    >>>
    >>> if I do this
    >>>
    >>> while (1)
    >>> {
    >>> $socket->recv($_, 512);
    >>> s/[\r\n]*$//;
    >>> last if length $_ == 0;
    >>> $msg .= $_;
    >>> }
    >>>
    >>> it reads the big messages but is blocked for messages smaller than
    >>> 512. Any idea ?

    >>
    >> If you're using a record protocol layered above a bytestream transport,
    >> you'll need to ;'frame' your records/ messages with sufficient metadata
    >> so that the application can detect message boundaries.
    >>

    >
    > yes I know this, I was hoping for an "easier" answer !


    Easier method which works (but is inefficient): Do it "HTTP/1.0-style",
    open a new connection for each request/reply exchange. Whoever is
    currently sending a message can then do a 'write shutdown' afterwards
    and the receiver sees this as 'EOF indicator' but it can still reply.

    Easier method which doesn't work (but you might get away with it until
    someone runs half of your code in a nuclear reactor close to your home
    and the other in a remote-control station located in Anarctica to have a
    good chance to be unaffected by 'funny networking issues'): Use a
    receive timeout (setsockopt/ SO_RCVTIMEO or via select) you believe to
    be 'large enough'.
    Rainer Weikusat, Apr 28, 2014
    #4
  5. George Mpouras

    $Bill Guest

    On 4/28/2014 07:42, George Mpouras wrote:
    > I have a small problem receiving data using the IO::Socket
    >
    >
    > if the message is bigger than the buffer (512) I loose data with the simple statement
    >
    > $socket->recv($_, 512)
    >
    > if I do this
    >
    > while (1)
    > {
    > $socket->recv($_, 512);
    > s/[\r\n]*$//;
    > last if length $_ == 0;
    > $msg .= $_;
    > }
    >
    > it reads the big messages but is blocked for messages smaller than 512. Any idea ?


    Are you on NIX or Doze ? Are you using UDP/TCP/UNIX sockets ?

    Have you tried using sysread instead of recv ?

    If that doesn't help, you could try setting the socket to
    non-blocking OR use FIONBIO ioctl to set non-blocking and
    test for EWOULDBLOCK return OR test the amount of data
    avail using FIONREAD ioctl before reading (I'd try the
    latter which gives you the info you really need).

    It's harder on Doze to get any of these to work properly
    than it is on NIX.
    $Bill, Apr 29, 2014
    #5
  6. On 04/28/14 08:25, George Mpouras wrote:
    > Στις 28/4/2014 18:07, ο/η Rainer Weikusat έγÏαψε:
    >> George Mpouras <> writes:
    >>> if the message is bigger than the buffer (512) I loose data with the
    >>> simple statement
    >>>
    >>> $socket->recv($_, 512)
    >>>
    >>> if I do this
    >>>
    >>> while (1)
    >>> {
    >>> $socket->recv($_, 512);
    >>> s/[\r\n]*$//;
    >>> last if length $_ == 0;
    >>> $msg .= $_;
    >>> }
    >>>
    >>> it reads the big messages but is blocked for messages smaller than
    >>> 512. Any idea ?

    >>
    >> If you're using a record protocol layered above a bytestream transport,
    >> you'll need to ;'frame' your records/ messages with sufficient metadata
    >> so that the application can detect message boundaries.
    >>

    >
    >
    > yes I know this, I was hoping for an "easier" answer !


    Consult a psychic.

    It is easy, every time.

    Xho
    Xho Jingleheimerschmidt, Apr 29, 2014
    #6
  7. >
    > Consult a psychic.
    >
    > It is easy, every time.
    >
    > Xho
    >


    are you one ?
    George Mpouras, Apr 29, 2014
    #7
  8. Στις 28/4/2014 18:07, ο/η Rainer Weikusat έγÏαψε:
    > George Mpouras <> writes:
    >> if the message is bigger than the buffer (512) I loose data with the
    >> simple statement
    >>
    >> $socket->recv($_, 512)
    >>
    >> if I do this
    >>
    >> while (1)
    >> {
    >> $socket->recv($_, 512);
    >> s/[\r\n]*$//;
    >> last if length $_ == 0;
    >> $msg .= $_;
    >> }
    >>
    >> it reads the big messages but is blocked for messages smaller than
    >> 512. Any idea ?

    >
    > If you're using a record protocol layered above a bytestream transport,
    > you'll need to ;'frame' your records/ messages with sufficient metadata
    > so that the application can detect message boundaries.
    >



    after many tests I think that there is no easy workaround, so I will
    implement a simple protocol, thanks
    George Mpouras, Apr 29, 2014
    #8
  9. George Mpouras

    $Bill Guest

    On 4/29/2014 04:01, George Mpouras wrote:
    >
    > after many tests I think that there is no easy workaround, so I will implement a simple protocol, thanks


    What have you tried? I gave you several suggestions - at least
    one of them should work for you depending somewhat on platform.
    $Bill, Apr 30, 2014
    #9
  10. George Mpouras

    $Bill Guest

    On 4/29/2014 19:33, $Bill wrote:
    > On 4/29/2014 04:01, George Mpouras wrote:
    >>
    >> after many tests I think that there is no easy workaround, so I will implement a simple protocol, thanks

    >
    > What have you tried? I gave you several suggestions - at least
    > one of them should work for you depending somewhat on platform.


    A really small test script showing the problem would probably get
    you a more usable solution by those of us that have the time to
    test/try it.
    $Bill, Apr 30, 2014
    #10
  11. Στις 30/4/2014 09:52, ο/η $Bill έγÏαψε:
    > On 4/29/2014 19:33, $Bill wrote:
    >> On 4/29/2014 04:01, George Mpouras wrote:
    >>>
    >>> after many tests I think that there is no easy workaround, so I will
    >>> implement a simple protocol, thanks

    >>
    >> What have you tried? I gave you several suggestions - at least
    >> one of them should work for you depending somewhat on platform.

    >
    > A really small test script showing the problem would probably get
    > you a more usable solution by those of us that have the time to
    > test/try it.
    >



    I test everything you mention Bill. Actually it was not really a
    problem but an understanding that you can not get away using simple approach
    while ($socket) ... or ... $socket->recv($var, 1024) ...

    when you want complex things like.

    -- hey client, I am going to send you a file so big
    ++ ok server
    -- now lets exchange some small control messages
    ++ ok server
    -- now get a big serialized data structure
    ++ not everything is defined
    -- ops client I did not like your answer, wait for three messages for me

    Thanks
    George Mpouras, Apr 30, 2014
    #11
  12. George Mpouras

    $Bill Guest

    On 4/30/2014 02:38, George Mpouras wrote:
    >
    > I test everything you mention Bill. Actually it was not really a problem but an understanding that you can not get away using simple approach
    > while ($socket) ... or ... $socket->recv($var, 1024) ...
    >
    > when you want complex things like.
    >
    > -- hey client, I am going to send you a file so big
    > ++ ok server


    You should be able to just send the file with a proper header saying what it is.

    > -- now lets exchange some small control messages
    > ++ ok server


    Again, you don't have to tell the server what you're going to do,
    just prepend a header saying what it is.

    > -- now get a big serialized data structure
    > ++ not everything is defined


    Asking for something back - that needs a request. You have to be prepared
    to then receive the data back.

    > -- ops client I did not like your answer, wait for three messages for me


    That's not something I would do. Normally, you would just re-request the
    data, but you can implement whatever you need obviously.

    Obviously you have to have a control structure of some kind if you're doing
    more than one thing in the server, but the code to read the socket can still
    be simple before farming the record off to the proper sub to handle it.

    I've done similar logic for a PTP app where two or more users can share a
    directory on their disk with others. Basically I implemented logic to connect
    to a master server to find other sharers; logic to connect to a share server
    and get a list of files being shared; logic to register with the master server;
    logic to retrieve a file from the sharer; a GUI to show sharers directories
    and active xfers, etc. in basically two tasks (a client and a server).

    I wrote a layer on top of TCP/IP Sockets in my case to create a record structure
    to pass between them with a common record header structure. I used select
    and sysread/syswrite for my I/O on Doze (harder than UNIX).

    Good luck doing what you're doing - feel free to ask for any further assistance.
    $Bill, May 1, 2014
    #12
  13. On 2014-05-01 04:01, $Bill <> wrote:
    > On 4/30/2014 02:38, George Mpouras wrote:
    >> -- hey client, I am going to send you a file so big
    >> ++ ok server

    >
    > You should be able to just send the file with a proper header saying what it is.
    >
    >> -- now lets exchange some small control messages
    >> ++ ok server

    >
    > Again, you don't have to tell the server what you're going to do,
    > just prepend a header saying what it is.


    What's the difference between "telling the server what you are going to
    do" and "prepending a header saying what it is"?

    hp


    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
    Peter J. Holzer, May 1, 2014
    #13
  14. George Mpouras

    $Bill Guest

    On 4/30/2014 23:05, Peter J. Holzer wrote:
    > On 2014-05-01 04:01, $Bill <> wrote:
    >> On 4/30/2014 02:38, George Mpouras wrote:
    >>> -- hey client, I am going to send you a file so big
    >>> ++ ok server

    >>
    >> You should be able to just send the file with a proper header saying what it is.
    >>
    >>> -- now lets exchange some small control messages
    >>> ++ ok server

    >>
    >> Again, you don't have to tell the server what you're going to do,
    >> just prepend a header saying what it is.

    >
    > What's the difference between "telling the server what you are going to
    > do" and "prepending a header saying what it is"?


    One msg instead of 2 or 3 ?
    $Bill, May 1, 2014
    #14
  15. George Mpouras

    $Bill Guest

    On 5/1/2014 02:46, $Bill wrote:
    > On 4/30/2014 23:05, Peter J. Holzer wrote:
    >> On 2014-05-01 04:01, $Bill <> wrote:
    >>> On 4/30/2014 02:38, George Mpouras wrote:
    >>>> -- hey client, I am going to send you a file so big
    >>>> ++ ok server
    >>>
    >>> You should be able to just send the file with a proper header saying what it is.
    >>>
    >>>> -- now lets exchange some small control messages
    >>>> ++ ok server
    >>>
    >>> Again, you don't have to tell the server what you're going to do,
    >>> just prepend a header saying what it is.

    >>
    >> What's the difference between "telling the server what you are going to
    >> do" and "prepending a header saying what it is"?

    >
    > One msg instead of 2 or 3 ?


    Plus maybe some extra unnecessary state information that the
    server has to keep track of. Instead of sending control msgs,
    you just send the data or a request for data - much simpler if
    the scenario allows for it.
    $Bill, May 1, 2014
    #15
  16. On 2014-05-01 09:46, $Bill <> wrote:
    > On 4/30/2014 23:05, Peter J. Holzer wrote:
    >> On 2014-05-01 04:01, $Bill <> wrote:
    >>> On 4/30/2014 02:38, George Mpouras wrote:
    >>>> -- hey client, I am going to send you a file so big
    >>>> ++ ok server
    >>> You should be able to just send the file with a proper header saying
    >>> what it is.
    >>>
    >>>> -- now lets exchange some small control messages
    >>>> ++ ok server
    >>>
    >>> Again, you don't have to tell the server what you're going to do,
    >>> just prepend a header saying what it is.

    >>
    >> What's the difference between "telling the server what you are going to
    >> do" and "prepending a header saying what it is"?

    >
    > One msg instead of 2 or 3 ?


    Nobody said that you have to "tell the server what you are going to do"
    in a separate message. And the concept of "message" in a stream-based
    protocol is a bit hazy, anyway.

    If an HTTP server sends something like this

    200 OK
    Content-Type: text/html
    Size: 1234

    <html>...

    is this one message (a file with a prepended header) or two (the first
    one tells the client "I'm going to send to you an HTML file with 1234
    bytes", the second contains the file)? I don't see the difference. In
    any case, after reading the header the client knows what the server is
    going to do (or at least promised to do).

    hp


    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
    Peter J. Holzer, May 1, 2014
    #16
  17. George Mpouras

    $Bill Guest

    On 5/1/2014 04:26, Peter J. Holzer wrote:
    >
    > Nobody said that you have to "tell the server what you are going to do"
    > in a separate message. And the concept of "message" in a stream-based
    > protocol is a bit hazy, anyway.
    >
    > If an HTTP server sends something like this
    >
    > 200 OK
    > Content-Type: text/html
    > Size: 1234
    >
    > <html>...
    >
    > is this one message (a file with a prepended header) or two (the first
    > one tells the client "I'm going to send to you an HTML file with 1234
    > bytes", the second contains the file)? I don't see the difference. In
    > any case, after reading the header the client knows what the server is
    > going to do (or at least promised to do).


    If you do it with a single msg, it's easier than with 2 or 3 msgs.
    You don't have to keep state information on the socket (or keep the same
    socket open for that matter).

    All I'm saying is it's cleaner to keep the msgs to a minimum and eliminate
    most control msgs. If you feel differently - fine - we all have opinions.
    $Bill, May 2, 2014
    #17
  18. On 2014-05-02 01:06, $Bill <> wrote:
    > On 5/1/2014 04:26, Peter J. Holzer wrote:
    >> Nobody said that you have to "tell the server what you are going to do"
    >> in a separate message. And the concept of "message" in a stream-based
    >> protocol is a bit hazy, anyway.
    >>
    >> If an HTTP server sends something like this
    >>
    >> 200 OK
    >> Content-Type: text/html
    >> Size: 1234
    >>
    >> <html>...
    >>
    >> is this one message (a file with a prepended header) or two (the first
    >> one tells the client "I'm going to send to you an HTML file with 1234
    >> bytes", the second contains the file)? I don't see the difference. In
    >> any case, after reading the header the client knows what the server is
    >> going to do (or at least promised to do).

    >
    > If you do it with a single msg, it's easier than with 2 or 3 msgs.


    You haven't answered any of my points. Neither have you given a reason
    why "telling the server what you are going to do" would have to be in a
    separate message, nor have you answered whether the above is one message
    or two (or five). And for the latter question I would like to see a
    better reason than "because rfc 2616 says so".

    > You don't have to keep state information on the socket (or keep the same
    > socket open for that matter).


    How does the state information needed to read the HTTP response above
    change whether you view it as one or two (or five) messages?

    > All I'm saying is it's cleaner to keep the msgs to a minimum and eliminate
    > most control msgs. If you feel differently - fine - we all have opinions.


    No, that's not my point at all. My point is that you are making
    assumptions which aren't founded in anything written before in this
    thread.

    hp


    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
    Peter J. Holzer, May 2, 2014
    #18
  19. George Mpouras

    $Bill Guest

    On 5/2/2014 02:03, Peter J. Holzer wrote:
    >
    > You haven't answered any of my points. Neither have you given a reason
    > why "telling the server what you are going to do" would have to be in a
    > separate message, nor have you answered whether the above is one message
    > or two (or five). And for the latter question I would like to see a
    > better reason than "because rfc 2616 says so".


    Sure I did. Using more than 1 msg necessitates state info and probably
    maintaining an open socket.

    >> You don't have to keep state information on the socket (or keep the same
    >> socket open for that matter).

    >
    > How does the state information needed to read the HTTP response above
    > change whether you view it as one or two (or five) messages?


    You don't need state info if there's only 1 msg.

    >> All I'm saying is it's cleaner to keep the msgs to a minimum and eliminate
    >> most control msgs. If you feel differently - fine - we all have opinions.

    >
    > No, that's not my point at all. My point is that you are making
    > assumptions which aren't founded in anything written before in this
    > thread.


    I think you just want to argue. I was responding to George's scenario:

    when you want complex things like.

    -- hey client, I am going to send you a file so big
    ++ ok server
    -- now lets exchange some small control messages
    ++ ok server
    -- now get a big serialized data structure
    ++ not everything is defined
    -- ops client I did not like your answer, wait for three messages for me

    Like I said earlier - if you want to interpret that differently than I - go ahead,
    but it seemed overly complex to me. OAO
    $Bill, May 2, 2014
    #19
  20. On 2014-05-02 10:01, $Bill <> wrote:
    > On 5/2/2014 02:03, Peter J. Holzer wrote:
    >>
    >> You haven't answered any of my points. Neither have you given a reason
    >> why "telling the server what you are going to do" would have to be in a
    >> separate message, nor have you answered whether the above is one message
    >> or two (or five). And for the latter question I would like to see a
    >> better reason than "because rfc 2616 says so".

    >
    > Sure I did. Using more than 1 msg necessitates state info and probably
    > maintaining an open socket.


    Ok, again:

    Question 1: Why does "telling the server what you are going to do"
    have to be in a separate message?

    "Using more than 1 msg necessitates state info and probably
    maintaining an open socket." doesn't answer the question. On the
    contrary, it argues that you should tell the server in the same
    message, not in a separate message. It doesn't even try to
    argue why this would be impossible.

    Question 2: Is the above one message or two or five?

    "Using more than 1 msg necessitates state info and probably
    maintaining an open socket." doesn't answer the question. The
    answer would have to be include one of the numbers I suggested
    (or possibly a different number).

    So. No, you haven't.

    >>> You don't have to keep state information on the socket (or keep the same
    >>> socket open for that matter).

    >>
    >> How does the state information needed to read the HTTP response above
    >> change whether you view it as one or two (or five) messages?

    >
    > You don't need state info if there's only 1 msg.


    You keep repeating this but you don't give any reason for it. The
    sequence of bytes on the wire doesn't change. Why would the amount of
    state info change? In any case you need two pieces of state info:

    * Whether you are in the header/message 1 or the body/message 2
    * If you are in the latter, how much you have already read.

    hp


    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
    Peter J. Holzer, May 2, 2014
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Laszlo Nagy
    Replies:
    1
    Views:
    4,762
    Mark Wooding
    Jan 27, 2009
  2. Jean-Paul Calderone
    Replies:
    0
    Views:
    947
    Jean-Paul Calderone
    Jan 27, 2009
  3. Laszlo Nagy
    Replies:
    0
    Views:
    528
    Laszlo Nagy
    Feb 1, 2009
  4. Steve Holden
    Replies:
    0
    Views:
    646
    Steve Holden
    Feb 1, 2009
  5. Steve Holden
    Replies:
    1
    Views:
    699
Loading...

Share This Page