transfering a string from client to server without deadlock?

Discussion in 'Perl Misc' started by peter pilsl, May 17, 2006.

  1. peter pilsl

    peter pilsl Guest

    I use Net::Daemon for a very simple server-client-szenario

    client is sending a string to the server and the server is processing
    the string and returning a answerstring to the client and the client is
    terminating.

    Problem: the string is very long and can contain virtually any character
    (its the output of freeze() from the Storable-Module which serializes
    even complex hashes)

    So how does the Server know when the Client has finished sending its string?

    I tried several attempts that all works, but I'm sure they will fail on
    certain data:

    * the client sends the length of the string first, so the server knows
    how much data it needs to wait for. (this will sure fail when using
    utf8-encoded data and client/server will be on different machines with
    different perls)
    * I use a very complex "EOF"-pattern like \n\0\0\0\0\n and pray to my
    gods that this pattern will never be part of the string.

    I am sure that there is a much better/simple/prooved solution to this
    problem that must be a standard-problem.

    thnx,
    peter
    peter pilsl, May 17, 2006
    #1
    1. Advertising

  2. peter pilsl

    Ben Morrow Guest

    Quoth peter pilsl <>:
    >
    >
    > I use Net::Daemon for a very simple server-client-szenario
    >
    > client is sending a string to the server and the server is processing
    > the string and returning a answerstring to the client and the client is
    > terminating.
    >
    > Problem: the string is very long and can contain virtually any character
    > (its the output of freeze() from the Storable-Module which serializes
    > even complex hashes)
    >
    > So how does the Server know when the Client has finished sending its string?
    >
    > I tried several attempts that all works, but I'm sure they will fail on
    > certain data:
    >
    > * the client sends the length of the string first, so the server knows
    > how much data it needs to wait for. (this will sure fail when using
    > utf8-encoded data and client/server will be on different machines with
    > different perls)


    This is the right answer. UTF-8 doesn't come into it: Storable data is
    binary, not text, so it has no encoding. Just make sure you binmode all
    the filehandles concerned.

    A very good example of a protocol like this is HTTP/1.1; I would
    recommend you find and read RFC2616. You could consider using HTTP
    instead: there are many very good Perl modules that implement HTTP
    clients and servers, already written and debugged for you. Plus you get
    a whole lot of good stuff like proxy support for free :).

    > * I use a very complex "EOF"-pattern like \n\0\0\0\0\n and pray to my
    > gods that this pattern will never be part of the string.


    Never do this. Murphy will bite you.

    The other sensible option is to escape the data somehow: Base64 encode
    it, or backwhack all newlines, or something, so that you can produce an
    EOT string that you are *certain* isn't in your data. This is the
    approach used by MIME emails; IMHO it is inferior to the HTTP approach.

    Ben

    --
    #!/bin/sh
    quine="echo 'eval \$quine' >> \$0; echo quined"
    eval $quine
    # []
    Ben Morrow, May 17, 2006
    #2
    1. Advertising

  3. peter pilsl

    Tim Shoppa Guest

    peter pilsl wrote:
    > Problem: the string is very long and can contain virtually any character
    > (its the output of freeze() from the Storable-Module which serializes
    > even complex hashes)
    >
    > So how does the Server know when the Client has finished sending its string?
    >
    > I tried several attempts that all works, but I'm sure they will fail on
    > certain data:
    >
    > * the client sends the length of the string first, so the server knows
    > how much data it needs to wait for. (this will sure fail when using
    > utf8-encoded data and client/server will be on different machines with
    > different perls)


    What's wrong with the "do { use bytes; length(EXPR) }" idiom for
    getting the length in bytes (as opposed to characters)?

    > * I use a very complex "EOF"-pattern like \n\0\0\0\0\n and pray to my
    > gods that this pattern will never be part of the string.


    Even better is to, for example, escape or MIME-encode or uuencode the
    data so that you know that the end character/pattern never occurs in
    the string. This is in effect how SMTP and some other transports move
    binary stuff while not getting trapped by delimiter strings.

    I personally don't really like MIME encoding or uuencoding but they're
    simple and lots of examples in CPAN.

    > I am sure that there is a much better/simple/prooved solution to this
    > problem that must be a standard-problem.


    Actually, you seem to have formulated the two common solutions yourself
    but just missed some of the "as commonly implemented" details.

    Tim.
    Tim Shoppa, May 17, 2006
    #3
  4. peter pilsl

    Tim Shoppa Guest

    Ben Morrow wrote:
    > Peter wrote:
    > > * I use a very complex "EOF"-pattern like \n\0\0\0\0\n and pray to my
    > > gods that this pattern will never be part of the string.

    >
    > Never do this. Murphy will bite you.


    Nevertheless it is used in a couple of well-known protocols and a
    couple of commercial products. Yeah, it sucks rocks, but it's out there
    in the wild (rarely for the better and as you point out always for the
    worse).

    > The other sensible option is to escape the data somehow: Base64 encode
    > it, or backwhack all newlines, or something, so that you can produce an
    > EOT string that you are *certain* isn't in your data. This is the
    > approach used by MIME emails; IMHO it is inferior to the HTTP approach.


    Sometimes (especially when moving stuff over "all the world's a stream
    of bytes" paradigms like pipes and stdin/stdout) it is pressed into
    service, despite its shortcomings, because nobody genuinely knows
    beforehand how long the data is going tol be and buffering it all up is
    foolhardy as well for either latency or storage limitations.

    Tim.
    Tim Shoppa, May 17, 2006
    #4
  5. peter pilsl

    Ben Morrow Guest

    Quoth "Tim Shoppa" <>:
    > Ben Morrow wrote:
    > > The other sensible option is to escape the data somehow: Base64 encode
    > > it, or backwhack all newlines, or something, so that you can produce an
    > > EOT string that you are *certain* isn't in your data. This is the
    > > approach used by MIME emails; IMHO it is inferior to the HTTP approach.

    >
    > Sometimes (especially when moving stuff over "all the world's a stream
    > of bytes" paradigms like pipes and stdin/stdout) it is pressed into
    > service, despite its shortcomings, because nobody genuinely knows
    > beforehand how long the data is going tol be and buffering it all up is
    > foolhardy as well for either latency or storage limitations.


    ....and the correct answer (again IMHO) in this case is to use HTTP
    chunking (another goodie you get for free if you use HTTP as your
    protocol). But this is now definitely OT... :)

    Ben

    --
    The cosmos, at best, is like a rubbish heap scattered at random.
    Heraclitus
    Ben Morrow, May 17, 2006
    #5
  6. peter pilsl

    Guest

    peter pilsl <> wrote:
    > I use Net::Daemon for a very simple server-client-szenario
    >
    > client is sending a string to the server and the server is processing
    > the string and returning a answerstring to the client and the client is
    > terminating.
    >
    > Problem: the string is very long and can contain virtually any character
    > (its the output of freeze() from the Storable-Module which serializes
    > even complex hashes)
    >
    > So how does the Server know when the Client has finished sending its
    > string?


    When the client's fd_retrieve call returns the object, then server must be
    done sending it.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
    , May 17, 2006
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Patrick Steele [MVP]
    Replies:
    4
    Views:
    2,341
    M. Posseth
    Jun 17, 2004
  2. Ramon
    Replies:
    1
    Views:
    475
    Lothar Kimmeringer
    Feb 25, 2009
  3. Clive Lin
    Replies:
    0
    Views:
    83
    Clive Lin
    Jan 4, 2009
  4. Replies:
    1
    Views:
    80
  5. Jens Axel Søgaard
    Replies:
    25
    Views:
    241
Loading...

Share This Page