transfering a string from client to server without deadlock?

[peter pilsl]

I use Net:aemon for a very simple server-client-szenario

client is sending a string to the server and the server is processing
the string and returning a answerstring to the client and the client is
terminating.

Problem: the string is very long and can contain virtually any character
(its the output of freeze() from the Storable-Module which serializes
even complex hashes)

So how does the Server know when the Client has finished sending its string?

I tried several attempts that all works, but I'm sure they will fail on
certain data:

* the client sends the length of the string first, so the server knows
how much data it needs to wait for. (this will sure fail when using
utf8-encoded data and client/server will be on different machines with
different perls)
* I use a very complex "EOF"-pattern like \n\0\0\0\0\n and pray to my
gods that this pattern will never be part of the string.

I am sure that there is a much better/simple/prooved solution to this
problem that must be a standard-problem.

thnx,
peter

 

[Ben Morrow]

I use Net:aemon for a very simple server-client-szenario

client is sending a string to the server and the server is processing
the string and returning a answerstring to the client and the client is
terminating.

Problem: the string is very long and can contain virtually any character
(its the output of freeze() from the Storable-Module which serializes
even complex hashes)

So how does the Server know when the Client has finished sending its string?

I tried several attempts that all works, but I'm sure they will fail on
certain data:

* the client sends the length of the string first, so the server knows
how much data it needs to wait for. (this will sure fail when using
utf8-encoded data and client/server will be on different machines with
different perls)

This is the right answer. UTF-8 doesn't come into it: Storable data is
binary, not text, so it has no encoding. Just make sure you binmode all
the filehandles concerned.

A very good example of a protocol like this is HTTP/1.1; I would
recommend you find and read RFC2616. You could consider using HTTP
instead: there are many very good Perl modules that implement HTTP
clients and servers, already written and debugged for you. Plus you get
a whole lot of good stuff like proxy support for free .

* I use a very complex "EOF"-pattern like \n\0\0\0\0\n and pray to my
gods that this pattern will never be part of the string.

Never do this. Murphy will bite you.

The other sensible option is to escape the data somehow: Base64 encode
it, or backwhack all newlines, or something, so that you can produce an
EOT string that you are *certain* isn't in your data. This is the
approach used by MIME emails; IMHO it is inferior to the HTTP approach.

Ben

 

[Tim Shoppa]

Problem: the string is very long and can contain virtually any character
(its the output of freeze() from the Storable-Module which serializes
even complex hashes)

So how does the Server know when the Client has finished sending its string?

I tried several attempts that all works, but I'm sure they will fail on
certain data:

* the client sends the length of the string first, so the server knows
how much data it needs to wait for. (this will sure fail when using
utf8-encoded data and client/server will be on different machines with
different perls)

What's wrong with the "do { use bytes; length(EXPR) }" idiom for
getting the length in bytes (as opposed to characters)?

* I use a very complex "EOF"-pattern like \n\0\0\0\0\n and pray to my
gods that this pattern will never be part of the string.

Even better is to, for example, escape or MIME-encode or uuencode the
data so that you know that the end character/pattern never occurs in
the string. This is in effect how SMTP and some other transports move
binary stuff while not getting trapped by delimiter strings.

I personally don't really like MIME encoding or uuencoding but they're
simple and lots of examples in CPAN.

I am sure that there is a much better/simple/prooved solution to this
problem that must be a standard-problem.

Actually, you seem to have formulated the two common solutions yourself
but just missed some of the "as commonly implemented" details.

Tim.

 

[Tim Shoppa]

Never do this. Murphy will bite you.

Nevertheless it is used in a couple of well-known protocols and a
couple of commercial products. Yeah, it sucks rocks, but it's out there
in the wild (rarely for the better and as you point out always for the
worse).

The other sensible option is to escape the data somehow: Base64 encode
it, or backwhack all newlines, or something, so that you can produce an
EOT string that you are *certain* isn't in your data. This is the
approach used by MIME emails; IMHO it is inferior to the HTTP approach.

Sometimes (especially when moving stuff over "all the world's a stream
of bytes" paradigms like pipes and stdin/stdout) it is pressed into
service, despite its shortcomings, because nobody genuinely knows
beforehand how long the data is going tol be and buffering it all up is
foolhardy as well for either latency or storage limitations.

Tim.

 

[Ben Morrow]

Sometimes (especially when moving stuff over "all the world's a stream
of bytes" paradigms like pipes and stdin/stdout) it is pressed into
service, despite its shortcomings, because nobody genuinely knows
beforehand how long the data is going tol be and buffering it all up is
foolhardy as well for either latency or storage limitations.

....and the correct answer (again IMHO) in this case is to use HTTP
chunking (another goodie you get for free if you use HTTP as your
protocol). But this is now definitely OT...

Ben

 

[xhoster]

I use Net:aemon for a very simple server-client-szenario

client is sending a string to the server and the server is processing
the string and returning a answerstring to the client and the client is
terminating.

Problem: the string is very long and can contain virtually any character
(its the output of freeze() from the Storable-Module which serializes
even complex hashes)

So how does the Server know when the Client has finished sending its
string?

When the client's fd_retrieve call returns the object, then server must be
done sending it.

Xho