help on pickle tool

[virg]

Hi,
i have client-server application which is written in python using
XMLRPC protocol. The existing client is a command line. Now client
application we are converting it as Web UI using java. I have seen some
problems in writing a java client. At the server for each request from
client, the server sends a response in hashtable and is serialized
using "pickle". The python function we call at the server is

pickle.dumps(r, 2) -- where r is a hash table which needs to be
serialized

At the client (existing client which in python) we use this call to
get the original data.

r = pickle.loads(result)

Since i am writing this client as Web UI client using java, i am not
able to deserialize this data
using java function "ObjectInputStream" and "ObjectInputStream" which
are most common functions for deserialization and i getting error as
invalid header.

Is it possible to deserialize the data by java which serialized by
Python or is there any compatibility issue. Is there any equivalent
pickle tool on java which supports this operation. so that i can use
across languages.

Because of this problem i am not able to proceed further. Any body has
any pointers for this problem. Any help is highly appreciated

Thanks in advance

regards
Virg

 

[hanumizzle]

Hi,
i have client-server application which is written in python using
XMLRPC protocol. The existing client is a command line. Now client
application we are converting it as Web UI using java. I have seen some
problems in writing a java client. At the server for each request from
client, the server sends a response in hashtable and is serialized
using "pickle". The python function we call at the server is...

What kind of data are we talking about? Is it strictly dictionary /
list stuff or is it more complex (functions & other objects)? I can
see 2 possible options: Jython or YAML. (I have been answering a lot
of questions re: serialization today and mentioned YAML in all of them
)

 

[Paddy]

Hi,
i have client-server application which is written in python using
XMLRPC protocol. The existing client is a command line. Now client
application we are converting it as Web UI using java. I have seen some
problems in writing a java client. At the server for each request from
client, the server sends a response in hashtable and is serialized
using "pickle". The python function we call at the server is

pickle.dumps(r, 2) -- where r is a hash table which needs to be
serialized

At the client (existing client which in python) we use this call to
get the original data.

r = pickle.loads(result)

Since i am writing this client as Web UI client using java, i am not
able to deserialize this data
using java function "ObjectInputStream" and "ObjectInputStream" which
are most common functions for deserialization and i getting error as
invalid header.

Is it possible to deserialize the data by java which serialized by
Python or is there any compatibility issue. Is there any equivalent
pickle tool on java which supports this operation. so that i can use
across languages.

Because of this problem i am not able to proceed further. Any body has
any pointers for this problem. Any help is highly appreciated

Thanks in advance

regards
Virg

You might try picking the data with a different pickle formatter that
your Java can use. Maybe an XML pickler
(http://www.gnosis.cx/download/Gnosis_Utils.More/Gnosis_Utils-1.2.1.ANNOUNCE
untested by me).
You might also use a JSON/YAML pickler. JSON is now a subset of YAML:
http://cheeseshop.python.org/pypi/python-json/3.4
http://pyyaml.org/wiki/PyYAML
http://jyaml.sourceforge.net/
http://www.yaml.org/
http://www.json.org/
http://www.json.org/java/

- Paddy.

 

[hanumizzle]

You might try picking the data with a different pickle formatter that
your Java can use. Maybe an XML pickler
(http://www.gnosis.cx/download/Gnosis_Utils.More/Gnosis_Utils-1.2.1.ANNOUNCE
untested by me).
You might also use a JSON/YAML pickler. JSON is now a subset of YAML:
http://cheeseshop.python.org/pypi/python-json/3.4
http://pyyaml.org/wiki/PyYAML
http://jyaml.sourceforge.net/
http://www.yaml.org/
http://www.json.org/
http://www.json.org/java/

Why a subset?

 

[MonkeeSage]

Why a subset?

I don't think JSON is a subset of YAML.

Regards,
Jordan

 

[hanumizzle]

I don't think JSON is a subset of YAML.

Apparent slip of the fingers by OP. From JSON website:

JSON (JavaScript Object Notation) is a lightweight data-interchange
format. It is easy for humans to read and write. It is easy for
machines to parse and generate. It is based on a subset of the
JavaScript Programming Language, Standard ECMA-262 3rd Edition -
December 1999. JSON is a text format that is completely language
independent but uses conventions that are familiar to programmers of
the C-family of languages, including C, C++, C#, Java, JavaScript,
Perl, Python, and many others. These properties make JSON an ideal
data-interchange language.

I'm happy with my Pythonesque YAML syntax, thank you.

 

[MonkeeSage]

I'm happy with my Pythonesque YAML syntax, thank you.

YAML is a little more complex, and a little more mature. But JSON
should not be ruled out. I actually like JSON personally.

Regards,
Jordan

 

[hanumizzle]

YAML is a little more complex, and a little more mature. But JSON
should not be ruled out. I actually like JSON personally.

I guess I'll keep an open mind. But I like editing YAML for the same
reason that I like editing Python. (Although I admit I usually I gen
my data structures in ipython and then dump them.)

-- Theerasak

 

[Fredrik Lundh]

MonkeeSage wrote:

YAML is a little more complex

a little? when did you last look at the spec?

and a little more mature.

than JavaScript's expression syntax? are you sure you're not confusing
libraries with standards here? (has anyone even managed to write a YAML
library that's small and simple enough to be "obviously correct"?)

</F>

 

[hanumizzle]

MonkeeSage wrote:



a little? when did you last look at the spec?


than JavaScript's expression syntax? are you sure you're not confusing
libraries with standards here? (has anyone even managed to write a YAML
library that's small and simple enough to be "obviously correct"?)

Tell. I'm interested in knowing.

-- Theerasak

 

[Fredrik Lundh]

I guess I'll keep an open mind. But I like editing YAML for the same
reason that I like editing Python.

JSON is almost identical to Python's expression syntax, of course, while
YAML isn't even close.

</F>

 

[hanumizzle]

JSON is almost identical to Python's expression syntax, of course, while
YAML isn't even close.

Getting the source now. S'pose it isn't too late to convert my project over...

-- Theerasak

 

[MonkeeSage]

when did you last look at the spec?

I'm fairly versed in JS objects, having written 10 or so extensions for
firefox; but I've only used YAML for trivial tasks like config files.
So I can't really say how they stack up in "the big picture". But from
what I have seen, JSON is a simpler and easier to use format than YAML.

Regards,
Jordan

 

[hanumizzle]

I'm fairly versed in JS objects, having written 10 or so extensions for
firefox; but I've only used YAML for trivial tasks like config files.
So I can't really say how they stack up in "the big picture". But from
what I have seen, JSON is a simpler and easier to use format than YAML.

That and it sounds like Strongbad pronouncing the name 'Jason'.

-- Theerasak

 

[virg]

Hi,
The data is simple dictionary with one or more keys. If i use YAML at
the client (webui) do i have to change serialisation method to YAML at
server also. Without changing serialisation method at server, can i use
any of the deserialisation methods at the client. We cannot change the
serialisation methods at the server since it is not under our control.

Thanks & reagards
virg

 

[hanumizzle]

Hi,
The data is simple dictionary with one or more keys. If i use YAML at
the client (webui) do i have to change serialisation method to YAML at
server also. Without changing serialisation method at server, can i use
any of the deserialisation methods at the client. We cannot change the
serialisation methods at the server since it is not under our control.

Oh, poopy.

What do you have at the server end?

-- Theerasak

 

[Steve Holden]

MonkeeSage wrote:





a little? when did you last look at the spec?




than JavaScript's expression syntax? are you sure you're not confusing
libraries with standards here? (has anyone even managed to write a YAML
library that's small and simple enough to be "obviously correct"?)

I have to agree that YAML, having started out with simplicity in mind,
has become a monster that threatens to collapse under its own weight.
The very existence of JSON is a good indicator that YAML has failed to
meet its design goals for a significant proportion of application
developers.

regards
Steve

 

[virg]

At the server, based on client request it does some computations , it
sends the result as dictionary (serialized) to the client.

 

[hanumizzle]

I have to agree that YAML, having started out with simplicity in mind,
has become a monster that threatens to collapse under its own weight.
The very existence of JSON is a good indicator that YAML has failed to
meet its design goals for a significant proportion of application
developers.

I am looking at JSON, but YAML does work fine for my purposes w/ no
discernable disadvantages. Of course I didn't implement the library,
so I can't really speak with any kind of expertise on this matter. I'm
just saying that, at least at the front end, it's pretty simple to
use. (JMO)

-- Theerasak

 

[hanumizzle]

At the server, based on client request it does some computations , it
sends the result as dictionary (serialized) to the client.

If I interpret your message correctly, you are receiving a Python
dictionary object from the server. Yes? In this case, I guess it might
be necessary to use Jython. I'm drawing a blank otherwise.

(This may be naive, but can you write the client in Python as well?)

-- Theerasak