Object Diffs

[Croepha]

Hello Python list:

I am doing research into doing network based propagation of python
objects. In order to maintain network efficiency. I wan't to just
send the differences of python objects, I was wondering if there
was/is any other research or development in this area? I was thinking
that I could look at how pickle works an implement a diff system
there, or I could actually create a pickle of the object and then use
difflib to compare the flattened text...

Ideas, or comments welcome

Thanks,
Dave Butler

 

[Steven D'Aprano]

Hello Python list:

I am doing research into doing network based propagation of python
objects. In order to maintain network efficiency. I wan't to just
send the differences of python objects,

Can you explain in more detail what you mean?

Are there constraints on what the objects can be? If you have two objects:

a = None
b = (1, 2, "hello world", [])

how would you describe the diff between a and b?

If a and b are both numbers, the diff between them is just a-b, but this
itself is a number and so transmitting it takes exactly as much memory as
just sending b.

Obviously if a and b are strings, the problem becomes trivial (just
transmitted the string diff, which is a solved problem).

I was wondering if there
was/is any other research or development in this area? I was thinking
that I could look at how pickle works an implement a diff system
there, or I could actually create a pickle of the object and then use
difflib to compare the flattened text...

The source code for pickle is available. See the pickle.py and
pickletools.py files in your Python installation.

Also, these resources may help:

http://wiki.python.org/moin/DistributedProgramming
http://wiki.python.org/moin/ParallelProcessing

 

[Irmen de Jong]

Hello Python list:

I am doing research into doing network based propagation of python
objects. In order to maintain network efficiency. I wan't to just
send the differences of python objects, I was wondering if there
was/is any other research or development in this area? I was thinking
that I could look at how pickle works an implement a diff system
there, or I could actually create a pickle of the object and then use
difflib to compare the flattened text...

Ideas, or comments welcome

No Python code, but you may want to have a look at the DeltaCompressor
of the Kryo serializer (written for Java).
http://code.google.com/p/kryo/

(disclaimer: I haven't used it, just read a little bit about it)

Irmen