XML vs. cPickle

[Mike]

I know XML is more (processor) costly than cPickle, but how bad is it?
The idea is I want to store data that can be described as XML into my
database as cPickle objects. Except my web framework has no support for
BLOB datatype yet, and I might have to go with XML.

Ideas are appreciated,

Thanks,
Mike

 

[Alan Kennedy]

[Mike]

I know XML is more (processor) costly than cPickle, but how bad is it?

Are you sure you know that?

I'd guess that XML serialisation with cElementTree is both cpu and
memory competitive with cpickle, if not superior. Although I'm too lazy
to fire up the timeit module right now

Also, how quickly the relevant parsers work depends on the input, i.e.
your data structures. Only you can take measurements with your data
structures ....

The idea is I want to store data that can be described as XML

can != should

into my
database as cPickle objects. Except my web framework has no support for
BLOB datatype yet, and I might have to go with XML.

Or you could encode the binary pickle in a text-safe encoding such as
base64, and store the result in a text column. Although that will
obviously increase your processing time, both going in and out of the
database.

Ideas are appreciated,

I'd write a few simple prototypes and take some empirical measurements.

HTH,

 

[Mike]

I'd guess that XML serialisation with cElementTree is both cpu and

memory competitive with cpickle, if not superior. Although I'm too lazy
to fire up the timeit module right now

That maybe true, but I bet Marshal is the fastest. ...right?

Also, how quickly the relevant parsers work depends on the input, i.e.
your data structures. Only you can take measurements with your data
structures ....
True.

can != should

I certainly 'can', I don't think I should.

Or you could encode the binary pickle in a text-safe encoding such as
base64, and store the result in a text column. Although that will
obviously increase your processing time, both going in and out of the
database.

base64... (used to convert arbitrary binary data to plain text), sounds
fantastic. Except, I don't think I need it when marshaling anymore
since marshaling gives you clear text anyways. (right?)

I'd write a few simple prototypes and take some empirical measurements.

I am doing it now. Thanks,

Mike