Save/load like matlab?

[Sang-Ho Yun]

I wonder if there is a way to save and load all python variables just like
matlab does, so I can build a code step by step by loading previous states.

I am handling a python processing code for very large files and multiple
processing steps. Each time I find a bug, I have to run the whole thing
again, which is time consuming.

Thank you,
Sang-Ho

 

[Daniel Fetchinson]

I wonder if there is a way to save and load all python variables just like

matlab does, so I can build a code step by step by loading previous states.

I am handling a python processing code for very large files and multiple
processing steps. Each time I find a bug, I have to run the whole thing
again, which is time consuming.

Perhaps pickle is the thing you are looking for?

http://docs.python.org/library/pickle.html

HTH,
Daniel

 

[Stef Mientki]

Perhaps pickle is the thing you are looking for?

http://docs.python.org/library/pickle.html

or even cpickle which is lot faster
cheers,
Stef

 

[geremy condra]

I wonder if there is a way to save and load all python variables just like
matlab does, so I can build a code step by step by loading previous states.

I am handling a python processing code for very large files and multiple
processing steps. Each time I find a bug, I have to run the whole thing
again, which is time consuming.

Thank you,
Sang-Ho

You may also be interested in Sage if you're trying to do Matlab-like
things in Python.

http://www.sagemath.org/

Geremy Condra

 

[Robert Kern]

I wonder if there is a way to save and load all python variables just like
matlab does, so I can build a code step by step by loading previous states.

I am handling a python processing code for very large files and multiple
processing steps. Each time I find a bug, I have to run the whole thing
again, which is time consuming.

Test each piece of code in isolation from the rest rather than relying on a
complete run to test everything in one go.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

Test each piece of code in isolation from the rest rather than relying on a
complete run to test everything in one go.

Having said that, you will want to take a look into using joblib to structure
your larger runs:

http://pypi.python.org/pypi/joblib

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Brian Blais]

A year ago or so I designed a simple file format that could do that
and is also human readable (binary data is compressed and then
base64 encoded). I use it extensively to store experiment data for
my research and also for configuration files for two open source
projects that I own:
http://code.google.com/p/ssdf/

this is fantastic! what a great format! I've been looking for
something like this for quite some time.

thanks!


bb

 

[Mark Leander]

A year ago or so I designed a simple file format that could do that and is also
human readable (binary data is compressed and then base64 encoded). I use it
extensively to store experiment data for my research and also for configuration
files for two open source projects that I own:http://code.google.com/p/ssdf/

That looks quite nice! Thank you for sharing!

I see that at least in some case you use the generally unsafe eval() for parsing.
For instance the following will consume CPU and memory for quite a while:

ssdf.loads(u'a = [0xffffffffffffffff**0xffffffffffffffff]')

Regards
Mark

http://blog.topicbranch.net/2010/08/ssdf-simple-structured-data-format.html