How to get all the variables in a python shell

[lixinyi.23]

Hi!

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -> 100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
a database. Is there anyway to access a shell's local memory buffer?
I tried to use shell.interp.locals() in wxPython, but there's too many
variables in the list which I don't actually need.

Come on guys, give me some ideas. Thanks in advance!

 

[Tim Golden]

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -> 100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

I stronly suggest you look at IPython [1]. To do what I think
you're describing, you'd need to hack or reimplement the interpreter.
And that's what they've done. ISTR that they even have a branch
which is dealing with parallel instances.

TJG

[1] http://ipython.scipy.org/moin/

 

[Tim Golden]

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -> 100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

I stronly suggest you look at IPython [1]. To do what I think
you're describing, you'd need to hack or reimplement the interpreter.
And that's what they've done. ISTR that they even have a branch
which is dealing with parallel instances.

TJG

[1] http://ipython.scipy.org/moin/

 

[Dahlstrom, Roger]

-----Original Message-----

From: (e-mail address removed) [mailto[email protected]] On Behalf Of Tim Golden
Sent: Thursday, May 29, 2008 4:11 AM
To: Python-Win32 List; (e-mail address removed)
Cc: Python-Win32 List
Subject: Re: [python-win32] How to get all the variables in a python shell

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -> 100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

I stronly suggest you look at IPython [1]. To do what I think
you're describing, you'd need to hack or reimplement the interpreter.
And that's what they've done. ISTR that they even have a branch
which is dealing with parallel instances.

TJG

[1] http://ipython.scipy.org/moin/
_______________________________________________
python-win32 mailing list
(e-mail address removed)
http://mail.python.org/mailman/listinfo/python-win32

I'd try looking at memcached (http://www.danga.com/memcached/apis.html). No hacking or reimplementation of the interpreter would be necessary, and there's a
Python api available. I haven't used it for anything production related, but I have played with it a bit, and it's fast and stable.



DISCLAIMER:
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and
may contain legally privileged and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and
any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify
me and permanently delete the original and any copy of any e-mail and any printout thereof.
E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept
liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission.

NOTICE REGARDING PRIVACY AND CONFIDENTIALITY
Direct Edge ECN LLC may, at its discretion, monitor and review the content of all e-mail communications.

www.directedge.com

 

[Paul Moore]

I'd try looking at memcached (http://www.danga.com/memcached/apis.html).
No hacking or reimplementation of the interpreter would be necessary, and
there's a Python api available. I haven't used it for anything production related,
but I have played with it a bit, and it's fast and stable.

Is memcached available for Windows, then? I've heard nice things about
it, but thought it was Unix-only.

Paul.

 

[Dahlstrom, Roger]

-----Original Message-----
From: Paul Moore [mailto[email protected]]
Sent: Thursday, May 29, 2008 7:23 AM
To: Dahlstrom, Roger
Cc: Python-Win32 List; (e-mail address removed)
Subject: Re: [python-win32] How to get all the variables in a python shell

I'd try looking at memcached (http://www.danga.com/memcached/apis.html).
No hacking or reimplementation of the interpreter would be necessary, and
there's a Python api available. I haven't used it for anything production related,
but I have played with it a bit, and it's fast and stable.

Is memcached available for Windows, then? I've heard nice things about
it, but thought it was Unix-only.

Paul.


----------------------------

It is available for Windows, yes. As a matter of fact, I've never used it on Unix.


DISCLAIMER:
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and
may contain legally privileged and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and
any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify
me and permanently delete the original and any copy of any e-mail and any printout thereof.
E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept
liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission.

NOTICE REGARDING PRIVACY AND CONFIDENTIALITY
Direct Edge ECN LLC may, at its discretion, monitor and review the content of all e-mail communications.

www.directedge.com

 

[Tim Golden]

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -> 100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

I stronly suggest you look at IPython [1]. To do what I think
you're describing, you'd need to hack or reimplement the interpreter.
And that's what they've done. ISTR that they even have a branch
which is dealing with parallel instances.

Sorry, having seen Roger D's memcached suggestion, I realise I may
have misinterpreted your requirement. I thought that you wanted
a Python interpreter session to share its objects to another window,
hence my IPython suggestion. If you're writing your own console app
and want to share stuff, then there's any number of IPC possibilities.

Roger's already mentioned memcached which I've no more than a
passing knowledge of. But Pyro [1] is often a good bet for these
things, and the pyprocessing [2] module is gaining a fair bit of
traction at the moment. (To name just two out of many).

TJG

[1] http://pyro.sf.net
[2] http://pyprocessing.berlios.de/

 

[Tim Golden]

<<< No Message Collected >>>

 

[Tim Golden]

[Bizarrely, my mail system seems to be randomly misfiring.
If you've already seen this, please ignore!]

Tim Golden wrote:
Sorry, having seen Roger D's memcached suggestion, I realise I may
have misinterpreted your requirement. I thought that you wanted
a Python interpreter session to share its objects to another window,
hence my IPython suggestion. If you're writing your own console app
and want to share stuff, then there's any number of IPC possibilities.

Roger's already mentioned memcached which I've no more than a
passing knowledge of. But Pyro [1] is often a good bet for these
things, and the pyprocessing [2] module is gaining a fair bit of
traction at the moment. (To name just two out of many).

TJG

[1] http://pyro.sf.net
[2] http://pyprocessing.berlios.de/

 

[caca]

Your project interests me. Actually I was thinking about doing the
same. I hadn't worked on it at all, but I though about it and had the
idea about reading the session namespace directly, which I though
would be stored in the __dict__ attribute of something.

After reading your post, I have been trying a little bit, and I have
found a way to do it with ipython. If you open an ipython console,
press _ then hit TAB, you'll see it stores some useful information,
including all input, all output, and after some searching, a
dictionary matching all variables to its values.

__IPYTHON__.user_ns

There is a little extra stuff in there that you don't want, but that
can be easily filtered (the extra stuff is either 'In', 'Out', 'help'
or starts with '_'). I've tried it, and you can change the value in
that dict to alter the value of the real variable. Say you have a
variable 'test':

test=5
__IPYTHON__.user_ns['test']=4
print test #prints 5

If I get it right, python is a dynamic language, and you won't break
things by messing around with its inner stuff like this, but you
better check it.

Is this what you had in mind?

 

[caca]

I meant it prints 4, which means the value of test is modified by the
access to the dict

 

[Alan J. Salmoni]

I'm not certain if this is what you want but try this for the first
window:

import __main__
localvars = __main__.__dict__
dir(localvars) # lists names of all objects available to the
interpreter.

And then you can pass localvars anywhere in the program - so after a
command is entered in one window, you can import the all locals into
the other by putting this function in the second window:

def SetWindow2Locals(localvars):
__main__.__dict__ = localvars

and do the reverse by sending the updated localvars from window 2 to
window 1. Setting up an object between the 2 windows should be enough:

# window 1, must be created first
import __main__
localvars = __main__.__dict__
# create second window (say, win2)
win2.SetLocals(localvars)

in win2 after creation, there is a method/function:

SetLocals(localvars):
__main__.__dict__ = localvars

If you want to store the object in something like a DB, you can store
this localvars dictionary rather than the actual objects. This should
be efficient because it stores the object name and ID only but not the
objects themselves. However, if the objects are deleted, after saving
and before re-loading, you may get some odd results but I'm not sure.
I would be very careful of using this in the wild.

You can also add objects singly to the dictionary by hand with a
function:

def importobject(obj):
__main__.__dict__[obj.__name__] = obj

Which should make them available in the other window easily. If a new
object is added to the __main__.__dict__, just grab it and send it to
this function. Make sure that you send the object and not just its
name because the name is only a string.

Sorry if this is not what you were after.

Alan

 

[Lie]

Hi!

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -> 100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
a database. Is there anyway to access a shell's local memory buffer?
I tried to use shell.interp.locals() in wxPython, but there's too many
variables in the list which I don't actually need.

Come on guys, give me some ideas. Thanks in advance!

In all kinds of code, it's best to seperate the workers code and the
UI code, in your case, you should create a backend (worker), which is
a class that stands on its own, and two foreground class (UI) that is
completely independent of each other but have the same interface. The
backend code would have an event that is raised when it is changed to
notify the UI (esp. The gui one) that it has changed since last time,
possibly passing info on what have been changed. The front ends, would
watch for this event as necessary (I don't think it is necessary for
the command line to watch this event) and react to it as necessary
like refreshing the view. The front-end window may only call functions
on the backend to interact with the data being worked on (in short the
backend class is opaque).

This obliviate the need to share data between the two (or more)
windows (UI) because all the data are contained in the backend class
that the frontend can't access directly.

 

[Lie]

In all kinds of code, it's best to seperate the workers code and the
UI code, in your case, you should create a backend (worker), which is
a class that stands on its own, and two foreground class (UI) that is
completely independent of each other but have the same interface. The
backend code would have an event that is raised when it is changed to
notify the UI (esp. The gui one) that it has changed since last time,
possibly passing info on what have been changed. The front ends, would
watch for this event as necessary (I don't think it is necessary for
the command line to watch this event) and react to it as necessary
like refreshing the view. The front-end window may only call functions
on the backend to interact with the data being worked on (in short the
backend class is opaque).

This obliviate the need to share data between the two (or more)
windows (UI) because all the data are contained in the backend class
that the frontend can't access directly.

To clarify what I meant, the front ends should never contain any
working data except the ones needed for the UI to illustrate what it
wanted to show at the moment, and even then, it is accessed in read
only fashion.

And actually because of Python's Global Interpreter Lock, which means
that your program would all be contained in the same python
interpreter instance (unless you do some workarounds), passing objects/
lists around between python program is cheap because they're just a
"reference" passing (like pointer passing in C/C++)

This approach is a simple "server-client" method (not a true server-
client method though, since a true one cannot share unserialized
data), and is extremely scalable, it's easy to add a third window for
example, there is no need for every front end to be aware that there
are other front ends, since it just watches for the "Changed" event
from the backend.

 

[Lie]

Hi!

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -> 100 etc,  after which if you go back to the command window and
type 'a'  and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
a database. Is there anyway to access a shell's local memory buffer?
I tried to use shell.interp.locals() in wxPython, but there's too many
variables in the list which I don't actually need.

Come on guys, give me some ideas. Thanks in advance!

As an addition: Don't try to share data between windows, it's messy,
fragile, and easy to make bugs.

PS: Do not confuse Lie (Me) and Lee (OP)