How to get all the variables in a python shell

Discussion in 'Python' started by lixinyi.23@gmail.com, May 29, 2008.

  1. Guest

    Hi!

    I'm currently working on a scientific computation software built in
    python.
    What I want to implement is a Matlab style command window <->
    workspace interaction.

    For example, you type 'a=1' in the command window, and you see a list
    item named 'a' in the workspace.
    You double click the icon of the item, and you see its value. You can
    modify the value of the list item,
    1 -> 100 etc, after which if you go back to the command window and
    type 'a' and press enter, you see that
    varable a's value has been changed to 100.

    So my question is : if you have two DOS command windows running under
    WINDOWS OS, how can you make them share the same internal variable
    buffer? Or is there any easier way to implement such kind of
    interaction?

    Maybe I could just build a small database to store all the values and
    access them from both programs, but chances are sometimes I have to
    deal with big arrays, and they will eat extra memory if I keep them in
    a database. Is there anyway to access a shell's local memory buffer?
    I tried to use shell.interp.locals() in wxPython, but there's too many
    variables in the list which I don't actually need.

    Come on guys, give me some ideas. Thanks in advance!
     
    , May 29, 2008
    #1
    1. Advertising

  2. Tim Golden Guest

    wrote:
    > I'm currently working on a scientific computation software built in
    > python.
    > What I want to implement is a Matlab style command window <->
    > workspace interaction.
    >
    > For example, you type 'a=1' in the command window, and you see a list
    > item named 'a' in the workspace.
    > You double click the icon of the item, and you see its value. You can
    > modify the value of the list item,
    > 1 -> 100 etc, after which if you go back to the command window and
    > type 'a' and press enter, you see that
    > varable a's value has been changed to 100.
    >
    > So my question is : if you have two DOS command windows running under
    > WINDOWS OS, how can you make them share the same internal variable
    > buffer? Or is there any easier way to implement such kind of
    > interaction?


    I stronly suggest you look at IPython [1]. To do what I think
    you're describing, you'd need to hack or reimplement the interpreter.
    And that's what they've done. ISTR that they even have a branch
    which is dealing with parallel instances.

    TJG

    [1] http://ipython.scipy.org/moin/
     
    Tim Golden, May 29, 2008
    #2
    1. Advertising

  3. Tim Golden Guest

    wrote:
    > I'm currently working on a scientific computation software built in
    > python.
    > What I want to implement is a Matlab style command window <->
    > workspace interaction.
    >
    > For example, you type 'a=1' in the command window, and you see a list
    > item named 'a' in the workspace.
    > You double click the icon of the item, and you see its value. You can
    > modify the value of the list item,
    > 1 -> 100 etc, after which if you go back to the command window and
    > type 'a' and press enter, you see that
    > varable a's value has been changed to 100.
    >
    > So my question is : if you have two DOS command windows running under
    > WINDOWS OS, how can you make them share the same internal variable
    > buffer? Or is there any easier way to implement such kind of
    > interaction?


    I stronly suggest you look at IPython [1]. To do what I think
    you're describing, you'd need to hack or reimplement the interpreter.
    And that's what they've done. ISTR that they even have a branch
    which is dealing with parallel instances.

    TJG

    [1] http://ipython.scipy.org/moin/
     
    Tim Golden, May 29, 2008
    #3
  4. RE: [python-win32] How to get all the variables in a python shell

    -----Original Message-----
    > From: [mailto:p] On Behalf Of Tim Golden
    > Sent: Thursday, May 29, 2008 4:11 AM
    > To: Python-Win32 List;
    > Cc: Python-Win32 List
    > Subject: Re: [python-win32] How to get all the variables in a python shell
    >
    > wrote:
    > > I'm currently working on a scientific computation software built in
    > > python.
    > > What I want to implement is a Matlab style command window <->
    > > workspace interaction.
    > >
    > > For example, you type 'a=1' in the command window, and you see a list
    > > item named 'a' in the workspace.
    > > You double click the icon of the item, and you see its value. You can
    > > modify the value of the list item,
    > > 1 -> 100 etc, after which if you go back to the command window and
    > > type 'a' and press enter, you see that
    > > varable a's value has been changed to 100.
    > >
    > > So my question is : if you have two DOS command windows running under
    > > WINDOWS OS, how can you make them share the same internal variable
    > > buffer? Or is there any easier way to implement such kind of
    > > interaction?

    >
    > I stronly suggest you look at IPython [1]. To do what I think
    > you're describing, you'd need to hack or reimplement the interpreter.
    > And that's what they've done. ISTR that they even have a branch
    > which is dealing with parallel instances.
    >
    > TJG
    >
    > [1] http://ipython.scipy.org/moin/
    > _______________________________________________
    > python-win32 mailing list
    >
    > http://mail.python.org/mailman/listinfo/python-win32



    I'd try looking at memcached (http://www.danga.com/memcached/apis.html). No hacking or reimplementation of the interpreter would be necessary, and there's a
    Python api available. I haven't used it for anything production related, but I have played with it a bit, and it's fast and stable.



    DISCLAIMER:
    This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and
    may contain legally privileged and/or confidential information. If you are not the intended recipient
    of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and
    any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify
    me and permanently delete the original and any copy of any e-mail and any printout thereof.
    E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept
    liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission.

    NOTICE REGARDING PRIVACY AND CONFIDENTIALITY
    Direct Edge ECN LLC may, at its discretion, monitor and review the content of all e-mail communications.

    www.directedge.com
     
    Dahlstrom, Roger, May 29, 2008
    #4
  5. Paul Moore Guest

    Re: [python-win32] How to get all the variables in a python shell

    On 29/05/2008, Dahlstrom, Roger <> wrote:
    > I'd try looking at memcached (http://www.danga.com/memcached/apis.html).
    > No hacking or reimplementation of the interpreter would be necessary, and
    > there's a Python api available. I haven't used it for anything production related,
    > but I have played with it a bit, and it's fast and stable.


    Is memcached available for Windows, then? I've heard nice things about
    it, but thought it was Unix-only.

    Paul.
     
    Paul Moore, May 29, 2008
    #5
  6. RE: [python-win32] How to get all the variables in a python shell

    -----Original Message-----
    From: Paul Moore [mailto:p]
    Sent: Thursday, May 29, 2008 7:23 AM
    To: Dahlstrom, Roger
    Cc: Python-Win32 List;
    Subject: Re: [python-win32] How to get all the variables in a python shell

    On 29/05/2008, Dahlstrom, Roger <> wrote:
    > I'd try looking at memcached (http://www.danga.com/memcached/apis.html).
    > No hacking or reimplementation of the interpreter would be necessary, and
    > there's a Python api available. I haven't used it for anything production related,
    > but I have played with it a bit, and it's fast and stable.


    Is memcached available for Windows, then? I've heard nice things about
    it, but thought it was Unix-only.

    Paul.


    ----------------------------

    It is available for Windows, yes. As a matter of fact, I've never used it on Unix.


    DISCLAIMER:
    This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and
    may contain legally privileged and/or confidential information. If you are not the intended recipient
    of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and
    any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify
    me and permanently delete the original and any copy of any e-mail and any printout thereof.
    E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept
    liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission.

    NOTICE REGARDING PRIVACY AND CONFIDENTIALITY
    Direct Edge ECN LLC may, at its discretion, monitor and review the content of all e-mail communications.

    www.directedge.com
     
    Dahlstrom, Roger, May 29, 2008
    #6
  7. Tim Golden Guest

    Re: [python-win32] How to get all the variables in a python shell

    Tim Golden wrote:
    > wrote:
    >> I'm currently working on a scientific computation software built in
    >> python.
    >> What I want to implement is a Matlab style command window <->
    >> workspace interaction.
    >>
    >> For example, you type 'a=1' in the command window, and you see a list
    >> item named 'a' in the workspace.
    >> You double click the icon of the item, and you see its value. You can
    >> modify the value of the list item,
    >> 1 -> 100 etc, after which if you go back to the command window and
    >> type 'a' and press enter, you see that
    >> varable a's value has been changed to 100.
    >>
    >> So my question is : if you have two DOS command windows running under
    >> WINDOWS OS, how can you make them share the same internal variable
    >> buffer? Or is there any easier way to implement such kind of
    >> interaction?

    >
    > I stronly suggest you look at IPython [1]. To do what I think
    > you're describing, you'd need to hack or reimplement the interpreter.
    > And that's what they've done. ISTR that they even have a branch
    > which is dealing with parallel instances.


    Sorry, having seen Roger D's memcached suggestion, I realise I may
    have misinterpreted your requirement. I thought that you wanted
    a Python interpreter session to share its objects to another window,
    hence my IPython suggestion. If you're writing your own console app
    and want to share stuff, then there's any number of IPC possibilities.

    Roger's already mentioned memcached which I've no more than a
    passing knowledge of. But Pyro [1] is often a good bet for these
    things, and the pyprocessing [2] module is gaining a fair bit of
    traction at the moment. (To name just two out of many).

    TJG

    [1] http://pyro.sf.net
    [2] http://pyprocessing.berlios.de/
     
    Tim Golden, May 29, 2008
    #7
  8. Tim Golden Guest

    Re: [python-win32] How to get all the variables in a python shell

    <<< No Message Collected >>>
     
    Tim Golden, May 29, 2008
    #8
  9. Tim Golden Guest

    Re: [python-win32] How to get all the variables in a python shell

    [Bizarrely, my mail system seems to be randomly misfiring.
    If you've already seen this, please ignore!]

    Tim Golden wrote:
    Sorry, having seen Roger D's memcached suggestion, I realise I may
    have misinterpreted your requirement. I thought that you wanted
    a Python interpreter session to share its objects to another window,
    hence my IPython suggestion. If you're writing your own console app
    and want to share stuff, then there's any number of IPC possibilities.

    Roger's already mentioned memcached which I've no more than a
    passing knowledge of. But Pyro [1] is often a good bet for these
    things, and the pyprocessing [2] module is gaining a fair bit of
    traction at the moment. (To name just two out of many).

    TJG

    [1] http://pyro.sf.net
    [2] http://pyprocessing.berlios.de/
     
    Tim Golden, May 29, 2008
    #9
  10. Guest

    Your project interests me. Actually I was thinking about doing the
    same. I hadn't worked on it at all, but I though about it and had the
    idea about reading the session namespace directly, which I though
    would be stored in the __dict__ attribute of something.

    After reading your post, I have been trying a little bit, and I have
    found a way to do it with ipython. If you open an ipython console,
    press _ then hit TAB, you'll see it stores some useful information,
    including all input, all output, and after some searching, a
    dictionary matching all variables to its values.

    __IPYTHON__.user_ns

    There is a little extra stuff in there that you don't want, but that
    can be easily filtered (the extra stuff is either 'In', 'Out', 'help'
    or starts with '_'). I've tried it, and you can change the value in
    that dict to alter the value of the real variable. Say you have a
    variable 'test':

    test=5
    __IPYTHON__.user_ns['test']=4
    print test #prints 5

    If I get it right, python is a dynamic language, and you won't break
    things by messing around with its inner stuff like this, but you
    better check it.

    Is this what you had in mind?
     
    , May 30, 2008
    #10
  11. Guest

    I meant it prints 4, which means the value of test is modified by the
    access to the dict

    > test=5
    > __IPYTHON__.user_ns['test']=4
    > print test #prints 4
    >
     
    , May 31, 2008
    #11
  12. I'm not certain if this is what you want but try this for the first
    window:

    import __main__
    localvars = __main__.__dict__
    dir(localvars) # lists names of all objects available to the
    interpreter.

    And then you can pass localvars anywhere in the program - so after a
    command is entered in one window, you can import the all locals into
    the other by putting this function in the second window:

    def SetWindow2Locals(localvars):
    __main__.__dict__ = localvars

    and do the reverse by sending the updated localvars from window 2 to
    window 1. Setting up an object between the 2 windows should be enough:

    # window 1, must be created first
    import __main__
    localvars = __main__.__dict__
    # create second window (say, win2)
    win2.SetLocals(localvars)

    in win2 after creation, there is a method/function:

    SetLocals(localvars):
    __main__.__dict__ = localvars

    If you want to store the object in something like a DB, you can store
    this localvars dictionary rather than the actual objects. This should
    be efficient because it stores the object name and ID only but not the
    objects themselves. However, if the objects are deleted, after saving
    and before re-loading, you may get some odd results but I'm not sure.
    I would be very careful of using this in the wild.

    You can also add objects singly to the dictionary by hand with a
    function:

    def importobject(obj):
    __main__.__dict__[obj.__name__] = obj

    Which should make them available in the other window easily. If a new
    object is added to the __main__.__dict__, just grab it and send it to
    this function. Make sure that you send the object and not just its
    name because the name is only a string.

    Sorry if this is not what you were after.

    Alan

    On May 29, 2:47 pm, wrote:
    > Hi!
    >
    > I'm currently working on a scientific computation software built in
    > python.
    > What I want to implement is a Matlab style command window <->
    > workspace interaction.
    >
    > For example, you type 'a=1' in the command window, and you see a list
    > item named 'a' in the workspace.
    > You double click the icon of the item, and you see its value. You can
    > modify the value of the list item,
    > 1 -> 100 etc, after which if you go back to the command window and
    > type 'a' and press enter, you see that
    > varable a's value has been changed to 100.
    >
    > So my question is : if you have two DOS command windows running under
    > WINDOWS OS, how can you make them share the same internal variable
    > buffer? Or is there any easier way to implement such kind of
    > interaction?
    >
    > Maybe I could just build a small database to store all the values and
    > access them from both programs, but chances are sometimes I have to
    > deal with big arrays, and they will eat extra memory if I keep them in
    > a database. Is there anyway to access a shell's local memory buffer?
    > I tried to use shell.interp.locals() in wxPython, but there's too many
    > variables in the list which I don't actually need.
    >
    > Come on guys, give me some ideas. Thanks in advance!
     
    Alan J. Salmoni, May 31, 2008
    #12
  13. Lie Guest

    wrote:
    > Hi!
    >
    > I'm currently working on a scientific computation software built in
    > python.
    > What I want to implement is a Matlab style command window <->
    > workspace interaction.
    >
    > For example, you type 'a=1' in the command window, and you see a list
    > item named 'a' in the workspace.
    > You double click the icon of the item, and you see its value. You can
    > modify the value of the list item,
    > 1 -> 100 etc, after which if you go back to the command window and
    > type 'a' and press enter, you see that
    > varable a's value has been changed to 100.
    >
    > So my question is : if you have two DOS command windows running under
    > WINDOWS OS, how can you make them share the same internal variable
    > buffer? Or is there any easier way to implement such kind of
    > interaction?
    >
    > Maybe I could just build a small database to store all the values and
    > access them from both programs, but chances are sometimes I have to
    > deal with big arrays, and they will eat extra memory if I keep them in
    > a database. Is there anyway to access a shell's local memory buffer?
    > I tried to use shell.interp.locals() in wxPython, but there's too many
    > variables in the list which I don't actually need.
    >
    > Come on guys, give me some ideas. Thanks in advance!


    In all kinds of code, it's best to seperate the workers code and the
    UI code, in your case, you should create a backend (worker), which is
    a class that stands on its own, and two foreground class (UI) that is
    completely independent of each other but have the same interface. The
    backend code would have an event that is raised when it is changed to
    notify the UI (esp. The gui one) that it has changed since last time,
    possibly passing info on what have been changed. The front ends, would
    watch for this event as necessary (I don't think it is necessary for
    the command line to watch this event) and react to it as necessary
    like refreshing the view. The front-end window may only call functions
    on the backend to interact with the data being worked on (in short the
    backend class is opaque).

    This obliviate the need to share data between the two (or more)
    windows (UI) because all the data are contained in the backend class
    that the frontend can't access directly.
     
    Lie, Jun 1, 2008
    #13
  14. Lie Guest

    On Jun 2, 1:29 am, Lie <> wrote:
    > wrote:
    > > Hi!

    >
    > > I'm currently working on a scientific computation software built in
    > > python.
    > > What I want to implement is a Matlab style command window <->
    > > workspace interaction.

    >
    > > For example, you type 'a=1' in the command window, and you see a list
    > > item named 'a' in the workspace.
    > > You double click the icon of the item, and you see its value. You can
    > > modify the value of the list item,
    > > 1 -> 100 etc,  after which if you go back to the command window and
    > > type 'a'  and press enter, you see that
    > > varable a's value has been changed to 100.

    >
    > > So my question is : if you have two DOS command windows running under
    > > WINDOWS OS, how can you make them share the same internal variable
    > > buffer? Or is there any easier way to implement such kind of
    > > interaction?

    >
    > > Maybe I could just build a small database to store all the values and
    > > access them from both programs, but chances are sometimes I have to
    > > deal with big arrays, and they will eat extra memory if I keep them in
    > > a database. Is there anyway to access a shell's local memory buffer?
    > > I tried to use shell.interp.locals() in wxPython, but there's too many
    > > variables in the list which I don't actually need.

    >
    > > Come on guys, give me some ideas. Thanks in advance!

    >
    > In all kinds of code, it's best to seperate the workers code and the
    > UI code, in your case, you should create a backend (worker), which is
    > a class that stands on its own, and two foreground class (UI) that is
    > completely independent of each other but have the same interface. The
    > backend code would have an event that is raised when it is changed to
    > notify the UI (esp. The gui one) that it has changed since last time,
    > possibly passing info on what have been changed. The front ends, would
    > watch for this event as necessary (I don't think it is necessary for
    > the command line to watch this event) and react to it as necessary
    > like refreshing the view. The front-end window may only call functions
    > on the backend to interact with the data being worked on (in short the
    > backend class is opaque).
    >
    > This obliviate the need to share data between the two (or more)
    > windows (UI) because all the data are contained in the backend class
    > that the frontend can't access directly.


    To clarify what I meant, the front ends should never contain any
    working data except the ones needed for the UI to illustrate what it
    wanted to show at the moment, and even then, it is accessed in read
    only fashion.

    And actually because of Python's Global Interpreter Lock, which means
    that your program would all be contained in the same python
    interpreter instance (unless you do some workarounds), passing objects/
    lists around between python program is cheap because they're just a
    "reference" passing (like pointer passing in C/C++)

    This approach is a simple "server-client" method (not a true server-
    client method though, since a true one cannot share unserialized
    data), and is extremely scalable, it's easy to add a third window for
    example, there is no need for every front end to be aware that there
    are other front ends, since it just watches for the "Changed" event
    from the backend.
     
    Lie, Jun 1, 2008
    #14
  15. Lie Guest

    On May 29, 1:47 pm, wrote:
    > Hi!
    >
    > I'm currently working on a scientific computation software built in
    > python.
    > What I want to implement is a Matlab style command window <->
    > workspace interaction.
    >
    > For example, you type 'a=1' in the command window, and you see a list
    > item named 'a' in the workspace.
    > You double click the icon of the item, and you see its value. You can
    > modify the value of the list item,
    > 1 -> 100 etc,  after which if you go back to the command window and
    > type 'a'  and press enter, you see that
    > varable a's value has been changed to 100.
    >
    > So my question is : if you have two DOS command windows running under
    > WINDOWS OS, how can you make them share the same internal variable
    > buffer? Or is there any easier way to implement such kind of
    > interaction?
    >
    > Maybe I could just build a small database to store all the values and
    > access them from both programs, but chances are sometimes I have to
    > deal with big arrays, and they will eat extra memory if I keep them in
    > a database. Is there anyway to access a shell's local memory buffer?
    > I tried to use shell.interp.locals() in wxPython, but there's too many
    > variables in the list which I don't actually need.
    >
    > Come on guys, give me some ideas. Thanks in advance!


    As an addition: Don't try to share data between windows, it's messy,
    fragile, and easy to make bugs.

    PS: Do not confuse Lie (Me) and Lee (OP)
     
    Lie, Jun 1, 2008
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Christian Heimes
    Replies:
    0
    Views:
    611
    Christian Heimes
    Feb 27, 2008
  2. Gerardo Herzig
    Replies:
    1
    Views:
    1,106
    Philipp Pagel
    Feb 27, 2008
  3. D'Arcy J.M. Cain
    Replies:
    0
    Views:
    879
    D'Arcy J.M. Cain
    Feb 27, 2008
  4. Lee
    Replies:
    1
    Views:
    322
  5. edwithad

    Bash shell to Python shell?

    edwithad, Sep 28, 2009, in forum: Python
    Replies:
    4
    Views:
    1,767
Loading...

Share This Page