Running external module and accessing the created objects

[Kene Meniru]

Program summary:

I have a module called user.py that imports another module called
app.py. Functions in app.py are used in user.py to describe 3D
objects. These objects are saved in another object described in
doc.py.

app.py contains a function called view(). When called in user.py, it
signals the end of object descriptions. Presently all objects
contained in doc.py are exported to either POV-Ray or OpenSCAD file
format depending on the argument given to view().

My Issues:

I have decided I want to provide a preview of the objects using opengl
(pyglet). So I am trying to create another module called appwin.py
which the user can launch with user.py as an argument. When each
object is described in user.py, I want the user to be able to switch
to appwin.py, provide a signal that makes appwin.py redraw the screen
to show any modifications (perhaps with the enter key).

I do not want to invest much time with appwin.py now as I am still
coding app.py. Right now, appwin.py just subclasses
pyglet.window.Window().

I do not want to merge app.py and appwin.py. I want them to be two
separate applications because I want to retain the option of either
console or many different window interfaces.

The problem then is:

How can I run appwin.py which will then execute user.py to create the
objects to be saved in doc.py. Then when view() is encountered to be
able to access the objects stored in doc.py in appwin.py?

Any ideas will help.

 

[Steven D'Aprano]

Program summary:

I have a module called user.py that imports another module called
app.py. Functions in app.py are used in user.py to describe 3D objects.
These objects are saved in another object described in doc.py.

What do you mean, "objects are saved in another object"?

app.py contains a function called view(). When called in user.py, it
signals the end of object descriptions. Presently all objects contained
in doc.py are exported to either POV-Ray or OpenSCAD file format
depending on the argument given to view().

My Issues:

I have decided I want to provide a preview of the objects using opengl
(pyglet). So I am trying to create another module called appwin.py which
the user can launch with user.py as an argument.

What happens if the user launches appwin with a different argument?

If appwin can only take one, compulsory, argument, then it's silly to
require it as an argument. Just have appwin automatically import user.py,
and do whatever it needs.

When each object is
described in user.py, I want the user to be able to switch to appwin.py,
provide a signal that makes appwin.py redraw the screen to show any
modifications (perhaps with the enter key).

This makes no sense to me. Are you saying that appwin opens a text editor
that allows the user to edit the user.py source code?

I do not want to invest much time with appwin.py now as I am still
coding app.py. Right now, appwin.py just subclasses
pyglet.window.Window().

I do not want to merge app.py and appwin.py. I want them to be two
separate applications because I want to retain the option of either
console or many different window interfaces.

Making them a single module still retains the option of console or many
different window interfaces.

The problem then is:

How can I run appwin.py which will then execute user.py to create the
objects to be saved in doc.py.

I don't know. How does user.py create the objects? Suppose it users a
function called "create". Then you would do this in appwin:


import user
user.create()

Then when view() is encountered to be
able to access the objects stored in doc.py in appwin.py?

What do you mean by "view() is encounted"?

How would you access the objects stored in doc.py? Suppose you access
them using a list called "list_of_objects". Then in appwin.py:

import doc
for obj in doc.list_of_objects:
do_something_with(obj)

where you have to write the function "do_something_with", to do whatever
it is you want to do.


By the way, calling a module "doc" which is not for *documentation* is a
bad idea.

 

[Kene Meniru]

Steven D'Aprano wrote:

What do you mean, "objects are saved in another object"?

doc.py has a dictionary. When the user describes a wall, it is passed
to the doc object to be saved in the dictionary.

What happens if the user launches appwin with a different argument?

If appwin can only take one, compulsory, argument, then it's silly to
require it as an argument. Just have appwin automatically import user.py,
and do whatever it needs.

I guess my description does not come across well.

I thought user.py sounded like it is, a user created file. It will be
named differently for each user to created different types of 3D
objects. It does not make sense to expect every user to name their
design file the same. So there will have to be a differently named
single argument for buildeswin.

This makes no sense to me. Are you saying that appwin opens a text editor
that allows the user to edit the user.py source code?

No. appwin opens an opengl (pyglet.window.Window()) graphics window as
I mentioned.

I don't know. How does user.py create the objects? Suppose it users a
function called "create". Then you would do this in appwin:


import user
user.create()

I have tried importing "user.py" and/or "app.py". However there is no
single command to call.

My program is designed to assist in the building design process. It is
a big program and the creation of building components takes quite a
few steps. I do not want to support these steps in a graphics window
which is why the user uses any text editor they prefer to create
"user.py". I want "appwin.py" (which has a graphics window) to be able
to access objects stored in "doc.py" (which has a dictionary) after a
command like "python user.py" so that the objects saved in "doc.py"
after execution can be retrieved and drawn in "appwin.py".

What do you mean by "view() is encounted"?

This is a command that the user can enter in user.py. app.py will then
encounter this command as python parses the file.

How would you access the objects stored in doc.py? Suppose you access
them using a list called "list_of_objects". Then in appwin.py:

import doc
for obj in doc.list_of_objects:
do_something_with(obj)

I have tried this but there are no objects found in the dictionary in
doc.py. I am guessing that I have to execute appwin.py so that it
shares the same namespace with user.py or maybe a way to access that
namespace. This is the reason for my question.

 

[Steven D'Aprano]

I thought user.py sounded like it is, a user created file.

No. It sounded like a file called literally "user.py".

As for the rest, I shall think about it, and hopefully either I or
someone else will write back later with suggestions.


P.S. your posts have the followup header set to Gmane. Please do not do
that. Many people are reading this via the comp.lang.python newsgroup, or
via email.

 

[Chris Angelico]

I have tried importing "user.py" and/or "app.py". However there is no
single command to call.

I haven't followed the thread in detail, but I gather you're trying to
import a file with a variable name? Instead of 'import user', try:

user = __import__("foobar")

You can then replace the quoted string with whatever you need (note,
leave off the .py extension). In your code, it'll be as if you did:

import foobar as user

but with the flexibility of using whatever run-time-chosen name you need.

ChrisA

 

[Dave Angel]

(lots of stuff that was more confusing to me than helpful)

You use the words launch, encountered, execute, and others in ways that
do not make sense to me, or are at least ambiguous.

You have an explicitly named user.py, which apparently is *not*
generally named that.

I could give you lots of random facts and suggestions, and one of the
might hit home. For example the __import__() function can import a
module that you don't know the name of ahead of time. It's not often
the right answer, though, so I hesitate to suggest it.

For another example, if you import a module by two different names, or
if you import the "module" that is your starting script, then you can
end up with two instances of such a module, with all sorts of negative
implications about global data or class attributes stored in that
module, or even more subtle problems.

For a final example, having a circular import tree can cause problems if
any of those imports have any global code (like class initialization,
defining constants, etc.). It's generally much better to define a
strict hierarchy of who imports whom.

But I think instead that it'd be better for you to make a clearer
statement about how your code is structured.

I'm guessing that all of this is intended to be in one executable -- no
child processes, etc. So don't say launch, say import, or
function-call, or whatever you are really doing.

I'm guessing that you're running this on Python 3.3 under Linux.

I'm guessing that "user.py" is one possible name that a particular user
calls his script. And that script is what he runs on the Python
commandline.

python user.py

And that script calls functions in app.py, doc.py, and/or appwin.py.
And that currently, you're trying to import user.py from one of your own
modules, for purposes of either callback functions or global data
access. That's a mistake. When you import your *script* (using
__import__() as suggested above) you get a new copy of the script, and
new copies of all its global data.

I can't even guess how you're intending to mix the commandline stuff of
app.py with gui stuff in one or more appwin.py variants. You'll have to
be explicit, if it even matters yet.

I suggest that rather than responding to these points, you restate your
problem, in one place, with coherent detail that eliminates these
guesses and replaces them with reality. Start with the environment this
is running in, and the commandline typically used to launch the code,
and the interdepencies of the various modules.

 

[Kene Meniru]

OK. Sorry to have caused all the confusion. Let me try this again.

To use my program the user needs a script file I will call user.py.
Functions from my program must be imported into this file with
something like "from myapp import *".

myapp.py is a module in my program that has all the functions that the
user needs to describe building components. There is also a main
controller object that is imported into myapp.py module called app.

When python parses user.py module, the functions the user has provided
creates building components which are then saved in a dictionary
located in an object called doc that is part of my program.

The user is free to use the functions in myapp.py to describe building
components. When this process is complete or when the user wants to
view their work-in-progress, they have to place a function called
view() on the last line in user.py. This function currently exports
the components into POV-Ray or OpenSCAD format. These are CAD programs
that read script files to render or create images of the described
artifacts. So this means that to view their work the user needs to run
python on user.py to export the building and then run POV-Ray or
OpenSCAD on the exported file to see the building.

I want to make it possible for the user to preview the building
components without having to use the external rendering programs. I
can currently have the app object provide this preview but it means
that the user will have to process user.py with python then exit the
graphic window each time they need to see changes.

So the solution I am looking for is to have a graphic window open that
watches user.py for changes. If there is a change the graphic window
updates the rendition of the created components without further
intervention by the user. However this means that the graphic must
somehow run python to parse user.py and then be able to access the
objects stored in doc so that the coordinates can be used to update
the view. This is where I am having difficulty.

I hope this is clearer.

 

[Dave Angel]

OK. Sorry to have caused all the confusion. Let me try this again.

Thank you very much. This is much clearer, though it's not all here.

To use my program the user needs a script file I will call user.py.
Functions from my program must be imported into this file with
something like "from myapp import *".

And does the user run this script by doing
python user.py

myapp.py is a module in my program that has all the functions that the
user needs to describe building components. There is also a main
controller object that is imported into myapp.py module called app.

When python parses user.py module,

You presumably mean, "When Python runs the script user.py"

the functions the user has provided
creates building components which are then saved in a dictionary
located in an object called doc that is part of my program.

What program is that? Looks to me like you're writing a series of
modules, a framework perhaps, that is invoked by the user script.

The user is free to use the functions in myapp.py to describe building
components. When this process is complete or when the user wants to
view their work-in-progress, they have to place a function called
view() on the last line in user.py.

Don't you mean the code in user.py has to *call* the function app.view()
when they're all done with defining the components? That has nothing to
do with being the last line.

This function currently exports

You mean writes files into the file system?

the components into POV-Ray or OpenSCAD format. These are CAD programs
that read script files to render or create images of the described
artifacts. So this means that to view their work the user needs to run
python on user.py to export the building and then run POV-Ray or
OpenSCAD on the exported file to see the building.

At that point, the user.py script has completed, and Python is done, right?

I want to make it possible for the user to preview the building
components without having to use the external rendering programs. I
can currently have the app object provide this preview but it means
that the user will have to process user.py with python then exit the
graphic window each time they need to see changes.

What changes are those? You can't change a script while it's executing.
Could you clarify this so I can rethink the following paragraph?

 

[Rick Johnson]

OK. Sorry to have caused all the confusion. Let me try
this again.

Sounds to me like you should solve this problem in two manners:

============================================================
Interactive Input
============================================================

Create an interactive environment where the user can enter commands directly into the namespace using your API in real time, then when he is ready to see the result of those commands, he can call "display command" or push a "display button" and the App will run the appropriate outside programs to show the visualization.

============================================================
Scripts loaded at runtime
============================================================

If you prefer the user to write "scripts", and then have your App read these scripts (and create visualizations from the commands within the scripts) then the only difference between step 1 and step 2 is *when* the commands are interpreted. In this case your app will *load* the scripts AFTER they are completely written (this could be done automatically by the app at runtime IF the path to these scripts is known, or could be accomplished by the user picking files in a dialog (Menu->RunScript).

I think you are trying to create step 1 without the interactive environment.. This is your mistake. Having users work with "raw files" and then *somehow* watching a "raw file" for certain "display commands" (in real time) is folly.

Heck, when you edit a file in a text editor you are only seeing a representation of the file from the last time it was saved, and your app could neverknow when the "file data" and the "file view" where synchronized; without an asinine amount of foolish programming of course.

If this is not what you want, then i suggest you create a simple representation of your code (or link to the actual code).

 

[Kene Meniru]

On 03/09/2013 10:34 AM, Kene Meniru wrote:

And does the user run this script by doing
python user.py

Yes


You presumably mean, "When Python runs the script user.py"

When the user types and enters at the command line: "python user.py"

What program is that? Looks to me like you're writing a series of
modules, a framework perhaps, that is invoked by the user script.

Yes. I am writing many functions and classes in many modules that work
together.

Don't you mean the code in user.py has to *call* the function app.view()
when they're all done with defining the components? That has nothing to
do with being the last line.

Well... yes. The function is app.view() but it is not called automatically
by the code in user.py. The following is an example of the contents of
user.py

################### begin user.py #############################
from buildes import *

site("Willow_Creek_Ct", 5)
"""Create a site with name and number of boundaries."""
level("level1", 3000)
level("level2", 3000)
"""Create levels with name and height."""
setLevel("level1")
"""Set the current level to place objects"""
space("Dining", 5)
"""Create a space with 5 sides (walls)"""
linearSide("Dining-S1", 3683, 152, 0)
"""Initialize the first side of space called Dining"""
offset("Dining-S1", (3000, 0, 0))
"""Install the first side of Dining space called Dining-S1"""
view(POV)
"""Exports objects to POV-Ray format"""
################### end user.py #############################

You mean writes files into the file system?

Yes.


At that point, the user.py script has completed, and Python is done,
right?

Yes


What changes are those? You can't change a script while it's executing.
Could you clarify this so I can rethink the following paragraph?

Working on the user.py is an iterative process. The user adds new objects or
changes the parameters of the objects already there and envokes "python
user.py" each time to see the changes made. Just like using a program like
LaTeX. You edit your text file and run LaTeX on the file to see the changes
you made. The process continues until you finish the document.

 

[Dave Angel]

When the user types and enters at the command line: "python user.py"


Yes. I am writing many functions and classes in many modules that work
together.


Well... yes. The function is app.view() but it is not called automatically
by the code in user.py. The following is an example of the contents of
user.py

################### begin user.py #############################
from buildes import *

site("Willow_Creek_Ct", 5)
"""Create a site with name and number of boundaries."""
level("level1", 3000)
level("level2", 3000)
"""Create levels with name and height."""
setLevel("level1")
"""Set the current level to place objects"""
space("Dining", 5)
"""Create a space with 5 sides (walls)"""
linearSide("Dining-S1", 3683, 152, 0)
"""Initialize the first side of space called Dining"""
offset("Dining-S1", (3000, 0, 0))
"""Install the first side of Dining space called Dining-S1"""
view(POV)
"""Exports objects to POV-Ray format"""
################### end user.py #############################


Working on the user.py is an iterative process. The user adds new objects or
changes the parameters of the objects already there and envokes "python
user.py" each time to see the changes made. Just like using a program like
LaTeX. You edit your text file and run LaTeX on the file to see the changes
you made. The process continues until you finish the document.

It would then have to be a separate executable. Are you really saying
you want this window to keep "running" after the script ends? And that
somehow it notices that the user has rerun the user.py script? Or
could it simply be a window created during the user.py run that stays
running till it's closed, which ends the script as well?

Is running the script considered intervention? Or do you mean literally
that somebody watches the script for changes (eg. by watching the
timestamp)?

Either the "graphic" is in the same process, in which case the script
can only be "parsed" once, or the "graphic" is in a second process, in
which case you're talking some interesting interprocess communication
(ipc). That's certainly do-able, but it's way beyond the scope of the
kind of programming you've been talking about. But sometimes just using
a regular file for transfer is adequate.

If you really want two processes, then you should consider having the
user run the the graphic app, with a commandline parameter of user.py,
and have it create the user.py process. The user.py process runs till
it has created all the data, then sends it via some ipc to the graphic
app. Once sent, it terminates. The graphic app reads the ipc stuff,
updates its graphics, then idles, watching for timestamp changes on the
user.py file.

If you choose to use external file(s) to communicate between the two
processes, you might use pickle or shelve.

 

[Kene Meniru]

============================================================
Interactive Input
============================================================

Create an interactive environment where the user can enter commands
directly into the namespace using your API in real time, then when he is
ready to see the result of those commands, he can call "display command"
or push a "display button" and the App will run the appropriate outside
programs to show the visualization.

============================================================
Scripts loaded at runtime
============================================================

If you prefer the user to write "scripts", and then have your App read
these scripts (and create visualizations from the commands within the
scripts) then the only difference between step 1 and step 2 is *when* the
commands are interpreted. In this case your app will *load* the scripts
AFTER they are completely written (this could be done automatically by the
app at runtime IF the path to these scripts is known, or could be
accomplished by the user picking files in a dialog (Menu->RunScript).

Please see my last response to Dave Angel. I think it is possible for a
program to watch a file. I am not interested in menus which is why I am
going this route. I could easily use PyQt to make this but I am not
interested in graphical user interfaces. Are you are familiar with LaTeX?
That is the system I very much want to emulate.

I think you are trying to create step 1 without the interactive
environment. This is your mistake. Having users work with "raw files" and
then *somehow* watching a "raw file" for certain "display commands" (in
real time) is folly.

It is possible to watch files for changes without user intervention. If a
change is detected (whether by file size or time) the file can be executed
as all python modules can be using "python FILENAME.py". How is this folly?

Heck, when you edit a file in a text editor you are only seeing a
representation of the file from the last time it was saved, and your app
could never know when the "file data" and the "file view" where
synchronized; without an asinine amount of foolish programming of course.

If this is not what you want, then i suggest you create a simple
representation of your code (or link to the actual code).

My code is available at http://sourceforge.net/projects/buildes. The
documentation is at http://buildes.sourceforge.net/.

 

[Kene Meniru]

It would then have to be a separate executable.

This is my thinking too.

Are you really saying
you want this window to keep "running" after the script ends? And that
somehow it notices that the user has rerun the user.py script? Or
could it simply be a window created during the user.py run that stays
running till it's closed, which ends the script as well?

Following the idea that it should be a separate executable, the user does
not have to run the script by doing "python user.py" as they would normally
do. The window takes this over. It sits there watching for changes to
user.py such as change in file size and/or time or if the user hits the
enter key while it has focus.

My question really is how can the graphics window have access to the objects
saved in the dictionary of the doc object after running "python user.py". It
need to do this to be able to use their coordinates to redraw the view.

Is running the script considered intervention? Or do you mean literally
that somebody watches the script for changes (eg. by watching the
timestamp)?

I do not understand. What I mean is that this happens automatically. The
user no longer needs to do "python user.py" the graphics window now should
do this for the user when changes are detected in user.py by watching
timestamp for example.

If you really want two processes, then you should consider having the
user run the the graphic app, with a commandline parameter of user.py,
and have it create the user.py process. The user.py process runs till
it has created all the data, then sends it via some ipc to the graphic
app. Once sent, it terminates. The graphic app reads the ipc stuff,
updates its graphics, then idles, watching for timestamp changes on the
user.py file.

This sounds interesting. What is ipc. Can you give me an example?

 

[Kene Meniru]

Dave Angel wrote:

This sounds interesting. What is ipc. Can you give me an example?

Actually there is a possible simple solution consistent with the way my
program works already.

I can just provide another exporter for OpenGL that pyglet.window.Window()
subclass will be able to read. So I will provide another parameter for OGL
so the user can use view(OGL). When the user.py script is run, all objects
in doc are converted and exported to a file that the graphics window is
watching. The contents will then be used to update the view. So this means
that the user or the graphics window object can run "python user.py" it no
longer matters how.

Thanks for your comments.

 

[Rick Johnson]

Please see my last response to Dave Angel. I think it is
possible for a program to watch a file. I am not
interested in menus which is why I am going this route. I
could easily use PyQt to make this but I am not interested
in graphical user interfaces.

Yes, using a non-GUI interface is always much simpler to code than a GUI. Ican see the merit in that approach.

It is possible to watch files for changes without user
intervention. If a change is detected (whether by file
size or time)

Indeed, but the file data on disc and the file data in the text editor cannot be guaranteed to be exact representations of each other at any random sample of time.

the file can be executed as all python
modules can be using "python FILENAME.py".

I understand that Python files can be executed (as we all do).

How is this folly?

Well i don't have enough information *yet* to decide if your program designis "folly", but i can assure you that your explanations of how this program will function are. My problem with your workflow is that it ignores the need to constantly save the edited file data back to disc, because i "assume" you are using a text editor to edit the ".py" files. Sure, you could create a behavior of the editor to save the changes after every keystroke, or every newline, or whatever; but you failed to mentioned how any of this willwork.

My code is available at
http://sourceforge.net/projects/buildes. The documentation
is at http://buildes.sourceforge.net/.

Well GEE-GOLLY-GARSH, thanks for finally posting something we can draw coherency from. May i suggest that next time you post the links in the very first post? I mean, i like playing "guess the number" as much as the next guy,but geez!

 

[Kene Meniru]

Here's the answer to this question.

The summary of the question: how to run a module (called myapp.py) from
another module (called myappwin.py) and be able to access the namespace of
myapp.py from myappwin.py.

------------------------------------------
# contents of myapp.py
import math

class MyApp(object):
def __init__(self):
super(MyApp, self).__init__()
self.name = "MyAppName"


def testFunction():
boke = "Smilling"
print math.sin(1), boke
-----------------------------------------
# contents of myappwin
def test():
dic = {}
execfile("myapp.py", dic)
testObj = dic["MyApp"]() # access MyApp class
dic["testFunction"]() # execute testFunction
print testObj.name # print string


test()

 

[Dave Angel]

Here's the answer to this question.

The summary of the question: how to run a module (called myapp.py) from
another module (called myappwin.py) and be able to access the namespace of
myapp.py from myappwin.py.

------------------------------------------
# contents of myapp.py
import math

class MyApp(object):
def __init__(self):
super(MyApp, self).__init__()
self.name = "MyAppName"


def testFunction():
boke = "Smilling"
print math.sin(1), boke
-----------------------------------------
# contents of myappwin
def test():
dic = {}
execfile("myapp.py", dic)
testObj = dic["MyApp"]() # access MyApp class
dic["testFunction"]() # execute testFunction
print testObj.name # print string


test()
-----------------------------------------
# OUTPUT
$ python myappwin.py
0.841470984808 Smilling
MyAppName

I hope you're just kidding. execfile() and exec() are two of the most
dangerous mechanisms around. import or __import__() would be much
better, as long as your user hasn't already run myapp.py as his script.

 

[Kene Meniru]

On 03/11/2013 07:57 PM, Kene Meniru wrote:

I hope you're just kidding. execfile() and exec() are two of the most
dangerous mechanisms around. import or __import__() would be much
better, as long as your user hasn't already run myapp.py as his script.

It does what I want. Security is another issue and I understand but can't
help it until I find another solution.

import does not do what I want.

How does __import__() work and what do you mean "as long as your user hasn't
already run myapp.py as his script."?

 

[Kene Meniru]

On 03/11/2013 07:57 PM, Kene Meniru wrote:

I hope you're just kidding. execfile() and exec() are two of the most
dangerous mechanisms around. import or __import__() would be much
better, as long as your user hasn't already run myapp.py as his script.

Tried __import__ and it seems to execute the myapp.py just like execfile
however it only provides access to objects defined in the module myapp.py
only. Like I mentioned, my program has multiple packages and the objects
myappwin needs access to are stored in an object in another module called
doc.py.

- myapp.py provides the functions used to describe 3D objects
- app.py is imported into myapp.py and is called by the functions after they
create the 3D objects.
- app.py uses another module called doc.py which app.py imports to save the
objects in a class with a dictionary.
- myappwin needs to access the dictionary in the class inside the doc
module.

 

[Dave Angel]

It does what I want. Security is another issue and I understand but can't
help it until I find another solution.

import does not do what I want.

How does __import__() work and what do you mean "as long as your user hasn't
already run myapp.py as his script."?

The __import__() function is defined
http://docs.python.org/2/library/functions.html#__import__


appname = "myapp"
usermodule = __import__(appname, globals(), locals(), [], -1)

And now you can use usermodule as though you had imported it in the
usual way.

As for my other caveat, I've said it before in this thread. Make sure
you don't ever load a module by more than one name, or you'll end up
with a mess. And that includes the original script, which is loaded by
the name '__main__'

You also should avoid any circular import, as it can be very tricky to
deal with them.

As I said earlier in the thread: