Python Embedding Importing relative modules

[moerchendiser2k3]

Hi all,

I have a serious problem I haven't solved yet, hope one of you can
help me. The first thing is, I embedded Python into my app and I
execute several scripts in this environment.

The problem is, the scripts don't import modules from their relative
path. I guess this is related to the sys.path ['',...] and the current
working directory which is set to the directory of my host
application.

I could set that path manually, but all the scripts might be stored in
different locations. So now I try to find a way to handle that. Any
suggestions?

A solution would be, that each script, appends its own directory to
the system path, but this might lead to problems. Imagine all of them
have a module called 'foo.py' and its not the same. This might lead to
name conflicts, wouldnt it?

Btw, I found a source code line in the documentation, where I should
really get rid of the ['', ...] path in the system path due to
security reasons.

import sys; sys.path.pop(0)


Hope one of you can help me out here. Really thanks!!

Bye,

moerchendiser2k3

 

[Aahz]

I have a serious problem I haven't solved yet, hope one of you can
help me. The first thing is, I embedded Python into my app and I
execute several scripts in this environment.

The problem is, the scripts don't import modules from their relative
path. I guess this is related to the sys.path ['',...] and the current
working directory which is set to the directory of my host
application.

I could set that path manually, but all the scripts might be stored in
different locations. So now I try to find a way to handle that. Any
suggestions?

Set sys.path to include each script's base dir before running it, then
restore after each script.

 

[moerchendiser2k3]

Set sys.path to include each script's base dir before running it, then

restore after each script.

That works, but doesnt solve the problem.

ScriptA.py has a module in its directory called 'bar.py'
ScriptB.py has a module in its directory called 'bar.py'

Imagine the 'bar.py' modules dont have the same content, so
they are not equal.

Now when the first bar.py is imported, the second import for
a "import bar" imports the first one, because its already
stored in sys.modules.

 

[Aahz]

Aahz:

That works, but doesnt solve the problem.

ScriptA.py has a module in its directory called 'bar.py'
ScriptB.py has a module in its directory called 'bar.py'

Imagine the 'bar.py' modules dont have the same content, so they are
not equal.

Now when the first bar.py is imported, the second import for a "import
bar" imports the first one, because its already stored in sys.modules.

Good point, you'll need to save/restore sys.modules, too. That gets you
90-95% of complete namespace separation; if you need more than that, your
best bet is to use separate processes. Full-blown namepace isolation is
a *hard* problem, just look at all the past attempts to create secure
Python (and what you're trying to do is roughly equivalent).

 

[moerchendiser2k3]

Good idea. Just one thing I thought about:

Imagine I load them parallel so the GIL might
interrupt the save-process of sys.modules,

[ENSURE GIL]
Load Script
Save sys.modules
[interrupt]
Load Script
Save sys.modules
....

so this might run into several other problems.

But maybe I change that so I load the scripts in a row.
Thanks for your shoulder I can cry on

Bye,

moerchendiser2k3

 

[Aahz]

Imagine I load them parallel so the GIL might interrupt the
save-process of sys.modules,

[ENSURE GIL]
Load Script
Save sys.modules
[interrupt]
Load Script
Save sys.modules
...

so this might run into several other problems.

Very yes; if you're trying to maintain script independence, you should
either run them sequentially (so you can safely save/restore the
environment) or use processes.

 

[Thomas Jollans]

Good point, you'll need to save/restore sys.modules, too. That gets you
90-95% of complete namespace separation; if you need more than that, your
best bet is to use separate processes. Full-blown namepace isolation is
a *hard* problem, just look at all the past attempts to create secure
Python (and what you're trying to do is roughly equivalent).

I believe Python (at least in the 3.x series) supports multiple
interpreter instances per process.

 

[Aahz]

I believe Python (at least in the 3.x series) supports multiple
interpreter instances per process.

Perhaps things have changed in 3.x, but although 2.x supposedly supported
multiple interpreter instances per process, the impression I got was that
most people would rather poke their eyes out with a dull stick than try
to make multiple interpreters per process actually work.

Multiple processes are easy, OTOH.