Python Embedding Importing relative modules

Discussion in 'Python' started by moerchendiser2k3, Jul 6, 2010.

  1. Hi all,

    I have a serious problem I haven't solved yet, hope one of you can
    help me. The first thing is, I embedded Python into my app and I
    execute several scripts in this environment.

    The problem is, the scripts don't import modules from their relative
    path. I guess this is related to the sys.path ['',...] and the current
    working directory which is set to the directory of my host
    application.

    I could set that path manually, but all the scripts might be stored in
    different locations. So now I try to find a way to handle that. Any
    suggestions?

    A solution would be, that each script, appends its own directory to
    the system path, but this might lead to problems. Imagine all of them
    have a module called 'foo.py' and its not the same. This might lead to
    name conflicts, wouldnt it?

    Btw, I found a source code line in the documentation, where I should
    really get rid of the ['', ...] path in the system path due to
    security reasons.

    import sys; sys.path.pop(0)


    Hope one of you can help me out here. Really thanks!!

    Bye,

    moerchendiser2k3
     
    moerchendiser2k3, Jul 6, 2010
    #1
    1. Advertising

  2. moerchendiser2k3

    Aahz Guest

    In article <>,
    moerchendiser2k3 <> wrote:
    >
    >I have a serious problem I haven't solved yet, hope one of you can
    >help me. The first thing is, I embedded Python into my app and I
    >execute several scripts in this environment.
    >
    >The problem is, the scripts don't import modules from their relative
    >path. I guess this is related to the sys.path ['',...] and the current
    >working directory which is set to the directory of my host
    >application.
    >
    >I could set that path manually, but all the scripts might be stored in
    >different locations. So now I try to find a way to handle that. Any
    >suggestions?


    Set sys.path to include each script's base dir before running it, then
    restore after each script.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "If you don't know what your program is supposed to do, you'd better not
    start writing it." --Dijkstra
     
    Aahz, Jul 6, 2010
    #2
    1. Advertising

  3. >Set sys.path to include each script's base dir before running it, then
    >restore after each script.


    That works, but doesnt solve the problem.

    ScriptA.py has a module in its directory called 'bar.py'
    ScriptB.py has a module in its directory called 'bar.py'

    Imagine the 'bar.py' modules dont have the same content, so
    they are not equal.

    Now when the first bar.py is imported, the second import for
    a "import bar" imports the first one, because its already
    stored in sys.modules.
     
    moerchendiser2k3, Jul 6, 2010
    #3
  4. moerchendiser2k3

    Aahz Guest

    In article <>,
    moerchendiser2k3 <> wrote:
    >Aahz:
    >>
    >>Set sys.path to include each script's base dir before running it, then
    >>restore after each script.

    >
    >That works, but doesnt solve the problem.
    >
    >ScriptA.py has a module in its directory called 'bar.py'
    >ScriptB.py has a module in its directory called 'bar.py'
    >
    >Imagine the 'bar.py' modules dont have the same content, so they are
    >not equal.
    >
    >Now when the first bar.py is imported, the second import for a "import
    >bar" imports the first one, because its already stored in sys.modules.


    Good point, you'll need to save/restore sys.modules, too. That gets you
    90-95% of complete namespace separation; if you need more than that, your
    best bet is to use separate processes. Full-blown namepace isolation is
    a *hard* problem, just look at all the past attempts to create secure
    Python (and what you're trying to do is roughly equivalent).
    --
    Aahz () <*> http://www.pythoncraft.com/

    "If you don't know what your program is supposed to do, you'd better not
    start writing it." --Dijkstra
     
    Aahz, Jul 6, 2010
    #4
  5. Good idea. Just one thing I thought about:

    Imagine I load them parallel so the GIL might
    interrupt the save-process of sys.modules,

    [ENSURE GIL]
    Load Script
    Save sys.modules
    [interrupt]
    Load Script
    Save sys.modules
    ....

    so this might run into several other problems.

    But maybe I change that so I load the scripts in a row.
    Thanks for your shoulder I can cry on ;)

    Bye,

    moerchendiser2k3
     
    moerchendiser2k3, Jul 6, 2010
    #5
  6. moerchendiser2k3

    Aahz Guest

    In article <>,
    moerchendiser2k3 <> wrote:
    >
    >Imagine I load them parallel so the GIL might interrupt the
    >save-process of sys.modules,
    >
    >[ENSURE GIL]
    >Load Script
    >Save sys.modules
    >[interrupt]
    >Load Script
    >Save sys.modules
    >...
    >
    >so this might run into several other problems.


    Very yes; if you're trying to maintain script independence, you should
    either run them sequentially (so you can safely save/restore the
    environment) or use processes.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "If you don't know what your program is supposed to do, you'd better not
    start writing it." --Dijkstra
     
    Aahz, Jul 6, 2010
    #6
  7. On 07/06/2010 09:11 PM, Aahz wrote:
    > In article <>,
    > moerchendiser2k3 <> wrote:
    >> Aahz:
    >>>
    >>> Set sys.path to include each script's base dir before running it, then
    >>> restore after each script.

    >>
    >> That works, but doesnt solve the problem.
    >>
    >> ScriptA.py has a module in its directory called 'bar.py'
    >> ScriptB.py has a module in its directory called 'bar.py'
    >>
    >> Imagine the 'bar.py' modules dont have the same content, so they are
    >> not equal.
    >>
    >> Now when the first bar.py is imported, the second import for a "import
    >> bar" imports the first one, because its already stored in sys.modules.

    >
    > Good point, you'll need to save/restore sys.modules, too. That gets you
    > 90-95% of complete namespace separation; if you need more than that, your
    > best bet is to use separate processes. Full-blown namepace isolation is
    > a *hard* problem, just look at all the past attempts to create secure
    > Python (and what you're trying to do is roughly equivalent).


    I believe Python (at least in the 3.x series) supports multiple
    interpreter instances per process.
     
    Thomas Jollans, Jul 6, 2010
    #7
  8. moerchendiser2k3

    Aahz Guest

    In article <>,
    Thomas Jollans <> wrote:
    >On 07/06/2010 09:11 PM, Aahz wrote:
    >> In article <>,
    >> moerchendiser2k3 <> wrote:
    >>> Aahz:
    >>>>
    >>>> Set sys.path to include each script's base dir before running it, then
    >>>> restore after each script.
    >>>
    >>> That works, but doesnt solve the problem.
    >>>
    >>> ScriptA.py has a module in its directory called 'bar.py'
    >>> ScriptB.py has a module in its directory called 'bar.py'
    >>>
    >>> Imagine the 'bar.py' modules dont have the same content, so they are
    >>> not equal.
    >>>
    >>> Now when the first bar.py is imported, the second import for a "import
    >>> bar" imports the first one, because its already stored in sys.modules.

    >>
    >> Good point, you'll need to save/restore sys.modules, too. That gets you
    >> 90-95% of complete namespace separation; if you need more than that, your
    >> best bet is to use separate processes. Full-blown namepace isolation is
    >> a *hard* problem, just look at all the past attempts to create secure
    >> Python (and what you're trying to do is roughly equivalent).

    >
    >I believe Python (at least in the 3.x series) supports multiple
    >interpreter instances per process.


    Perhaps things have changed in 3.x, but although 2.x supposedly supported
    multiple interpreter instances per process, the impression I got was that
    most people would rather poke their eyes out with a dull stick than try
    to make multiple interpreters per process actually work.

    Multiple processes are easy, OTOH.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "If you don't know what your program is supposed to do, you'd better not
    start writing it." --Dijkstra
     
    Aahz, Jul 6, 2010
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tobiah
    Replies:
    2
    Views:
    323
    Tobiah
    Sep 14, 2003
  2. Axel Diener
    Replies:
    0
    Views:
    358
    Axel Diener
    Feb 11, 2004
  3. stefan
    Replies:
    3
    Views:
    447
    stefan
    Dec 8, 2004
  4. John Millikin
    Replies:
    0
    Views:
    257
    John Millikin
    May 30, 2008
  5. Peter Peyman Puk

    Importing v reloading modules modules

    Peter Peyman Puk, Mar 19, 2010, in forum: Python
    Replies:
    0
    Views:
    314
    Peter Peyman Puk
    Mar 19, 2010
Loading...

Share This Page