Re: changing sys.path

Discussion in 'Python' started by Andrea Crotti, Feb 1, 2012.

  1. On 02/01/2012 05:13 PM, Eric Snow wrote:
    > On Wed, Feb 1, 2012 at 9:15 AM, Andrea Crotti<> wrote:
    >> So suppose I want to modify the sys.path on the fly before running some code
    >> which imports from one of the modules added.
    >>
    >> at run time I do
    >> sys.path.extend(paths_to_add)
    >>
    >> but it still doesn't work and I get an import error.

    > Make sure you are adding to sys.path the directories that your
    > packages/modules are in, and not the actual package directories.
    > During import Python looks for modules/packages _in_ each of the
    > directories on sys.path, but not _at_ those directories.


    Yes sure I do this..
    >
    >> If I take these paths and add them to site-packages/my_paths.pth
    >> everything works, but at run-time the paths which I actually see before
    >> importing are exactly the same.

    > You mean sys.path looks exactly the same in the two cases?
    >
    > -eric


    Yes they are exactly the same, because in that file I just write exactly
    the same list,
    but when modifying it at run-time it doesn't work, while if at the
    application start
    there is this file everything works correctly...

    That's what really puzzles me.. What could that be then?
     
    Andrea Crotti, Feb 1, 2012
    #1
    1. Advertising

  2. On Wed, 01 Feb 2012 17:47:22 +0000, Andrea Crotti wrote:

    > Yes they are exactly the same, because in that file I just write exactly
    > the same list,
    > but when modifying it at run-time it doesn't work, while if at the
    > application start
    > there is this file everything works correctly...
    >
    > That's what really puzzles me.. What could that be then?



    Are you using IDLE or WingIDE or some other IDE which may not be
    honouring sys.path? If so, that's a BAD bug in the IDE.

    Are you changing the working directory manually, by calling os.chdir? If
    so, that could be interfering with the import somehow. It shouldn't, but
    you never know...

    Are you adding absolute paths or relative paths?

    You say that you get an ImportError, but that covers a lot of things
    going wrong. Here's a story. Could it be correct? I can't tell because
    you haven't posted the traceback.

    When you set site-packages/my_paths.pth you get a sys path that looks
    like ['a', 'b', 'fe', 'fi', 'fo', 'fum']. You then call "import spam"
    which locates b/spam.py and everything works.

    But when you call sys.path.extend(['a', 'b']) you get a path that looks
    like ['fe', 'fi', 'fo', 'fum', 'a', 'b']. Calling "import spam" locates
    some left over junk file, fi/spam.py or fi/spam.pyc, which doesn't
    import, and you get an ImportError.


    --
    Steven
     
    Steven D'Aprano, Feb 2, 2012
    #2
    1. Advertising

  3. On 02/02/2012 12:51 AM, Steven D'Aprano wrote:
    > On Wed, 01 Feb 2012 17:47:22 +0000, Andrea Crotti wrote:
    >
    >> Yes they are exactly the same, because in that file I just write exactly
    >> the same list,
    >> but when modifying it at run-time it doesn't work, while if at the
    >> application start
    >> there is this file everything works correctly...
    >>
    >> That's what really puzzles me.. What could that be then?

    >
    > Are you using IDLE or WingIDE or some other IDE which may not be
    > honouring sys.path? If so, that's a BAD bug in the IDE.
    > Are you changing the working directory manually, by calling os.chdir? If
    > so, that could be interfering with the import somehow. It shouldn't, but
    > you never know...
    >
    > Are you adding absolute paths or relative paths?


    No, no and absolute paths..

    >
    > You say that you get an ImportError, but that covers a lot of things
    > going wrong. Here's a story. Could it be correct? I can't tell because
    > you haven't posted the traceback.
    >
    > When you set site-packages/my_paths.pth you get a sys path that looks
    > like ['a', 'b', 'fe', 'fi', 'fo', 'fum']. You then call "import spam"
    > which locates b/spam.py and everything works.
    >
    > But when you call sys.path.extend(['a', 'b']) you get a path that looks
    > like ['fe', 'fi', 'fo', 'fum', 'a', 'b']. Calling "import spam" locates
    > some left over junk file, fi/spam.py or fi/spam.pyc, which doesn't
    > import, and you get an ImportError.
    >
    >

    And no the problem is not that I already checked inspecting at run-time..
    This is the traceback and it might be related to the fact that it runs
    from the
    ..exe wrapper generated by setuptools:

    Traceback (most recent call last):
    File "c:\python25\scripts\dev_main-script.py", line 8, in <module>
    load_entry_point('psi.devsonly==0.1', 'console_scripts', 'dev_main')()
    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\bin\dev_main.py",
    line 152, in main
    Develer(ns).full_run()
    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\bin\dev_main.py",
    line 86, in full_run
    run(project_name, test_only=self.ns.test_only)
    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\environment.py",
    line 277, in run
    from psi.devsonly.run import Runner
    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\run.py", line 7, in
    <module>
    from psi.workbench.api import Workbench, set_new_dev_main
    ImportError: No module named workbench.api


    Another thing which might matter is that I'm launching Envisage
    applications, which
    heavily rely on the use of entry points, so I guess that if something is
    not in the path
    the entry point is not loaded automatically (but it can be forced I
    guess somehow).

    I solved in another way now, since I also need to keep a dev_main.pth in
    site-packages
    to make Eclipse happy, just respawning the same process on ImportError works
    already perfectly..
     
    Andrea Crotti, Feb 2, 2012
    #3
  4. Andrea Crotti

    jmfauth Guest

    On 2 fév, 11:03, Andrea Crotti <> wrote:
    > On 02/02/2012 12:51 AM, Steven D'Aprano wrote:
    >
    >
    >
    > > On Wed, 01 Feb 2012 17:47:22 +0000, Andrea Crotti wrote:

    >
    > >> Yes they are exactly the same, because in that file I just write exactly
    > >> the same list,
    > >> but when modifying it at run-time it doesn't work, while if at the
    > >> application start
    > >> there is this file everything works correctly...

    >
    > >> That's what really puzzles me.. What could that be then?

    >
    > > Are you using IDLE or WingIDE or some other IDE which may not be
    > > honouring sys.path? If so, that's a BAD bug in the IDE.
    > > Are you changing the working directory manually, by calling os.chdir? If
    > > so, that could be interfering with the import somehow. It shouldn't, but
    > > you never know...

    >
    > > Are you adding absolute paths or relative paths?

    >
    > No, no and absolute paths..
    >
    >
    >
    > > You say that you get an ImportError, but that covers a lot of things
    > > going wrong. Here's a story. Could it be correct? I can't tell because
    > > you haven't posted the traceback.

    >
    > > When you set site-packages/my_paths.pth you get a sys path that looks
    > > like ['a', 'b', 'fe', 'fi', 'fo', 'fum']. You then call "import spam"
    > > which locates b/spam.py and everything works.

    >
    > > But when you call sys.path.extend(['a', 'b']) you get a path that looks
    > > like ['fe', 'fi', 'fo', 'fum', 'a', 'b']. Calling "import spam" locates
    > > some left over junk file, fi/spam.py or fi/spam.pyc, which doesn't
    > > import, and you get an ImportError.

    >
    > And no the problem is not that I already checked inspecting at run-time..
    > This is the traceback and it might be related to the fact that it runs
    > from the
    > .exe wrapper generated by setuptools:
    >
    > Traceback (most recent call last):
    >    File "c:\python25\scripts\dev_main-script.py", line 8, in <module>
    >      load_entry_point('psi.devsonly==0.1', 'console_scripts', 'dev_main')()
    >    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\bin\dev_main.py",
    > line 152, in main
    >      Develer(ns).full_run()
    >    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\bin\dev_main.py",
    > line 86, in full_run
    >      run(project_name, test_only=self.ns.test_only)
    >    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\environment.py",
    > line 277, in run
    >      from psi.devsonly.run import Runner
    >    File "h:\git_projs\psi\psi.devsonly\psi\devsonly\run.py", line 7, in
    > <module>
    >      from psi.workbench.api import Workbench, set_new_dev_main
    > ImportError: No module named workbench.api
    >
    > Another thing which might matter is that I'm launching Envisage
    > applications, which
    > heavily rely on the use of entry points, so I guess that if something is
    > not in the path
    > the entry point is not loaded automatically (but it can be forced I
    > guess somehow).
    >
    > I solved in another way now, since I also need to keep a dev_main.pth in
    > site-packages
    > to make Eclipse happy, just respawning the same process on ImportError works
    > already perfectly..




    There is something strange here. I can not figure
    out how correct code will fail with the sys.path.
    It seems to me, the lib you are using is somehow not
    able to recognize its own structure ("his own sys.path").

    Idea. Are you sure you are modifying the sys.path at
    the right place, understand at the right time
    when Python processes?

    I'm using this sys.path tweaking at run time very often;
    eg to test or to run different versions of the same lib
    residing in different dirs, and this, in *any* dir and
    independently of *any* .pth file.

    jmf
     
    jmfauth, Feb 2, 2012
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jeff Wagner

    sys.path.append('/my/new/path')

    Jeff Wagner, Nov 24, 2003, in forum: Python
    Replies:
    3
    Views:
    562
    Patrick Useldinger
    Nov 24, 2003
  2. Replies:
    3
    Views:
    414
  3. wen
    Replies:
    10
    Views:
    1,014
    Peter Hansen
    Aug 19, 2005
  4. Replies:
    2
    Views:
    688
    velle
    Jan 5, 2006
  5. Andrea Crotti

    changing sys.path

    Andrea Crotti, Feb 1, 2012, in forum: Python
    Replies:
    13
    Views:
    683
    Andrea Crotti
    Feb 10, 2012
Loading...

Share This Page