how to get rid of pyc files ?

[Stef Mientki]

hello,

Moving my entire program section between windows and Ubuntu,
sometimes causes problems, due to the existence of pyc-files
(and probably because my program still has hard coded paths).

Now I want get rid of the pyc-files,
so I wrote a py-script to remoce all pyc-files,
but because it's run from the same program section,
a few pyc files are recreated.

Is there a way to prevent generating pyc-files ?
Or is there a way to redirect the generated pyc-files to a dedicated
location ?

btw, What commandline switches are available for python ?
(googling didn't give me any relevant hits )

thanks,
Stef Mientki

 

[John Machin]

hello,

Moving my entire program section between windows and Ubuntu,
sometimes causes problems, due to the existence of pyc-files

What problems? Like avoiding having to recompile your .py files makes
your app run too fast?

(and probably because my program still has hard coded paths).

Now I want get rid of the pyc-files,

Which does nothing for the hard-coded paths issue ...

 

[pythoncurious]

What problems? Like avoiding having to recompile your .py files makes
your app run too fast?

There are real problems with this. I'm having similar problems when
switching
between Solaris and Windows.
The code is in clearcase, which uses some sort of network file
system.
In our case, that means that I'll be accessing the same directories
from both
platforms, so .pyc-files from one platform will be visible when I run
the
code on the other platform.

The .pyc-file will contain a string pointing to the file is was built
from.
The path to that file will be different under different platforms, so
when the
string is used, there will be error printouts.
At least I think that's the problem, the error printouts contains
paths that
are only valid on the other platform.
I don't have access to those computers atm, so I can't show the exact
message.

The only solution I've seen is to make sure to clean up the .pyc files
each
time I switch platform.

 

[John Machin]

There are real problems with this. I'm having similar problems when
switching
between Solaris and Windows.
The code is in clearcase, which uses some sort of network file
system.
In our case, that means that I'll be accessing the same directories
from both
platforms, so .pyc-files from one platform will be visible when I run
the
code on the other platform.

"switching" scarcely seems to be the right description. You appear to
be running the same code from one repository simultaneously available
to two different platforms.

Try this: Instead of running your code straight from your repository,
set up a run-time "launch pad" in an appropriate place for each
platform. When there is a new release of your app, install it in each
launchpad.

The .pyc-file will contain a string pointing to the file is was built
from.
The path to that file will be different under different platforms, so
when the
string is used, there will be error printouts.
At least I think that's the problem, the error printouts contains
paths that
are only valid on the other platform.
I don't have access to those computers atm, so I can't show the exact
message.

The only solution I've seen is to make sure to clean up the .pyc files
each
time I switch platform.

I'm a bit lost here ... your clearcase repository uses a mysterious
"some sort of network file system" to which you have access
permissions which allow you to delete individual files??

 

[Dave Angel]

There are real problems with this. I'm having similar problems when
switching
between Solaris and Windows.
The code is in clearcase, which uses some sort of network file
system.
In our case, that means that I'll be accessing the same directories
from both
platforms, so .pyc-files from one platform will be visible when I run
the
code on the other platform.

The .pyc-file will contain a string pointing to the file is was built
from.
The path to that file will be different under different platforms, so
when the
string is used, there will be error printouts.
At least I think that's the problem, the error printouts contains
paths that
are only valid on the other platform.
I don't have access to those computers atm, so I can't show the exact
message.

The only solution I've seen is to make sure to clean up the .pyc files
each
time I switch platform.

Is Clearcase still around? I hope it works better than it did in 1992.

Somebody else has already pointed out that you can tell Python not to
create those files (during your development stages).

But if that won't work for some reason, perhaps you can do something
with symbolic links. I remember that RCS, for example, required that
the archives be located in a directory immediately below the one with
the sources. So in order to share those archives, you made the
subdirectory actually a link to a common network location.

Your case would seem to be the opposite. But I don't know enough about
the current state of either Solaris or Clearcase to know the best answer.

Perhaps Clearcase supports some form of "exclusion" parameter, wherein
you say not to do version control on files with certain patterns, like .pyc

 

[JKPeck]

Is Clearcase still around?  I hope it works better than it did in 1992.

Somebody else has already pointed out that you can tell Python not to
create those files (during your development stages).

But if that won't work for some reason, perhaps you can do something
with symbolic links.  I remember that RCS, for example, required that
the archives be located in a directory immediately below the one with
the sources.  So in order to share those archives, you made the
subdirectory actually a link to a common network location.

Your case would seem to be the opposite.  But I don't know enough about
the current state of either Solaris or Clearcase to know the best answer.

Perhaps Clearcase supports some form of "exclusion" parameter, wherein
you say not to do version control on files with certain patterns, like .pyc

ClearCase gives you tremendous control over what can be seen at any
point. Assuming that you are using dynamic views, the simplest way to
fix this is to use a different view for each platform. They would
both be able to see the py files (although you could control that with
the configspec)as checked in, but the pyc files would not be checked
in and would automatically be view private. So with two different
views, each platform would only see its own pyc files.

HTH,
Jon Peck

 

[Dennis Lee Bieber]

Your case would seem to be the opposite. But I don't know enough about
the current state of either Solaris or Clearcase to know the best answer.

Perhaps Clearcase supports some form of "exclusion" parameter, wherein
you say not to do version control on files with certain patterns, like .pyc

Clearcase repositories are mounted as something they call a
"multi-version file system"... The repository, therefore, appears as a
set of directories within a "view" (if one has not set a view, none of
the controlled files are visible; once a view is set, based upon the
rules [controlling what branch/version is to be seen] one has access to
pretty much anything). .pyc files would be created as view-private files
-- they appear in the repository directory to anyone using the very same
view, but are not themselves "controlled".

Where I work, we have a rather nasty makefile used to build (Ada
sources) so that Solaris and IRIX binaries are placed into separate
directories. The thing is -- both OS specific directories are visible
from either...
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Steven D'Aprano]

Is there a way to prevent generating pyc-files ?

Put the .py files in a read-only directory.

Or is there a way to
redirect the generated pyc-files to a dedicated location ?
No.


btw, What commandline switches are available for python ? (googling
didn't give me any relevant hits )

python --help


at the shell (not Python) prompt.

 

[pythoncurious]

Is Clearcase still around?  I hope it works better than it did in 1992.

I don't know how it worked back then, but if it's worse than today, I
don't know how they ever managed to get people to use it. I'm not a
fan and I don't have a choice about using it

Somebody else has already pointed out that you can tell Python not to
create thosefiles(during your development stages).

Yes, that's probably the best way to sort out the problems.
It would of course be nice if python could detect that these files
aren't usable and just regenerate them. I realise it might not be a
trivial feature to implement though.

Perhaps Clearcase supports some form of "exclusion" parameter, wherein
you say not to do version control onfileswith certain patterns, like .pyc

They're not actually version controlled, but "view private" as someone
described. This means I'll see them whenever I'm using the view they
were created in.

 

[pythoncurious]

"switching" scarcely seems to be the right description. You appear to
be running the same code from one repository simultaneously available
to two different platforms.

Try this: Instead of running your code straight from your repository,
set up a run-time "launch pad" in an appropriate place for each
platform. When there is a new release of your app, install it in each
launchpad.

For releases, that's a good enough idea. But while developing and
wanting to run units tests, it's sort of a big overhead. Especially
since I tend to run the unit test quite frequently.

I'm a bit lost here ... your clearcase repository uses a mysterious
"some sort of network file system" to which you have access
permissions which allow you to delete individualfiles??

Apparently I don't explain things well.
Others have explained it better in this thread or you can have a look
at http://en.wikipedia.org/wiki/Rational_ClearCase.

 

[John Machin]

For releases, that's a good enough idea. But while developing and
wanting to run units tests, it's sort of a big overhead. Especially
since I tend to run the unit test quite frequently.

Oh .. so there's no concept of copying only the changed files??

 

[Gabriel Genellina]

Yes, that's probably the best way to sort out the problems.
It would of course be nice if python could detect that these files
aren't usable and just regenerate them. I realise it might not be a
trivial feature to implement though.

..pyc files are platform independent; you *can* use the same .pyc files
both on Windows and Linux.
A .pyc/.pyo file contains two bits of info:
- a magic number that depends on the Python version that created it
- the last-modification-time of the corresponding .py source
When you import a module, both numbers are read from the .pyc file. If the
magic number matches the version of the running Python, and the
modification time matches the one of the .py file (or no .py file exists),
then the .pyc file is used. In any other case, the .py file is read,
compiled, and a .pyc file is written -- if possible.

So, if you ensure that both platforms "see" the same
last-modification-time, and you use the same Python version on both
platforms, then .pyc files may be shared. If not, each time you switch
platforms all .pyc files will be re-created (option -B avoids that, as
already noted).

I any case, I don't think leaving the wrong .pyc files should create any
problem (except some delay when the application starts, and unless you've
found a bug, of course)

 

[Gabriel Genellina]

[please keep the discussion on the list]

may be python need a parameter to regenerate .pyo/pyc explicit ,not
depending on magic number and modification time.
but i just wander if you simply just clear all .pyc than generate in
one system manually, can than program run without error in another
system?

Yes, you can always remove the .pyc files before starting, forcing a
recompile. Also, see the compileall module.

Provided that both systems use the same Python version, you can precompile
..pyc files on one and copy them to the other. Even more, some people
deploy only .pyc files.