How can I package a python script and modules into a single script?

Discussion in 'Python' started by Noah, Nov 3, 2005.

  1. Noah

    Noah Guest

    I would like to package my main script and all the
    modules it imports into a single script that will
    run just as the collection would. It should not
    need to package standard Python lib modules -- just
    my local modules. This is not the same question
    that would be answered with py2exe or py2app. I
    don't need to package the script into a binary
    along with Python. I expect Python to be installed.
    I also don't want to use distutils to install the
    script. The script will always run on Unix.

    I thought that there might be some way to run a
    script package from a tar file. One way would be to
    have the package script untar itself into a temp
    directory; run the main script; then delete the
    temporary package directory when done. That sounds
    clunky and prone to leave around trash if someone
    does a 'kill -9'. However, this might be an
    acceptable compromise. I'm sure that I've seen a
    tool around like this, but I can't find it anymore.
    I checked the usual places (google and sf.net).

    I also considered simply cutting and pasting all
    the modules I need into one single, giant script,
    but that's less appealing than the tarball idea
    (unless it could be done automatically).

    Yours,
    Noah
    Noah, Nov 3, 2005
    #1
    1. Advertising

  2. Re: How can I package a python script and modules into a singlescript?

    "Noah" <> writes:

    > I would like to package my main script and all the
    > modules it imports into a single script that will
    > run just as the collection would. It should not
    > need to package standard Python lib modules -- just
    > my local modules. This is not the same question
    > that would be answered with py2exe or py2app. I
    > don't need to package the script into a binary
    > along with Python. I expect Python to be installed.
    > I also don't want to use distutils to install the
    > script. The script will always run on Unix.
    >
    > I thought that there might be some way to run a
    > script package from a tar file. One way would be to
    > have the package script untar itself into a temp
    > directory; run the main script; then delete the
    > temporary package directory when done. That sounds
    > clunky and prone to leave around trash if someone
    > does a 'kill -9'. However, this might be an
    > acceptable compromise. I'm sure that I've seen a
    > tool around like this, but I can't find it anymore.


    Wasn't this named 'squeeze'?

    Thomas
    Thomas Heller, Nov 3, 2005
    #2
    1. Advertising

  3. Noah <> wrote:

    > I would like to package my main script and all the
    > modules it imports into a single script that will
    > run just as the collection would. It should not
    > need to package standard Python lib modules -- just
    > my local modules. This is not the same question
    > that would be answered with py2exe or py2app. I


    Make a zipfile with the .pyc of all the modules you want; put a small
    text header right before it to import the mainscript and run its main
    function; that's it, thanks to Python's ability to import from zipfiles.

    There's a recipe for that in the Python Cookbook (2nd edition).

    (I believe py2app doesn't need to package the standard library, either,
    since it comes with MacOSX 10.3 and later, but I haven't checked).


    Alex
    Alex Martelli, Nov 3, 2005
    #3
  4. Noah

    utabintarbo Guest

    utabintarbo, Nov 3, 2005
    #4
  5. Noah

    Noah Guest

    Freeze also packages the python interpreter into a binary.
    I need a cross platform solution that just packages the scripts.
    I expect the user to already have python installed.

    Yours,
    Noah
    Noah, Nov 4, 2005
    #5
  6. Noah

    Noah Guest

    Bingo! That is what I was thinking of.
    http://effbot.org/zone/squeeze.htm
    It turns out that it doesn't quite do what I want
    because the resulting script is tightly coupled
    to the version of Python used to build the package.
    It compiles the PYC byte-code into the package.
    It's neat and I may be able to use the ideas to do
    what I want.

    Yours,
    Noah
    Noah, Nov 4, 2005
    #6
  7. Would PythonEggs be close?

    It's basically a zip format with all the Python files inside, it would
    maintain the structure and not be a single script file, but would be a
    single .egg file... the end user would just run the script and all
    would be installed, essentially.

    http://peak.telecommunity.com/DevCenter/PythonEggs
    Brett Hoerner, Nov 4, 2005
    #7
  8. Noah

    Noah Guest

    This is interesting, but requires two separate files -- the ZIP file
    and the boot script.
    This is because zipimport can only import from file paths.
    It can't import from a string or an IOString (as far as I can tell).
    If only they would let you import from an already open file. Damn!

    Squeeze is 99% of what I want except for the fact that it precompiles
    the py files to bytecode
    which locks you to the version of python you compiled on. Python byte
    code is not guaranteed
    to be portable across different versions of Python. Squeeze relies on
    ihooks, which is
    a standard module, but it is undocumented.

    Anders Hammarquist has a Python Cookbook example that seems to do what
    I want.
    "Importing a Dynamically Generated Module". So far so good. I have to
    check it out a bit more.
    It relies on the "imp" standard library module.

    With a bit of inspiration from "Squeeze" and help from the "imp" module
    I might have what I want.

    Yours,
    Noah
    Noah, Nov 4, 2005
    #8
  9. Noah <> wrote:

    > This is interesting, but requires two separate files -- the ZIP file
    > and the boot script.


    No, it doesn't.

    > This is because zipimport can only import from file paths.


    It can import from a file, and the file (like all zipfiles) can have a
    prefix. That prefix is where you put the "boot" script.

    See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/215301
    (though I believe I did some minor enhancements, as well as adding
    useful discussion, in the 2nd edition of the O'Reilly printed version of
    the cookbook, Raedler's basic idea is presented here; you can get all
    the sources as they appear in the printed version as an archive, a
    zipfile I believe, on the O'Reilly site for the book).

    > Anders Hammarquist has a Python Cookbook example that seems to do what
    > I want.
    > "Importing a Dynamically Generated Module". So far so good. I have to
    > check it out a bit more.


    I know Anders (known to his friends as "Iko") quite well, worked at his
    side in Sweden in past years, and I selected his recipe for the first
    edition of the O'Reilly printed version of the Cookbook -- but it does
    not address the specific problem you desire. Raedler's recipe does.


    Alex
    Alex Martelli, Nov 4, 2005
    #9
  10. Noah

    Noah Guest

    Alex Martelli wrote:
    >
    > > This is because zipimport can only import from file paths.

    >
    > It can import from a file, and the file (like all zipfiles) can have a prefix
    > That prefix is where you put the "boot" script.
    > See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/215301


    Unfortunately, Raedler's boot script closes stdin, which is a fairly
    big limitation.
    The herefile in the shell script (END_OF_PYTHON_CODE) redirects stdin
    before
    python starts; stdin is closed at the end of the herefile. This
    disables raw_input()
    and anything else that reads sys.stdin. Enter the following at a shell
    prompt
    to demonstrate the problem:
    python - << HEREFILE
    > print raw_input("Enter something")
    > HEREFILE

    You will get the following error:
    Enter somethingTraceback (most recent call last):
    File "<stdin>", line 1, in ?
    EOFError: EOF when reading a line

    But all is not lost. The "zipheader.unix" script can be rewritten to
    use the '-c' option of python
    instead of stdin and a herefile. This also handles the case where
    Python may be even less
    than version 2.0. This works the same. Just "cat zipheader.unix
    main.zip > main" then run "main".

    ---- 8< ---- 8< ---- save as zipheader.unix ---- 8< ---- 8<
    --------------
    #!/bin/sh
    # This is a self-extracting executable.
    # Execute this like any normal executable.
    # You may need to "chmod a+x" this file.
    # This is a binary ZIP file with a Python loader header.
    #
    # Bourne shell loader:
    PYTHON=$(which python 2>/dev/null)
    if [ ! -x "$PYTHON" ] ; then
    echo "Python not found!"
    exit 1
    fi
    exec $PYTHON -c "
    # Python loader:
    import sys, os
    if int(sys.version[0])<2:
    print 'Python version 2.3 final or greater is required.'
    print 'Your version is', sys.version
    os._exit(1)
    major = sys.version_info[0]
    minor = sys.version_info[1]
    releaselevel = sys.version_info[3]
    if (major==2 and minor<3) or (major==2 and minor==3 and
    releaselevel!='final'):
    print 'Python version 2.3 final or greater is required.'
    print 'Your version is', sys.version
    os._exit(1)
    sys.path.insert(0, sys.argv[1])
    del sys.argv[0:1]
    print sys.argv[1]
    import main
    main.main()
    " $0 $@

    # Zip file:
    ---- end of zipheader.unix
    -----------------------------------------------

    Yours,
    Noah
    Noah, Nov 23, 2005
    #10
  11. Noah

    Fuzzyman Guest

    You could use my includer script.

    http://www.voidspace.org.uk/python/recipebook.shtml#includer

    It effectively adds an include direct to python scripts.

    ##include module.py
    from module import *

    You then run ``includer.py infilename outfilename``

    This replaces the ``##include ..`` with the source of the included
    module and *removes* the import statement.
    This makes it possible to maintain modules separately, but distribute
    as a single script.

    All the best,

    Fuzzyman
    http://www.voidspace.org.uk/python/index.shtml
    Fuzzyman, Nov 23, 2005
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Parvinder
    Replies:
    6
    Views:
    732
    Thomas G. Marshall
    Feb 27, 2005
  2. Dave
    Replies:
    2
    Views:
    455
  3. Giovanni Bajo
    Replies:
    3
    Views:
    331
    Trent Mick
    Sep 7, 2005
  4. ImpalerCore
    Replies:
    0
    Views:
    822
    ImpalerCore
    Mar 10, 2011
  5. ela
    Replies:
    12
    Views:
    332
    Uri Guttman
    Apr 6, 2009
Loading...

Share This Page