py2exe 0.6.1 released

[Thomas Heller]

py2exe 0.6.1 released
=====================

py2exe is a Python distutils extension which converts python scripts
into executable windows programs, able to run without requiring a
python installation. Console and Windows (GUI) applications, windows
NT services, exe and dll COM servers are supported.

Changes in this release:

* py2exe can now bundle binary extensions and dlls into the
library-archive or the executable itself. This allows to
finally build real single-file executables.

The bundled dlls and pyds are loaded at runtime by some special
code that emulates the Windows LoadLibrary function - they are
never unpacked to the file system.

This part of the code is distributed under the MPL 1.1, so this
license is now pulled in by py2exe.

* By default py2exe now includes the codecs module and the
encodings package.

* Several other fixes.

Homepage:

<http://starship.python.net/crew/theller/py2exe>

Download from the usual location:

<http://sourceforge.net/project/showfiles.php?group_id=15583>

Enjoy,

Thomas

 

[Bengt Richter]

py2exe 0.6.1 released
=====================

py2exe is a Python distutils extension which converts python scripts
into executable windows programs, able to run without requiring a
python installation. Console and Windows (GUI) applications, windows
NT services, exe and dll COM servers are supported.

Changes in this release:

* py2exe can now bundle binary extensions and dlls into the
library-archive or the executable itself. This allows to
finally build real single-file executables.

The bundled dlls and pyds are loaded at runtime by some special
code that emulates the Windows LoadLibrary function - they are
never unpacked to the file system.

So py2exe is windows-only (like exe ;-) or is there a py2elf or py2coff or such?

This part of the code is distributed under the MPL 1.1, so this
license is now pulled in by py2exe. Mozilla?


* By default py2exe now includes the codecs module and the
encodings package.

* Several other fixes.

Homepage:

<http://starship.python.net/crew/theller/py2exe>

Download from the usual location:

<http://sourceforge.net/project/showfiles.php?group_id=15583>

Enjoy,

Thomas

I haven't tried this, but I am getting interested in cannibalizing some functionality ;-)

Regards,
Bengt Richter

 

[Bugs]

Changes in this release:

* py2exe can now bundle binary extensions and dlls into the
library-archive or the executable itself. This allows to
finally build real single-file executables.

The bundled dlls and pyds are loaded at runtime by some special
code that emulates the Windows LoadLibrary function - they are
never unpacked to the file system.

Wow, that is fantastic Thomas, congratulations!

So far, I've only seen where Thinstall has been able to accomplish this
and at STEEP licensing cost, not free and open-source like py2exe!

Can this technology be applied to other platforms?
Could this all be modified in such a way that other scripting languages
could take advantage of your bundling technology?

Thanks!

 

[Thomas Heller]

So py2exe is windows-only (like exe ;-) or is there a py2elf or
py2coff or such?

py2exe is windows only. But, there are also (in no particular order)
PyInstaller, cx_Freeze, py2app, exemaker, freeze.

Mozilla?

Has nothing to do with Mozilla directly, it is just that parts of the
code are licensed under the Mozilla Public License Version 1.1.

I'm not too happy with this, but I have other things to do than to
rewrite software relased under the MPL just to release it under a more
liberal license (although I have considered that, and I certainly would
not reject a contribution ;-).

Thomas

 

[Thomas Heller]

Wow, that is fantastic Thomas, congratulations!

So far, I've only seen where Thinstall has been able to accomplish
this and at STEEP licensing cost, not free and open-source like py2exe!

At least there is (or was) a shareware tool named PEBundle, which
promised to do something similar. Haven't used it myself, though.

Can this technology be applied to other platforms?

Hehe. No idea - I don't know enough about shared libraries.

Could this all be modified in such a way that other scripting
languages could take advantage of your bundling technology?

Maybe, but I don't use other scripting languages. But doesn't tcl have
a pretty advanced packer? And perl?

Thanks!

You're welcome,

Thomas

 

[Giovanni Bajo]

* py2exe can now bundle binary extensions and dlls into the
library-archive or the executable itself. This allows to
finally build real single-file executables.

The bundled dlls and pyds are loaded at runtime by some special
code that emulates the Windows LoadLibrary function - they are
never unpacked to the file system.

Cute!

I tried it using the wx singlefile example, but unfortunately the resulting
executable segfaults at startup (using Python 2.3.3 on Windows 2000, with
latest wxWindows). How can I debug it?

 

[Thomas Heller]

Cute!

I tried it using the wx singlefile example, but unfortunately the resulting
executable segfaults at startup (using Python 2.3.3 on Windows 2000, with
latest wxWindows).

Yes, I can reproduce that. I'm still using wxPython 2.4.2.4 for Python
2.3.5, and that combo works. I have done a few tests, and wxPython
2.5.1.5 also works, while 2.5.5.1 crashes.

How can I debug it?

I'll assume that's a serious question.
There is no simple answer. First, the py2exe'd app responds to a
PY2EXE_VERBOSE environment variable, if you set it to '1', the exe will
reports imports (just as PYTHONVERBOSE does for python scripts). Of
course you have to change the sample so that it is built as console app
to be able to see the messages.

Then, you can throw some additional prints into
lib\site-packages\zipextimporter.py, to see what it does.

I've done all this, and it seems it is crashing when trying to import
_gdi.pyd. Next would be to debug through _memimported.pyd, but I don't
have a debug build of wxPython.

That's all I can say now.

I'll ask on the wxPython mailing list if they have an idea.

Thomas

 

[svbrk]

First: Thank you Thomas for the good work with py2exe. The single-file
option is something I have been waiting for
Will it also be possible to build independent files (such as
my_app_data.zip) into the executable?

Yes, I can reproduce that. I'm still using wxPython 2.4.2.4 for Python
2.3.5, and that combo works. I have done a few tests, and wxPython
2.5.1.5 also works, while 2.5.5.1 crashes.

I have tried two more combinations for this example
(samples/singlefile/gui/test_wx.py), both giving segfault on WindowsXP
/ SP2:
1. Python 2.3.3 with wxPython 2.6.1.0 unicode
2. Python 2.4.1 with wxPython 2.6.1.0 unicode

However, the combinations
Python 2.4.1 with wxPython 2.4.5.1 ansi
Python 2.4.1 with wxPython 2.6.1.0 ansi
do not cause segfault, but shows a dialog box pointing to a log file,
the contents of which is:

Traceback (most recent call last):
File "test_wx.py", line 1, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wxPython\__init__.pyo", line 10, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wxPython\_wx.pyo", line 3, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wxPython\_core.pyo", line 15, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wx\__init__.pyo", line 42, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wx\_core.pyo", line 4, in ?
File "zipextimporter.pyo", line 89, in load_module
ImportError: MemoryLoadLibrary failed loading _core_.pyd

Maybe this could help localizing the problem?

Svein Brekke

 

[Thomas Heller]

First: Thank you Thomas for the good work with py2exe. The single-file
option is something I have been waiting for
Will it also be possible to build independent files (such as
my_app_data.zip) into the executable?

Depends on how you want to access them at runtime.

What you can do is to include them as win32 resources, see the advanced
example - it puts the winXP manifest file as resource into the exe.

test_wx = Target(
...
other_resources = [(resource_type, resource_id, resource_data)],
)

resource_type and resource_id must be integers, resource_data
must be the data itself as a string.

At runtime you can access them with a win32api.LoadResource() call.

I have tried two more combinations for this example
(samples/singlefile/gui/test_wx.py), both giving segfault on WindowsXP
/ SP2:
1. Python 2.3.3 with wxPython 2.6.1.0 unicode
2. Python 2.4.1 with wxPython 2.6.1.0 unicode

However, the combinations
Python 2.4.1 with wxPython 2.4.5.1 ansi

You probably mean 2.5.5.1 ?

Python 2.4.1 with wxPython 2.6.1.0 ansi
do not cause segfault, but shows a dialog box pointing to a log file,
the contents of which is:

Traceback (most recent call last):
File "test_wx.py", line 1, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wxPython\__init__.pyo", line 10, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wxPython\_wx.pyo", line 3, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wxPython\_core.pyo", line 15, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wx\__init__.pyo", line 42, in ?
File "zipextimporter.pyo", line 78, in load_module
File "wx\_core.pyo", line 4, in ?
File "zipextimporter.pyo", line 89, in load_module
ImportError: MemoryLoadLibrary failed loading _core_.pyd

Maybe this could help localizing the problem?

I hope. Thanks for these,

Thomas

 

[svbrk]

However, the combinations

You probably mean 2.5.5.1 ?

It was 2.5.4.1 (an arbitrary choice of an old version, just to check if
a recent wxPython change was the cause ....).

Svein

 

[cmkl]

Hi,

I didnt succeed to bundle vpython-3.2.3 with py2exe-0.6.1 - regardless
if its build as single file or not:

"This application has requested the Runtime to terminate it in an
unusual way" and so on...

This error message seems not generated by py2exe. At least I could not
find a reference to it in the sources of py2exe.

Regards

Carl

 

[Thomas Heller]

Hi,

I didnt succeed to bundle vpython-3.2.3 with py2exe-0.6.1 - regardless
if its build as single file or not:

"This application has requested the Runtime to terminate it in an
unusual way" and so on...

This error message seems not generated by py2exe. At least I could not
find a reference to it in the sources of py2exe.

Often this is caused by py2exe not including some needed pyds, together
with 'sloppy' programming in the extensions themselves.

Let me explain: Sometimes, extensions depend on other extensions. Since
they cannot directly import functions from each other, Python has an API
for that: PyCObject_Import(module_name, cobject_name). This returns an
opaque pointer, which often points to a table of functions. Or NULL, in
case there's an error.

Normally, this import will never fail (because all the extensions are
available), so often the return value is *not* checked. Calling one of
these functions when the import has failed will then crash the
application.

In other cases the extension programmers see no other way to report the
error except calling PyFatal_Error() when the import failed, which would
then give what you see.

For py2exe, it may help to use include more modules or the whole package
in question and try to build again.

Of course, it can also be that py2exe 0.6 has a bug that 0.5.4 did not
have, can you try that version?

Thomas

 

[Giovanni Bajo]

Yes, I can reproduce that. I'm still using wxPython 2.4.2.4 for
Python
2.3.5, and that combo works. I have done a few tests, and wxPython
2.5.1.5 also works, while 2.5.5.1 crashes.

Ah that's fine, then. I thought it was one of those "only in my computer" kind
of issue

I'll assume that's a serious question.

Of course it was, I'm not sure why you should doubt it. I was just trying to
being helpful to you, thinking that it could have been hard to reproduce.
Luckily, you can look into it yourself.

I've done all this, and it seems it is crashing when trying to import
_gdi.pyd. Next would be to debug through _memimported.pyd, but I
don't have a debug build of wxPython.

OK. Do you believe that _memimported.pyd can eventually converge to something
stable? Emulating LoadLibrary for all versions of Windows is not an easy task
after all. Wine might provide some insights.

 

[Bengt Richter]

Yes, I can reproduce that. I'm still using wxPython 2.4.2.4 for Python
2.3.5, and that combo works. I have done a few tests, and wxPython
2.5.1.5 also works, while 2.5.5.1 crashes.


I'll assume that's a serious question.
There is no simple answer. First, the py2exe'd app responds to a
PY2EXE_VERBOSE environment variable, if you set it to '1', the exe will
reports imports (just as PYTHONVERBOSE does for python scripts). Of
course you have to change the sample so that it is built as console app
to be able to see the messages.

If you have a place in the program where output should never happen
except when you would want a console window to see it in, you can
call AllocConsole [1] safely even in multiple such places, just before
the printing, and the first such call will create the console and hook
up stdout and stderr ready to print. Subsequent calls to AllocConsole
are effectively ignored, so all the output goes to the same console
no matter which code section executed first. IMO this should be
built into at least the windows wpython to trigger at the first
attempt at stdout or stderr output. There could be an option to
override that default and thus ignore stdout/stderr output, but I
think it would be a useful default. Plus it would tell people early
that they had usesless prints going in their wpython programs.

Then, you can throw some additional prints into
lib\site-packages\zipextimporter.py, to see what it does.

I've done all this, and it seems it is crashing when trying to import
_gdi.pyd. Next would be to debug through _memimported.pyd, but I don't
have a debug build of wxPython.

That's all I can say now.

I'll ask on the wxPython mailing list if they have an idea.

Thomas

[1]

The AllocConsole function allocates a new console for the
calling process.

BOOL AllocConsole(VOID)

Parameters

This function has no parameters.

Return Value

If the function succeeds, the return value is TRUE. If the
function fails, the return value is FALSE. To get extended
error information, call GetLastError.

Remarks

A process can only be associated with one console, so
AllocConsole fails if the calling process already has a
console. A process can use the FreeConsole function to
detach itself from its current console, and then it can call
AllocConsole to create a new console. If the calling process
creates a child process, the child inherits the new console.
AllocConsole also sets up standard input, standard output,
and standard error handles for the new console. The standard
input handle is a handle to the console's input buffer, and
the standard output and standard error handles are handles
to the console's screen buffer. To retrieve these handles,
use the GetStdHandle function.

This function is primarily used by graphics applications to
create a console window. Graphics applications are
initialized without a console. Console applications are
normally initialized with a console, unless they are created
as detached processes (by calling the CreateProcess function
with the DETACHED_PROCESS flag).

See Also

CreateProcess, FreeConsole, GetStdHandle

Regards,
Bengt Richter

 

[cmkl]

I removed conditional imports from visual and after that I works like a
charm. Now I've got a VPython application within a single 3 Mbyte
exe-file (Python-2.3).
That is really cool.

Thanks

Carl

 

[Thomas Heller]

Ah that's fine, then. I thought it was one of those "only in my computer" kind
of issue


Of course it was, I'm not sure why you should doubt it. I was just trying to
being helpful to you, thinking that it could have been hard to reproduce.
Luckily, you can look into it yourself.

I wasn't offended ;-). Debugging the bundled executables is difficult -
because the source file debug info is lost (or at least MSVC isn't able
to access it). So you end up steppiung through the disassembly.

OK. Do you believe that _memimported.pyd can eventually converge to something
stable? Emulating LoadLibrary for all versions of Windows is not an easy task
after all. Wine might provide some insights.

Currently there's no platform specific code in this emulation. But I
have to admit I don't use win98 any more.
I hope that _memimporter.pyd eventually becomes stable, the new 0.6.2
release contains some important fixes.

For the original problem: the code to load extensions contained in
packages was buggy. With 0.6.2 the wxPython singlefile sample at least
works wit wxPython 2.4.2.4 + python 2.3.5, and wxPython 2.6.1.0 + Python
2.4.1.

Thomas

 

[Thomas Heller]

If you have a place in the program where output should never happen
except when you would want a console window to see it in, you can
call AllocConsole [1] safely even in multiple such places, just before
the printing, and the first such call will create the console and hook
up stdout and stderr ready to print. Subsequent calls to AllocConsole
are effectively ignored, so all the output goes to the same console
no matter which code section executed first. IMO this should be
built into at least the windows wpython to trigger at the first
attempt at stdout or stderr output. There could be an option to
override that default and thus ignore stdout/stderr output, but I
think it would be a useful default. Plus it would tell people early
that they had usesless prints going in their wpython programs.

IMO that would be a nice addition to pythonw.exe, but I have no time to
care about this myself. For py2exe, I'm still unsure how debugging
output from a frozen gui program should be handled.

Thomas

 

[Chris Lambacher]

I am also getting this. In my case I think it is related to win32com.shell
since when building I get the following error:
error: C:\work\...\build\bdist.win32\winexe\collect-2.4\win32com.shell\shell.pyd: No such file or directory

.... is a place holder for a very long path.

The problem is fixed by creating the directory win32com.shell which is
probably the wrong solution and causing the Unexpectedly terminated error.
It would appear that the win32com.shell hack
(http://starship.python.net/crew/theller/moin.cgi/WinShell) does not work in
single file executable mode.

-Chris