Distributing applications

[Phillip Mills]

I've learned enough of the Python language to be mildly dangerous and
have used it in a few personal projects. All my development of
commercial (or production) products over the past dozen years have been
done with C++ or Java.

For a program I'm planning -- to begin during the summer -- having an
interpreter as part of the application would be very desirable to allow
sophisticated users to provide their own extensions. Java would be
do-able, but....

My problems are:
- I'd like the process of installing the application to be one step;
no "first download a Python interpreter then a GUI library" kind of
thing.
- I also need the core part of the application to be reasonably
protected. I'm not looking to defeat hackers, but something equivalent
to the way Java's class files stored in jars stay where they're supposed
to be and aren't immediately readable.

I've looked at various web sites for this topic, but most I've found are
just arguments for using the Python language. OK, I'll pretend I'm
convinced...now any comments or references on the mechanics of creating
a self-contained distribution?

 

[Toby Dickenson]

now any comments or references on the mechanics of creating
a self-contained distribution?

Run to http://starship.python.net/crew/theller/py2exe/

 

[Jaime Wyant]

I wanted a nice tightly-wrapped distribution of my own and really
didn't want to go the py2exe route. I may have used a sledgehammer to
kill a fly, but here is what I did...

1) Downloaded python source (2.3.4)
2) Modified source, so that sys.path would pull from a registry key
setup by my installer.
2) Compiled python
3) Compiled wxPython 2.5.3.1
4) Bundled my app along with the custom python installation using NSIS.

My `setup.exe' file was 6654281 bytes -- not bad. Considering its the
entire python distribution (I may have missed a couple of modules --
but i don't think so) and wxPython.

I chose not to use py2exe because it made updating my app a bit
messier. For instance, suppose I don't use the smtplib module in
version 1 of my software. Py2exe realizes that and doesn't 'package'
smtplib with my executable. Now, if I release version 1.1 which does
use smtplib, then I'd have to figure out how to get the updates out
without having the user redownload the entire application. Granted, I
didn't put much thought into an update mechanism, but it seemed to be
messy at the time.

If I have my own distribution, I can simply give the users an "update"
program that will query my webserver which will download the latest
version for them automagically. Because my distribution has all of
the modules already available, I don't have to worry about sending
them any missing modules. Simply download the latest version of my
app and it works with my custom rolled distribution.

But, this only works for Windows...

jw

 

[=?ISO-8859-1?Q?Andr=E9_S=F8reng?=]

I've learned enough of the Python language to be mildly dangerous and
have used it in a few personal projects. All my development of
commercial (or production) products over the past dozen years have been
done with C++ or Java.

For a program I'm planning -- to begin during the summer -- having an
interpreter as part of the application would be very desirable to allow
sophisticated users to provide their own extensions. Java would be
do-able, but....

My problems are:
- I'd like the process of installing the application to be one step;
no "first download a Python interpreter then a GUI library" kind of
thing.

For creating a self-contained installation/distribution under Windows,
use py2exe:

http://starship.python.net/crew/theller/py2exe/

If you want to create a nice Windows installer for your module(s):

http://www.python.org/doc/current/dist/postinstallation-script.html

- I also need the core part of the application to be reasonably
protected. I'm not looking to defeat hackers, but something equivalent
to the way Java's class files stored in jars stay where they're supposed
to be and aren't immediately readable.

Hmm, not sure about that one. You mean that those users who write
extensions should not be able to modify the core code you wrote? Are you
talking about a restricted execution environment for untrusted code?
I'd rather make it so to only accept code which is signed by a trusted
party or something like that.

 

[Serge Orlov]

I chose not to use py2exe because it made updating my app a bit
messier. For instance, suppose I don't use the smtplib module in
version 1 of my software. Py2exe realizes that and doesn't 'package'
smtplib with my executable. Now, if I release version 1.1 which does
use smtplib, then I'd have to figure out how to get the updates out
without having the user redownload the entire application. Granted,
I didn't put much thought into an update mechanism, but it seemed to
be messy at the time.

I don't follow you. How is that different compared to adding a module
to your application? Let's say version 1.1 of your software has
module1.py updated, module2.py added and it needs smtplib. You just
bundle three compiled files and unpack them let's say to
\program files\my software\module1.pyc
\program files\my software\module2.pyc
\program files\my software\python\smtplib.pyc

I don't see any mess.

If I have my own distribution, I can simply give the users an "update"
program that will query my webserver which will download the latest
version for them automagically. Because my distribution has all of
the modules already available, I don't have to worry about sending
them any missing modules. Simply download the latest version of my
app and it works with my custom rolled distribution.

But, this only works for Windows...

Why? As I understand "update" program was written by you, so what
prevents it from working on other platforms besides testing?

Serge.

 

[Phillip Mills]

First, thanks to all the people who have answered so far for the
suggestions.

Phillip Mills wrote:

My problems are:

[...]

http://starship.python.net/crew/theller/py2exe/

Apparently I had more problems than I mentioned. One of them being
that a Windows-only solution is only a partial solution.

Hmm, not sure about that one. You mean that those users who write
extensions should not be able to modify the core code you wrote?

Partly that and partly a file management thing. For most end users a
..jar is one thing to deal with; it's the most recent one or it's not;
it's present in the right location or it's not....

 

[Jaime Wyant]

I don't follow you. How is that different compared to adding a module
to your application? Let's say version 1.1 of your software has
module1.py updated, module2.py added and it needs smtplib. You just
bundle three compiled files and unpack them let's say to
\program files\my software\module1.pyc
\program files\my software\module2.pyc
\program files\my software\python\smtplib.pyc

I don't see any mess.

The way I *think* about it is this --> To update my app, I only have
to download a new version of it... Being written in python, the app
itself is pretty small....

Suppose the following are true:
1) when the user installs my custom built python distribution they
also get version 1.0 of the app.
2) Suppose it is installed in c:\myapp (for simplicity).
3) I've just released version 1.1

Because *all* of the modules are in my custom built distribution, I
don't have to actively scan for new modules between releases. So....
I can write a simple "updater" script that will:
1) download a new myapp.zip
2) remove the c:\myapp directory and its subdirs
3) unzip it to c:\myapp, overwriting what is there

After those three steps are complete, then my app is updated. Super
simple. All I have to do is "zip" up my application and post it to a
website somewhere...

IIRC, py2exe bundles things up in a .zip file. So in order to update
a py2exe app i'd have to ->
1) check for new module dependencies
2) get the newly imported modules AND my newly updated modules back to
the client
3) update the zip file with the new modules.

This becomes especially hairy when someone is updating from 1.0 to say
1.5. Then I have to keep track of all the deltas between 1.0/1.5. My
way is much simpler because I don't have to keep up with *anything*.
As long as I test my code against my custom built distribution, it
ought to JUST WORK.

I don't trust myself to keep up with anything .

However, if you have an idea on updating py2exe bundled apps, I'm all ears...

Why? As I understand "update" program was written by you, so what
prevents it from working on other platforms besides testing?

Yes, but the *update* program runs in the context of my distribution
which is strictly bundled for windows. Now, someone *could* build a
distributable version for Linux somehow (I guess) and then the update
techniques I mentioned would probably work there.

jw

 

[Mitch Amiano]

I'm fairly new to python myself (with about 16 years of various
languages); I found py2exe fairly straightforward to use.
The instructions are ok, but if you care to see it I took some notes and
threw them into an article on my company site.

<http://home.agilemarkup.com/index.php?option=content&task=view&id=64&Itemid=27>

Regards

- Mitch

 

[Serge Orlov]

First, thanks to all the people who have answered so far for the
suggestions.

Phillip Mills wrote:

My problems are:

[...]

http://starship.python.net/crew/theller/py2exe/

Apparently I had more problems than I mentioned. One of them
being that a Windows-only solution is only a partial solution.

There is also py2app for Mac. Still partial? Then follow Jaime's
way: build and bundle python with your application.

Partly that and partly a file management thing. For most end users a

.jar is one thing to deal with; it's the most recent one or it's not;

it's present in the right location or it's not....

Python byte code is like java byte code and python supports importing
from zip files like java. Since python comes with a liberal license
you can change the importing code to decrypt your modules with a
"secret" key. That will be much safer than java. Of course that won't
stop real hackers.

Serge.

 

[Serge Orlov]

This becomes especially hairy when someone is updating from 1.0 to
say 1.5. Then I have to keep track of all the deltas between
1.0/1.5. My way is much simpler because I don't have to keep up with
*anything*. As long as I test my code against my custom built
distribution, it ought to JUST WORK.

I don't trust myself to keep up with anything .

Now I follow you I agree that updating py2exe apps requires package
management utilities. I don't think they will be messy, it's just
more code to maintain compared to your way. You only need to track
one delta (1.0 -> 1.1 ... -> latest) and publish two files latest.exe
and update.zip

Serge.

 

[Thomas Heller]

Now I follow you I agree that updating py2exe apps requires package
management utilities. I don't think they will be messy, it's just
more code to maintain compared to your way. You only need to track
one delta (1.0 -> 1.1 ... -> latest) and publish two files latest.exe
and update.zip

Sometimes I think that CVS or SVN may be the simplest solution to update
applications. Install a client on the target computer, it may be
invisible to the user, copy the CVS or SVN directories, and provide a
script the does (also invisible) 'cvs up -r version_a_b'.

Thomas

 

[Jaime Wyant]

Sneaky! I like it. Now if there was only a subversion python module...

jw

 

[Tim Jarman]

Sneaky! I like it. Now if there was only a subversion python module...

jw

GIYF: http://pysvn.tigris.org/

 

[Robert Kern]

Sneaky! I like it. Now if there was only a subversion python module...

Google, and you shall find.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Stephen Thorne]

However, if you have an idea on updating py2exe bundled apps, I'm all ears...

I'm working on a little project that requires remote updating. What I
basically came up with is two nested applications.

program/program.exe runs, and checks program/realprogram/ for updates
from a server. if md5s mismatch, it pulls new versions. after pulling
new version, it runs the program/realprogram.exe

realprogram.exe can, at some point in the future, be written to update
program/program.exe's code.

This isn't exactly the way I ended up doing it, due to the application
(all on a LAN, not the internet) I'm actually pulling the 'real'
program every time and leaving it in a directory under
tempfile.gettempdir()

Its a bit messy, but it actually works on both windows and linux.
(cx_Freeze under linux)

Stephen.