shipping python

[Kristen J. Webb]

I am new to python coming from the C/shell side.
We have been experimenting with some code
samples and now I'm looking at some command line
argument processing. I find

getopt older
optparse new in 2.3
argparse new in 2.7

I search around on some of my client systems and
find lots of people in the 2.4 - 2.6 range.

After some more digging I see that I can
easy_install argparse on my development system.

My question is will I be able to ship this
to a customer? Can I create .pyc files so
that the customer does not have to install the argparse
module?

If not, and I want to go back into the 2.3+ range,
should I just use optparse?

I guess what I am asking here is are there any
guidelines/recommendations for shipping python
programs to customers?

Thanks in advance,
Kris
--
Mr. Kristen J. Webb
Teradactyl LLC.

PHONE: 1-505-242-1091
EMAIL: (e-mail address removed)
VISIT: http://www.teradactyl.com

Home of the

True incremental Backup System

 

[Steven D'Aprano]

After some more digging I see that I can easy_install argparse on my
development system.

My question is will I be able to ship this to a customer? Can I create
.pyc files so that the customer does not have to install the argparse
module?

Yes, and yes. The licence of argparse is a very liberal licence, so you
can just include it in your application. There's no need for the user to
install it separately.

You could include just the .pyc file if you insist, but I personally
don't like or recommend .pyc only installations.

If not, and I want to go back into the 2.3+ range, should I just use
optparse?

That depends on how complex your command line arguments are.

 

[Kristen J. Webb]

I tried experimenting with .pyc files only to end up at:

RuntimeError: Bad magic number in .pyc file

can't run 2.5 pyc files on 2.6

My main motivation to use .pyc is to keep end users from changing scripts,
breaking things, and then calling us. Tamper proofing may be an
alternative approach if anyone has suggestions.

Otherwise, trying to summarize my goals:

1. Distribute python (and other stuff) across many platforms
(Linux, UNIXes, OSX, Windows)

2. Support the current 2.X version of python installed on the system

3. Take advantage of python modules that use shared OS libraries
(e.g. libssl.so.whatever)

4. For linux especially, support many distributions/versions
(getting openssl to work right, for example, is a tricky problem)

5. Take advantage of the latest python modules whenever possible
(hence the argparse testing). I realize that I may be able
to go back so far for this to make sense...

6. Provide additional product functionality through our own
shared libraries (c code).

I'm assuming that the customer system has python 2.X installed, otherwise they
can use their favorite OS method to get it in the standard way.

Looking at methods that try to package and run without python installed at
all do not look appealing, anywhere from does it work, to export issues,
to needing a C compiler on the client system.....

Shipping .py files and support for modules like argparse for
backward compatibility is what I've come up with so far.
Then they have everything I need to create and install a .pyc file.
(The can modify the installer .py files, but I expect this to be
less of an issue and may be able to make my own simple tamper detecting
for this).

Can anyone suggest better strategies, or point out where this just won't work?

Is distutils a good starting point?

Many thanks!

Kris

Yes, and yes. The licence of argparse is a very liberal licence, so you
can just include it in your application. There's no need for the user to
install it separately.

You could include just the .pyc file if you insist, but I personally
don't like or recommend .pyc only installations.



That depends on how complex your command line arguments are.

--
Mr. Kristen J. Webb
Teradactyl LLC.

PHONE: 1-505-242-1091
EMAIL: (e-mail address removed)
VISIT: http://www.teradactyl.com

Home of the

True incremental Backup System

 

[Chris Angelico]

My main motivation to use .pyc is to keep end users from changing scripts,
breaking things, and then calling us.  Tamper proofing may be an
alternative approach if anyone has suggestions.

I wouldn't bother; if you're worried about that, try a tamper-evident
system rather than tamper-proof - for instance, MD5 checksums of
everything:

$ md5sum * >checksum.txt
$ md5sum checksum.txt
123412341234123412341234 md5sum

Save that meta-checksum and have an easy way to read it back - that
way, you can be sure they haven't just rebuilt the checksum file. If
that one checksum has changed, you know something else has; to find
out which:

$ md5sum -c --quiet checksum.txt

No need to restrict what you send out that way, and you can still get
a guarantee that they haven't fiddled with anything.

ChrisA

 

[Andrea Crotti]

I tried experimenting with .pyc files only to end up at:

RuntimeError: Bad magic number in .pyc file

can't run 2.5 pyc files on 2.6

My main motivation to use .pyc is to keep end users from changing
scripts,
breaking things, and then calling us. Tamper proofing may be an
alternative approach if anyone has suggestions.

Otherwise, trying to summarize my goals:

1. Distribute python (and other stuff) across many platforms
(Linux, UNIXes, OSX, Windows)

2. Support the current 2.X version of python installed on the system

3. Take advantage of python modules that use shared OS libraries
(e.g. libssl.so.whatever)

4. For linux especially, support many distributions/versions
(getting openssl to work right, for example, is a tricky problem)

5. Take advantage of the latest python modules whenever possible
(hence the argparse testing). I realize that I may be able
to go back so far for this to make sense...

6. Provide additional product functionality through our own
shared libraries (c code).

I'm assuming that the customer system has python 2.X installed,
otherwise they
can use their favorite OS method to get it in the standard way.

Looking at methods that try to package and run without python
installed at
all do not look appealing, anywhere from does it work, to export issues,
to needing a C compiler on the client system.....

Shipping .py files and support for modules like argparse for
backward compatibility is what I've come up with so far.
Then they have everything I need to create and install a .pyc file.
(The can modify the installer .py files, but I expect this to be
less of an issue and may be able to make my own simple tamper detecting
for this).

Can anyone suggest better strategies, or point out where this just
won't work?

Is distutils a good starting point?

Many thanks!

Kris

Why do you want to use the system python and system libraries?
If you want to avoid too much troubles and compatibility hell maybe
give PyInstaller a try, you ship just one big executable and you're done.
But apart from that setuptools is very good to know.

And also the world is a funny place, but does it really happen that
a customer changes the code in your python scripts, and the *complain*
if it doens't work anymore??

 

[Roy Smith]

And also the world is a funny place, but does it really happen that
a customer changes the code in your python scripts, and the *complain*
if it doens't work anymore??

You sound like somebody who has never had to support paying customers

 

[Kristen J. Webb]

Why do you want to use the system python and system libraries?

It is a standard, just like glibc on linux, I'd like my code should work
on the OS with minimal intrusion.

If you want to avoid too much troubles and compatibility hell maybe
give PyInstaller a try, you ship just one big executable and you're done.
But apart from that setuptools is very good to know.

What about Solaris, NetBSD, FreeBSD, etc, etc.
If the executable includes bits from the openssl library, what about export
restrictions?
I'll take a look at setuptools, thanks.

And also the world is a funny place, but does it really happen that
a customer changes the code in your python scripts, and the *complain*
if it doens't work anymore??

Yes, support costs with very knowledgeable clients is a very important factor.

Thank you for the feedback!
K
--
Mr. Kristen J. Webb
Teradactyl LLC.

PHONE: 1-505-242-1091
EMAIL: (e-mail address removed)
VISIT: http://www.teradactyl.com

Home of the

True incremental Backup System

 

[Kristen J. Webb]

I wouldn't bother; if you're worried about that, try a tamper-evident
system rather than tamper-proof - for instance, MD5 checksums of
everything:

$ md5sum *>checksum.txt
$ md5sum checksum.txt
123412341234123412341234 md5sum

Save that meta-checksum and have an easy way to read it back - that
way, you can be sure they haven't just rebuilt the checksum file. If
that one checksum has changed, you know something else has; to find
out which:

$ md5sum -c --quiet checksum.txt

No need to restrict what you send out that way, and you can still get
a guarantee that they haven't fiddled with anything.

ChrisA

So you could think of the meta-checksum as a script wide version control
of what is installed. Then, at key entry points, take the hit
and re-generate the meta-checksum and compare. If bad, you
have a major version control error, do not process further.

Makes sense, so long as the user does not bother to regenerate
the meta-checksum as well.

Would be more extensible if in debugging, users can disable
this feature. or have a tool that updates the meta-checksum
so that they can run one-off code to fix problems. Oddly,
this feature would make it easier to change code and still
keep things running. But we could use the value in the support
chain to say, "something changed, what's going on?".

Thoughts?
K
--
Mr. Kristen J. Webb
Teradactyl LLC.

PHONE: 1-505-242-1091
EMAIL: (e-mail address removed)
VISIT: http://www.teradactyl.com

Home of the

True incremental Backup System