Pyhton Interpreter Startup time

[Neil Benn]

Hello,

I'm looking at a small app which would need a very quick
startup time for the Python interpreter. It doesn't do much (copying
and caching of files, no GUI) but I need the Python interpreter to start
up very quickly (<1 second on a Windows box). Is there a way to have a
'stripped' down Python interpreter which can start up very quickly on a
windows box. Once thing I was thinking of was to use PyExe to make a
quick startup (does it compile down to C code, therefore not using the
Python interpreter at runtime?). Is this a possible solution?

I observe that the second time I start python it starts up quicker
but I'm assuming that this is dependent on the environment and can't be
relied upon (or something like that).

Thanks, in advance for your help.

Cheers,

Neil

--

Neil Benn
Senior Automation Engineer
Cenix BioScience
BioInnovations Zentrum
Tatzberg 47
D-01307
Dresden
Germany

Tel : +49 (0)351 4173 154
e-mail : (e-mail address removed)
Cenix Website : http://www.cenix-bioscience.com

 

[Peter Hansen]

I'm looking at a small app which would need a very quick
startup time for the Python interpreter. It doesn't do much (copying
and caching of files, no GUI) but I need the Python interpreter to start
up very quickly (<1 second on a Windows box).

What kind of machine do you have?

On mine, Python starts up in about 0.06 seconds...

This primitive test shows these results on a Windows XP machine
(it won't work with Windows 98 as it can't chain commands on the
command line like that, but you could but it in a batch file).

c:\>echo. | time & python -c "import time; print time.time()"
The current time is: 8:59:59.67
Enter the new time:
1092315599.73

This is a fairly fast machine (Athlon 2500+) but I really
doubt slower machines would take much longer than 1 second
unless they are *really* old.

-Peter

 

[Cousin Stanley]

Peter Hansen wrote ....

What kind of machine do you have?

On mine, Python starts up in about 0.06 seconds...

This primitive test shows these results on a Windows XP machine
(it won't work with Windows 98 as it can't chain commands on the
command line like that, but you could but it in a batch file).

c:\>echo. | time & python -c "import time; print time.time()"
The current time is: 8:59:59.67
Enter the new time:
1092315599.73

This is a fairly fast machine (Athlon 2500+) but I really
doubt slower machines would take much longer than 1 second
unless they are *really* old.

-Peter

This variation on Peter's timing
is from a 5-year-old 250 MHz Compaq
running Linux/Debian ...

sk@cpq1 : ~/c $ ./gtod & python -c "import time ;
print ' Python ....' , time.time()"
[1] 1677

Number of seconds ......... 1092323003
Number of microseconds .... 194433
Time zone ................. 420
Daylight savings time ..... 0

Python .... 1092323003.36

 

[Christopher T King]

Is there a way to have a 'stripped' down Python interpreter which can
start up very quickly on a windows box.

Two things you can do are:

1) Run Python as "python -S". Normally, Python does an 'import site'
before running other code. Starting with 2.3, this does lots of expensive
localization goo, etc.; my guess is your script doesn't need it to execute
correctly, if it doesn't use any i18n functions.

2) Pre-compile the script. Normally, Python generates .pyc files for
imported modules so it doesn't have to re-parse the source code, but it
won't do this for a script specified on the command line. You can force
it to compile your script by importing the script in the interactive
interpreter, but this will also run your script (possibly not desirable).
You can compile the script without running it by using the py_compile
module:

python -c 'import py_compile; py_compile.compile("my_script.py")'

You can then run the compiled version using 'python my_script.pyc'
('python my_script.py' will ignore the compiled version).

Once thing I was thinking of was to use PyExe to make a
quick startup (does it compile down to C code, therefore not using the
Python interpreter at runtime?). Is this a possible solution?

Nope. py2exe just embeds the Python interpreter in the .exe. If
anything, startup would be slower, since py2exe also places all required
modules in a ZIP archive.

 

[simo]

Hello,

I'm looking at a small app which would need a very quick
startup time for the Python interpreter. It doesn't do much (copying
and caching of files, no GUI) but I need the Python interpreter to start
up very quickly (<1 second on a Windows box). Is there a way to have a
'stripped' down Python interpreter which can start up very quickly on a
windows box.

This has been discussed before, there's definitely something "odd"
about the Windows startup time in comparison to the UNIX "instant"
startup.

Once thing I was thinking of was to use PyExe to make a
quick startup (does it compile down to C code, therefore not using the
Python interpreter at runtime?). Is this a possible solution?

Nope, no compiling to C/machine code, just bytecode, and the
interpreter is a DLL.

I observe that the second time I start python it starts up quicker
but I'm assuming that this is dependent on the environment and can't be
relied upon (or something like that).

Yeah, I think that's something to do with Windows caching.

I keep saying, we need a Python compiler.....

 

[Peter Hansen]

This has been discussed before, there's definitely something "odd"
about the Windows startup time in comparison to the UNIX "instant"
startup.

Discussed, but not to a satisfactory conclusion I think.

The fact that some of us have near instantaneous startup times,
consistently (whether immediately after reboot or not) on Windows,
even with older machines, suggests strongly that those who do
not have something *wrong* with their system.

I don't recall the OP in the last thread that discussed this
ever coming back to report on exactly what OS and CPU etc.
he was using, and whether network issues might be involved,
etc, so until someone can prove otherwise, I think it's
safe to assume that anyone with a slow startup on Windows
has a misconfiguration or is doing something wrong.

-Peter

 

[Aahz]

2) Pre-compile the script. Normally, Python generates .pyc files for
imported modules so it doesn't have to re-parse the source code, but it
won't do this for a script specified on the command line. You can force
it to compile your script by importing the script in the interactive
interpreter, but this will also run your script (possibly not desirable).
You can compile the script without running it by using the py_compile
module:

python -c 'import py_compile; py_compile.compile("my_script.py")'

You can then run the compiled version using 'python my_script.pyc'
('python my_script.py' will ignore the compiled version).

That's a Bad Idea. Better Idea: move the code into a separate module
and put the code into a function. Then change this script into a very
short driver that imports/calls the function.
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"To me vi is Zen. To use vi is to practice zen. Every command is a
koan. Profound to the user, unintelligible to the uninitiated. You
discover truth everytime you use it." (e-mail address removed)

 

[3c273]

That's a Bad Idea.

*snip*

Hello,
I'm pretty new to Python and programming in general but I have done this
before. Why is this a bad idea?
Also (maybe not the right place to ask, but...), I was thinking of driving
up to BayPiggies because I am interested in tonights discussion but it's a
long drive and I'm afraid it would be way over my head. Do you get a lot of
newbie participants? Thanks for your input.
Louis

 

[Cousin Stanley]

| ....
| This primitive test shows these results on a Windows XP machine
| ( it won't work with Windows 98
| as it can't chain commands
| on the command line like that,
| but you could but it in a batch file ).
| ....

I stuck it in a .bat file under Win98_SE
on the same dual-boot machine that I used
for the Linux/Debian test ....

Win98_SE .... Python 2.3 Enthought Edition
Debian ...... Python 2.3.4

< K:\C\MinGWStudio\Samples\Time\Debug >
Thu 08-12-2004 18:49:03.23
py_time

Number of Seconds Since January 1, 1970 ....

1092361745
1092361746.75

The Linux/Debian number is only slightly quicker
with Python 2.3.4 and probably given a larger number
of tests, there might be no significant difference ....

Number of seconds ......... 1092323003
Python .................... 1092323003.36

So, for an old 250 MHz machine, I seem to be loading
Python OK for both Win98 & Linux ....

 

[Neil Benn]

<snip>
I don't recall the OP in the last thread that discussed this
ever coming back to report on exactly what OS and CPU etc.
he was using, and whether network issues might be involved,
etc, so until someone can prove otherwise, I think it's
safe to assume that anyone with a slow startup on Windows
has a misconfiguration or is doing something wrong.

<snip>
Hello,

Hi, I'm the OP, my box is :

WinXP Pro
2.4Gz Processor
256M RAM
Python2.3

As I said the first load takes 5 seconds and the consecutive loads
take fractions of a second - my assumption is that Windows is caching
something which is not suitable for my needs as it is not predictable
enough. There is no network issues involved in this, everything is
running off the local hard drive. I've just this moment run python
after the box having done some work and being asleep overnight and the
startup time was about 2 seconds. I may be doing something wrong but
the only configuration stuff I've done is to add into my environment
variables :

PYTHON_HOME=c:\program files\python23
PATH=%PATH%;PYTHON_HOME

This is simply to allow me to switch between different python
directories when I start to install different versions (I do the same
with the JVM). If python can't start up quick on windows, that's fine,
if I've got something miscofigured which can be fixed, that's fine too -
I just need to know if there is a 'quick start up' mode but if there
isn't then that's also fine. It's just info I need to design what will
be a very simple app.

Thanks, for your help.

Cheers,

Neil

--

Neil Benn
Senior Automation Engineer
Cenix BioScience
BioInnovations Zentrum
Tatzberg 47
D-01307
Dresden
Germany

Tel : +49 (0)351 4173 154
e-mail : (e-mail address removed)
Cenix Website : http://www.cenix-bioscience.com

 

[Neil Benn]

<snip>
PYTHON_HOME=c:\program files\python23
PATH=%PATH%;PYTHON_HOME
<snip>

PATH=%PATH%;%PYTHON_HOME%

Just in case someone mentions!

--

Neil Benn
Senior Automation Engineer
Cenix BioScience
BioInnovations Zentrum
Tatzberg 47
D-01307
Dresden
Germany

Tel : +49 (0)351 4173 154
e-mail : (e-mail address removed)
Cenix Website : http://www.cenix-bioscience.com

 

[Brian Quinlan]

As I said the first load takes 5 seconds and the consecutive loads
take fractions of a second - my assumption is that Windows is caching
something which is not suitable for my needs as it is not predictable
enough.

Neil, are you using Norton Antivirus with autoprotect enabled? I've
seen that cause problems in the past because NAV scans every file as it
is opened and Python loads a lot of file on startup.

Cheers,
Brian

 

[Brian Quinlan]

Good guess but no cigar!!!

Some other suggestions:

1. try running Python with the -v option is seeing if there is any
obvious pause between imports, any imports of uncompiled modules or
any importing of seemingly unnecessary modules

2. check your sys.path for references to potentially slow file systems
e.g. network drives

What version of Python are you using?

Cheers,
Brian

 

[Elbert Lev]

Hello,

I'm looking at a small app which would need a very quick
startup time for the Python interpreter...

On Windows you would be better of running your application as a
service. In this case you do not care about startup time. It can
automatically look for files you need or could have gui interface
(f.e. tray icon with menu).

 

[David Bolen]

I don't recall the OP in the last thread that discussed this
ever coming back to report on exactly what OS and CPU etc.
he was using, and whether network issues might be involved,
etc, so until someone can prove otherwise, I think it's
safe to assume that anyone with a slow startup on Windows
has a misconfiguration or is doing something wrong.

I don't think that's a safe assumption. For example, on a PIII-550
box running NT that I use, the first time I start up Python after a
prolonged absence (typically through a night when a virus scan is run)
it can take several seconds to start. The same holds true if I switch
versions of Python for comparisons (the first time a new version is
run takes a few seconds).

Subsequent executions of the same version are sub-second (albeit not
always by much).

There's no network issues involved (everything referenced during
startup is local), but to be honest, I'm not surprised, as I can
clearly hear my disk getting hit hard, and expect that absolutely
nothing was in the disk cache related to Python, so getting it loaded
and all the initial startup files loaded simply takes some time.

The fact that I experience similar noticeable delays with loading just
about everything else the first time after the nightly outage (e.g.,
the first time Outlook needs to load in IE to display an HTML message
encounters a very noticeable multi-second delay with heavy disk I/O)
makes me doubly think that the nightly virus scan is flushing just
about everything out of cache.

Similar startups on a PIII-333 running a Gentoo 2.4.x Linux install
definitely "feel" a bit faster. For example, using "python -v" on
both systems presents the same list of loaded modules (replacing
posix/posixpath with nt/ntpath under Windows) and then an additional
readline load once the interpreter starts under Linux. However, the
Linux session just streams those module load messages while you can
visually "see" each message coming out in turn with a tiny delay
between them under Windows. I haven't run timing measurements, but
it's noticeable enough that while they might not be major, I'm
confident they are there.

So for whatever reason, Windows does behave a bit more sluggishly than
Linux in my case. Maybe it's the use of the DLL with Windows (don't
know if any relocations have to take place) versus the static binary
under Linux. Maybe it's just slightly better filesystem I/O (my
Windows box may be more fragmented and/or fuller filesystems).
Certainly doesn't affect me in normal operation, but I guess if I was
starting Python very frequently I'd work on it more.

I also don't think there's much room for Python to be at fault at
least in my case - e.g., if anything it's system behavior and not
really local operations Python is performing.

But I don't think it automatically implies that my Windows system is
misconfigured or that I'm doing something wrong. Of course, my
timings aren't the consistent 5+ second startups that some others have
posted about, in which cases I do agree there might be an unexpected
network dependency or something else amiss.

-- David

 

[Peter Hansen]

I don't think that's a safe assumption. For example, on a PIII-550
box running NT that I use, the first time I start up Python after a
prolonged absence (typically through a night when a virus scan is run)
it can take several seconds to start.

[...snip description of how it affects everything else too]

I also don't think there's much room for Python to be at fault at
least in my case - e.g., if anything it's system behavior and not
really local operations Python is performing.

But I don't think it automatically implies that my Windows system is
misconfigured or that I'm doing something wrong.

You're quite right. I should have added the qualification that
if *other* software starts snappily, but Python does not, then
it is safe to assume it's a misconfiguration or someone doing
something wrong, whereas if all software starts slowly on
that machine then it may just be a sucky machine.

One assumes that those who are posting about Python starting
very slowly and needing faster startup time are not just
complaining indirectly about having slow machines in general.

-Peter

 

[Neuruss]

As far as I could see, the startup of python 2.3 is considerably
slower than python 2.2. It is something that really annoyed me when I
first installed Python 2.3 (and I did it in three different machines).
If the startup time is critical for your application, I suggest using
an older version of Python and, if possible, try to use Psyco to
improve its performance.