slow Python 3.0 write performance?

[Bruno Desthuilliers]

Istvan Albert a écrit :

Could someone run the code below on both Python 2.5 and 3.0

For me (on Windows) it runs over 7 times slower with Python 3.0

Already covered, I think:
http://groups.google.com/group/comp.lang.python/browse_frm/thread/9046eee09137c657#

 

[Istvan Albert]

Could someone run the code below on both Python 2.5 and 3.0

For me (on Windows) it runs over 7 times slower with Python 3.0

import time

lo, hi, step = 10**5, 10**6, 10**5

# writes increasingly more lines to a file
for N in range(lo, hi, step):
fp = open('foodata.txt', 'wt')
start = time.time()
for i in range( N ):
fp.write( '%s\n' % i)
fp.close()
stop = time.time()
print ( "%s\t%s" % (N, stop-start) )

 

[Mike Driscoll]

Could someone run the code below on both Python 2.5 and 3.0

For me (on Windows) it runs over 7 times slower with Python 3.0

import time

lo, hi, step = 10**5, 10**6, 10**5

# writes increasingly more lines to a file
for N in range(lo, hi, step):
    fp = open('foodata.txt', 'wt')
    start = time.time()
    for i in range( N ):
        fp.write( '%s\n' % i)
    fp.close()
    stop = time.time()
    print ( "%s\t%s" % (N, stop-start) )

Ran on Windows XP virtual machine:

3.0 output:

100000 0.889999866486
200000 1.79699993134
300000 2.875
400000 3.73399996758
500000 4.71899986267
600000 5.59400010109
700000 7.04699993134
800000 7.31299996376
900000 8.375


2.5.2 output:

100000 0.156000137329
200000 0.296999931335
300000 0.640000104904
400000 0.640000104904
500000 0.78200006485
600000 0.952999830246
700000 1.13999986649
800000 1.25
900000 1.42199993134

Slowness in this exercise is confirmed on Windows XP.

Mike

 

[skip]

Istvan> Could someone run the code below on both Python 2.5 and 3.0 For
Istvan> me (on Windows) it runs over 7 times slower with Python 3.0

...

I/O was completely rewritten for Python 3.0. Stdio is no longer used. At
the moment I believe much of the io subsystem is still implemented in
Python. Note these comments in io.py:

# New I/O library conforming to PEP 3116.

# This is a prototype; hopefully eventually some of this will be
# reimplemented in C.

It should get faster over time. It will get faster over a shorter period of
time if people contribute patches.

 

[Istvan Albert]

It should get faster over time.  It will get faster over a shorter period of
time if people contribute patches.

I see, thanks for the clarification.

I will make the point though that this makes python 3.0 unsuited for
anyone who has to process data. One could live with slowdowns of say
20-50 percent, to get the goodies that 3.0 offers, but when a process
that takes 1 second suddenly starts to take 10, it is makes the
situation untenable.

best,

Istvan

 

[MRAB]

I see, thanks for the clarification.

I will make the point though that this makes python 3.0 unsuited for
anyone who has to process data. One could live with slowdowns of say
20-50 percent, to get the goodies that 3.0 offers, but when a process
that takes 1 second suddenly starts to take 10, it is makes the
situation untenable.

Does pysco with with Python 3.0 (the homepage says 2.5)? If it does then
that might help!

 

[bearophileHUGS]

Christian Heimes:

I've fixed the read() slowness yesterday. You'll get the fix in the next
release of Python 3.0 in a couple of weeks.

Thank you.

Bye,
bearophile

 

[Istvan Albert]

I've fixed the read() slowness yesterday. You'll get the fix in the next
release of Python 3.0 in a couple of weeks.

Does this fix speed up the write() function as well?

A previous poster suggested that in this case the slowdown is caused
by the new io code being written in python rather than C.

Istvan