slow Python 3.0 write performance?

Discussion in 'Python' started by Bruno Desthuilliers, Dec 5, 2008.

  1. Istvan Albert a écrit :
    > Could someone run the code below on both Python 2.5 and 3.0
    >
    > For me (on Windows) it runs over 7 times slower with Python 3.0


    Already covered, I think:
    http://groups.google.com/group/comp.lang.python/browse_frm/thread/9046eee09137c657#


    > import time
    >
    > lo, hi, step = 10**5, 10**6, 10**5
    >
    > # writes increasingly more lines to a file
    > for N in range(lo, hi, step):
    > fp = open('foodata.txt', 'wt')
    > start = time.time()
    > for i in range( N ):
    > fp.write( '%s\n' % i)
    > fp.close()
    > stop = time.time()
    > print ( "%s\t%s" % (N, stop-start) )
    >
    >
    >
     
    Bruno Desthuilliers, Dec 5, 2008
    #1
    1. Advertising

  2. Could someone run the code below on both Python 2.5 and 3.0

    For me (on Windows) it runs over 7 times slower with Python 3.0

    import time

    lo, hi, step = 10**5, 10**6, 10**5

    # writes increasingly more lines to a file
    for N in range(lo, hi, step):
    fp = open('foodata.txt', 'wt')
    start = time.time()
    for i in range( N ):
    fp.write( '%s\n' % i)
    fp.close()
    stop = time.time()
    print ( "%s\t%s" % (N, stop-start) )
     
    Istvan Albert, Dec 5, 2008
    #2
    1. Advertising

  3. On Dec 5, 12:54 pm, Istvan Albert <> wrote:
    > Could someone run the code below on both Python 2.5 and 3.0
    >
    > For me (on Windows) it runs over 7 times slower with Python 3.0
    >
    > import time
    >
    > lo, hi, step = 10**5, 10**6, 10**5
    >
    > # writes increasingly more lines to a file
    > for N in range(lo, hi, step):
    >     fp = open('foodata.txt', 'wt')
    >     start = time.time()
    >     for i in range( N ):
    >         fp.write( '%s\n' % i)
    >     fp.close()
    >     stop = time.time()
    >     print ( "%s\t%s" % (N, stop-start) )


    Ran on Windows XP virtual machine:

    3.0 output:

    100000 0.889999866486
    200000 1.79699993134
    300000 2.875
    400000 3.73399996758
    500000 4.71899986267
    600000 5.59400010109
    700000 7.04699993134
    800000 7.31299996376
    900000 8.375


    2.5.2 output:

    100000 0.156000137329
    200000 0.296999931335
    300000 0.640000104904
    400000 0.640000104904
    500000 0.78200006485
    600000 0.952999830246
    700000 1.13999986649
    800000 1.25
    900000 1.42199993134

    Slowness in this exercise is confirmed on Windows XP.

    Mike
     
    Mike Driscoll, Dec 5, 2008
    #3
  4. Bruno Desthuilliers

    Guest

    Istvan> Could someone run the code below on both Python 2.5 and 3.0 For
    Istvan> me (on Windows) it runs over 7 times slower with Python 3.0

    ...

    I/O was completely rewritten for Python 3.0. Stdio is no longer used. At
    the moment I believe much of the io subsystem is still implemented in
    Python. Note these comments in io.py:

    # New I/O library conforming to PEP 3116.

    # This is a prototype; hopefully eventually some of this will be
    # reimplemented in C.

    It should get faster over time. It will get faster over a shorter period of
    time if people contribute patches.

    --
    Skip Montanaro - - http://smontanaro.dyndns.org/
     
    , Dec 5, 2008
    #4
  5. On Dec 5, 3:06 pm, wrote:

    > It should get faster over time.  It will get faster over a shorter period of
    > time if people contribute patches.


    I see, thanks for the clarification.

    I will make the point though that this makes python 3.0 unsuited for
    anyone who has to process data. One could live with slowdowns of say
    20-50 percent, to get the goodies that 3.0 offers, but when a process
    that takes 1 second suddenly starts to take 10, it is makes the
    situation untenable.

    best,

    Istvan
     
    Istvan Albert, Dec 5, 2008
    #5
  6. Bruno Desthuilliers

    MRAB Guest

    Istvan Albert wrote:
    > On Dec 5, 3:06 pm, wrote:
    >
    >> It should get faster over time. It will get faster over a shorter period of
    >> time if people contribute patches.

    >
    > I see, thanks for the clarification.
    >
    > I will make the point though that this makes python 3.0 unsuited for
    > anyone who has to process data. One could live with slowdowns of say
    > 20-50 percent, to get the goodies that 3.0 offers, but when a process
    > that takes 1 second suddenly starts to take 10, it is makes the
    > situation untenable.
    >

    Does pysco with with Python 3.0 (the homepage says 2.5)? If it does then
    that might help! :)
     
    MRAB, Dec 5, 2008
    #6
  7. Bruno Desthuilliers

    Guest

    Christian Heimes:
    > I've fixed the read() slowness yesterday. You'll get the fix in the next
    > release of Python 3.0 in a couple of weeks.


    Thank you.

    Bye,
    bearophile
     
    , Dec 5, 2008
    #7
  8. On Dec 5, 3:41 pm, Christian Heimes <> wrote:

    > I've fixed the read() slowness yesterday. You'll get the fix in the next
    > release of Python 3.0 in a couple of weeks.


    Does this fix speed up the write() function as well?

    A previous poster suggested that in this case the slowdown is caused
    by the new io code being written in python rather than C.

    Istvan
     
    Istvan Albert, Dec 6, 2008
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    3,047
  2. HK
    Replies:
    3
    Views:
    458
  3. mike
    Replies:
    3
    Views:
    399
    Virgil Green
    Jul 11, 2005
  4. JosephByrns

    Slow, then quick then slow

    JosephByrns, Jul 10, 2006, in forum: ASP .Net
    Replies:
    4
    Views:
    2,499
    codezilla94
    Nov 13, 2007
  5. con

    SLOW oh so slow

    con, May 26, 2008, in forum: ASP .Net
    Replies:
    4
    Views:
    431
Loading...

Share This Page