Re: loop in python

Discussion in 'Python' started by Peter Hansen, Aug 22, 2005.

  1. Peter Hansen

    Peter Hansen Guest

    km wrote:
    > ya i am sorry i tried with an empty loop first and then one which
    > emits a value as the snippet. I have tested it on my machine
    > and now ...
    >
    > what more do i need to accept python is slow when it comes to
    > loops concept ?


    You've sort of missed some of the points being made, which in essence
    are saying "your benchmark is less than useless, and you're confused
    about what matters".

    One key point about benchmarking is to make the test representative. If
    you are going to be running empty loops, then you're doing the right
    thing. If *all* your loops are going to do is print stuff, then you're
    doing the right thing with the version that "emits values".

    On the other hand, if you are really doing something else entirely, then
    your tests are telling you nothing about your real problem, and are in
    fact confusing you greatly, which is why they are less than useless.

    With the part about "confused about what matters" I mean that you are
    trying to optimize something a) when you don't have working code, and b)
    when you don't know that you need to make it any faster.

    The first rule of optimization is to make your code work first, and only
    then consider making it fast. The second rule of optimization is that
    you shouldn't waste time doing it unless you really need faster results
    (and it requires actual elapsed time or response time measurements to
    know this). Since you haven't got any working code, it's not possible
    that you *need* whatever negligible speed difference there might be
    between Python and Perl.

    And, lastly, in case this thought didn't get through either: Python is
    *not* significantly slower than Perl except in certain uncommon cases --
    perhaps including useless empty loops -- so if you really want to use
    Python, don't let your first attempts at benchmarking dissuade you.
    Really, trust us.

    Python's strengths lie in four things: the readability of the code, the
    huge range of library modules available, the elegance of its object
    oriented constructs, and the helpfulness of its community. Raw speed is
    not one of its strengths, but there are tens of thousands of people
    using it quite effectively and without daily concern for its speed (same
    as Perl, by the way since, again, they are _not_ significantly different
    in speed no matter what an empty loop test shows).

    -Peter
    Peter Hansen, Aug 22, 2005
    #1
    1. Advertising

  2. Peter Hansen

    Bill Mill Guest

    They come out even in the computer language shootout:

    http://shootout.alioth.debian.org/benchmark.php?test=all&lang=python&sort=fullcpu

    (tied 8-8 in execution time, although perl wins 4-12 on memory consumption)

    Peace
    Bill Mill

    On 8/23/05, km <> wrote:
    > Hi all,
    >
    > > thing. If *all* your loops are going to do is print stuff, then you're
    > > doing the right thing with the version that "emits values".

    >
    > ya most of the loops print values.
    >
    > > know this). Since you haven't got any working code, it's not possible
    > > that you *need* whatever negligible speed difference there might be
    > > between Python and Perl.
    > >
    > > Python, don't let your first attempts at benchmarking dissuade you.
    > > Really, trust us.

    >
    > ya i do.
    >
    > > Python's strengths lie in four things: the readability of the code, the
    > > huge range of library modules available, the elegance of its object
    > > oriented constructs, and the helpfulness of its community. Raw speed is
    > > not one of its strengths, but there are tens of thousands of people
    > > using it quite effectively and without daily concern for its speed (same
    > > as Perl, by the way since, again, they are _not_ significantly different
    > > in speed no matter what an empty loop test shows).

    >
    > I agree that python emphasizes on readability which i didnt see in many of the languages, but when the application concern is speed, does it mean that python is not yet ready? even most of the googling abt python vs perl convince me that perl is faster than python in most of the aspects. Also the first thing any newbie to python asks me is abt "raw speed in comparison with similar languages like perl" when i advocate python to perl.
    >
    >
    > regards,
    > KM
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
    Bill Mill, Aug 22, 2005
    #2
    1. Advertising

  3. Peter Hansen

    Peter Hansen Guest

    km wrote:
    >>thing. If *all* your loops are going to do is print stuff, then you're
    >>doing the right thing with the version that "emits values".

    >
    > ya most of the loops print values.


    No, you missed my point. I said if *all* the loops are going to do is
    print stuff. In other words, no other calculations, no function calls,
    no nothing. Just simple "print" statements. Doing this in your test
    just tests I/O in the C library and operating system, not anything to do
    with the speed of Python, as Bruno has clarified in detail.

    -Peter
    Peter Hansen, Aug 22, 2005
    #3
  4. Peter Hansen

    rafi Guest

    km wrote:

    > Also the first thing any newbie to python asks me is abt "raw speed
    > in comparison with similar languages like perl" when i advocate
    > python to perl.


    Always the same chorus... Just tell the newbies that speed is not on
    their top list.

    If most of the world's computer programs where running two times slower
    but with two times less bugs, imagine how many hours we would benefit...

    --
    rafi

    "Imagination is more important than knowledge."
    (Albert Einstein)
    rafi, Aug 22, 2005
    #4
  5. km a écrit :
    > Hi all,
    >
    > ya i am sorry i tried with an empty loop first and then one which emits a value as the snippet. I have tested it on my machine and now ...
    >
    > 1) perl (v 5.8) does the job in 0.005 seconds
    > 2) but python (v 2.4.1) is horribly slow its 0.61 seconds.
    > and using range() instead of xrange() in python snippet, it not better , it takes 0.57 seconds. just test it urself and see.
    >
    > what more do i need to accept python is slow when it comes to loops concept ?
    >


    #python -> python2.4.1:
    for i in xrange(100000):
    print i

    real 0m8.997s
    user 0m1.355s
    sys 0m0.483s


    /* C -> gcc -Wall -ansi -pedantic -O3 */
    # include <stdio.h>
    int main(void)
    {
    int i;
    for (i = 0; i < 100000; i++)
    printf("%d\n", i);
    return 0;
    }

    real 0m7.165s
    user 0m0.276s
    sys 0m0.430s

    Well... seems that Python is not *so* slow when compared to C on this
    one. But wait... Could it be possible that we're in fact merely
    benchmarking IO's ?-)

    lol...

    And what about this:

    # loops2.pl
    for $x (0..100000){}

    real 0m0.038s
    user 0m0.030s
    sys 0m0.002s

    # loops2.c
    int main(void)
    {
    int i;
    for (i = 0; i < 100000; i++);
    return 0;
    }

    real 0m0.002s
    user 0m0.001s
    sys 0m0.001s

    Oh my ! What more do you need to accept Perl is *so awfully slow*.

    If you want to keep on trolling on this, I suggest that you "benchmark"
    (lol) a Java, a PHP and a Ruby versions too...

    But I don't despair... Chances are that, one day, you understand that
    what is important is what you going on in the loop - not the loop itself.
    Bruno Desthuilliers, Aug 22, 2005
    #5
  6. km wrote:

    > Hi all,
    >
    >> thing. If *all* your loops are going to do is print stuff, then you're
    >> doing the right thing with the version that "emits values".

    >
    > ya most of the loops print values.


    In that case, you are interested in IO performance. The time spent handling
    the loop is not significant compared to the time spent executing the
    'print' statement - which is a very complex operation despite its simple
    usage.

    An implementation where a simple control flow structure has an measurable
    impact on the overall timing of an IO bound program would really suprising.
    Python suprises me often, but in different ways ;)

    Perhaps you could try to use sys.stdout.write() directly, avoiding the
    overhead of 'print'. Perhaps there are differences between the (default)
    output buffering settings of perl and python.

    --
    Benjamin Niemann
    Email: pink at odahoda dot de
    WWW: http://www.odahoda.de/
    Benjamin Niemann, Aug 22, 2005
    #6
  7. Peter Hansen

    Terry Reedy Guest

    "km" <> wrote in message
    news:...
    > I agree that python emphasizes on readability which i didnt see in many
    > of the languages, but when the application concern is speed, does it
    > mean that python is not yet ready? even most of the googling abt python


    Funny you should mention 'googling'. Are you aware that at least the first
    versions of the Google web spider and indexer were written in Python?
    Google continues to use Python. But then, they are smarter than most
    people when it comes to business economics and competitive advantage.

    > vs perl convince me that perl is faster than python in most of the
    > aspects.


    Total cost of a computer program project includes human programmer time to
    write, test, revise, and maintain. For many people, Python wins on this
    sometimes dominant aspect. For commercial projects, calendar time to
    market is also critical.

    > Also the first thing any newbie to python asks me is abt "raw speed in
    > comparison with similar languages like perl" when i advocate python to >
    > perl.


    I presume by 'raw speed' you/they mean cpu execution speed as opposed to
    human read, write, and understanding speed. I really don't know what to
    say to someone who counts their own time as worthless in comparision to
    penny a minute CPU time. But also, I would not advocate Python over Perl
    to a full-time programmer who is both comfortable and productive with Perl.

    Terry J. Reedy
    Terry Reedy, Aug 22, 2005
    #7
  8. Peter Hansen

    Terry Reedy Guest

    "Benjamin Niemann" <> wrote in message
    news:dedgo7$mru$...
    > km wrote:
    >
    >> Hi all,
    >>
    >>> thing. If *all* your loops are going to do is print stuff, then you're
    >>> doing the right thing with the version that "emits values".

    >>
    >> ya most of the loops print values.

    >
    > In that case, you are interested in IO performance. The time spent
    > handling
    > the loop is not significant compared to the time spent executing the
    > 'print' statement - which is a very complex operation despite its simple
    > usage.
    >
    > An implementation where a simple control flow structure has an measurable
    > impact on the overall timing of an IO bound program would really
    > suprising.
    > Python suprises me often, but in different ways ;)
    >
    > Perhaps you could try to use sys.stdout.write() directly, avoiding the
    > overhead of 'print'. Perhaps there are differences between the (default)
    > output buffering settings of perl and python.


    According to postings some years ago, the perl folks have worked specially
    hard to speed I/O, even to the point of replacing or rewriting some of the
    C stdio library, at least for some systems. Hence, at that time, Perl was
    expected and known to be noticeably faster than Python on such systems.
    Since then, Python I/O has been sped up to at least make better use of
    stdio, so the gap may be less.

    This suggests to me:
    1. One should not generalize to other aspects of computation. If, for
    instance, Python got Numerical Python before Perl got the equivalent, then
    it would have been multiples faster at numerical computation.
    2. One should compare the lastest with the lastest to compare the results
    of catch up with stay ahead efforts.
    3. One should test computations that are relevant to the question being
    asked. Python I/O speedups were aimed at 'for line in somefile: ...' and
    equivalent outputs, not in quickly writing thousands or millions of
    characters one at a time.

    Terry J. Reedy
    Terry Reedy, Aug 23, 2005
    #8
  9. Peter Hansen

    Peter Otten Guest

    km wrote:

    > the first thing any newbie to python asks me is abt "raw speed in
    > comparison with similar languages like perl" when i advocate python to
    > perl.


    In that case it is easy enough to follow the common industry practice and
    tweak your test cases until your favourite product comes out faster.

    $ time perl -e $'sub f() {for $i (0..10000000) {}} f();'

    real 0m1.075s
    user 0m1.073s
    sys 0m0.000s

    $ time python2.4 -c $'def f():\n for i in xrange(10000001): pass\nf()'

    real 0m0.878s
    user 0m0.871s
    sys 0m0.005s

    Working hypothesis: All benchmarks where Python comes out slower than Perl
    are not done properly :)

    Peter
    Peter Otten, Aug 23, 2005
    #9
  10. Peter Hansen

    km Guest

    Hi all,

    ya i am sorry i tried with an empty loop first and then one which emits a value as the snippet. I have tested it on my machine and now ...

    1) perl (v 5.8) does the job in 0.005 seconds
    2) but python (v 2.4.1) is horribly slow its 0.61 seconds.
    and using range() instead of xrange() in python snippet, it not better , it takes 0.57 seconds. just test it urself and see.

    what more do i need to accept python is slow when it comes to loops concept ?

    PS : my linux box is running fedora core 2 with 256 MB RAM

    regards,
    KM
    -----------------------------------------------------------------------
    On Tue, Aug 23, 2005 at 12:25:01PM +0530, km wrote:
    > Hi all,
    >
    > Why is it that the implementation of empty loop so slow in python when compared to perl ?
    >
    > #i did this in python (v 1.5)
    > for x in xrange(1000):
    > print x
    > # this took 0.017 seconds
    > --------------------------
    > #similar code in perl (v 5.6):
    > for $x (0..1000)
    > {
    > print $x;
    > }
    > # this took 0.005 seconds only !!!
    >
    > Is python runtime slow at all aspects when compared to perl ?
    > I really wonder what makes python slower than perl ?
    > Is there any proposal to make python faster in future versions ?
    >
    > curious to know all these ...
    >
    > regards,
    > KM
    >
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
    km, Aug 23, 2005
    #10
  11. Terry Reedy wrote:
    > "Benjamin Niemann" <> wrote in message
    > news:dedgo7$mru$...
    >

    (snip)
    >>In that case, you are interested in IO performance. The time spent
    >>handling
    >>the loop is not significant compared to the time spent executing the
    >>'print' statement - which is a very complex operation despite its simple
    >>usage.
    >>
    >>An implementation where a simple control flow structure has an measurable
    >>impact on the overall timing of an IO bound program would really
    >>suprising.
    >>Python suprises me often, but in different ways ;)
    >>
    >>Perhaps you could try to use sys.stdout.write() directly, avoiding the
    >>overhead of 'print'. Perhaps there are differences between the (default)
    >>output buffering settings of perl and python.

    >
    >
    > According to postings some years ago, the perl folks have worked specially
    > hard to speed I/O, even to the point of replacing or rewriting some of the
    > C stdio library, at least for some systems. Hence, at that time, Perl was
    > expected and known to be noticeably faster than Python on such systems.


    The truth is that the Perl version of the "print loop" program is
    incredibly faster than it's C counterpart (at least on my gentoo/linux
    box). Since the "no-op loop" program is faster in C (I guess that GCC
    'optimizes out' the whole loop !-), it seems clear that Perl's IO do not
    relie on the stdlib implementation.

    > Since then, Python I/O has been sped up to at least make better use of
    > stdio, so the gap may be less.


    The Python and C versions of the "print loop" program have execution
    times that are close enough...

    > 3. One should test computations that are relevant to the question being
    > asked.


    Of course !-)

    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in ''.split('@')])"
    bruno modulix, Aug 23, 2005
    #11
  12. Peter Hansen

    km Guest

    Hi all,

    > thing. If *all* your loops are going to do is print stuff, then you're
    > doing the right thing with the version that "emits values".


    ya most of the loops print values.

    > know this). Since you haven't got any working code, it's not possible
    > that you *need* whatever negligible speed difference there might be
    > between Python and Perl.
    >
    > Python, don't let your first attempts at benchmarking dissuade you.
    > Really, trust us.


    ya i do.

    > Python's strengths lie in four things: the readability of the code, the
    > huge range of library modules available, the elegance of its object
    > oriented constructs, and the helpfulness of its community. Raw speed is
    > not one of its strengths, but there are tens of thousands of people
    > using it quite effectively and without daily concern for its speed (same
    > as Perl, by the way since, again, they are _not_ significantly different
    > in speed no matter what an empty loop test shows).


    I agree that python emphasizes on readability which i didnt see in many of the languages, but when the application concern is speed, does it mean that python is not yet ready? even most of the googling abt python vs perl convince me that perl is faster than python in most of the aspects. Also the first thing any newbie to python asks me is abt "raw speed in comparison with similar languages like perl" when i advocate python to perl.


    regards,
    KM
    km, Aug 23, 2005
    #12
  13. Peter Hansen

    James Guest

    I am going to be a bit blunt. Don't get offended.

    >> Also the first thing any newbie to python asks me is abt "raw speed in comparison with similar languages like perl" when i advocate python to perl.


    Judging by your other posts, you are a newbie yourself. You are not
    really in a position to 'advocate' any language over others.

    >> I agree that python emphasizes on readability which i didnt see in many of the languages, but when the application concern is speed, does it mean that python is not yet ready? even most of the googling abt python vs perl convince me that perl is faster than python in most of the aspects.


    One does not compare speed when they use Perl/Python/Ruby/Tcl. They are
    all more or less in the same performance ball park.

    Next don't make up your own benchmarks and jump to conclusions. Writing
    good benchmarks is an art. If you need data, look at peer reviewed
    benchmarks such as this one. http://shootout.alioth.debian.org/
    As with all benchmarks, it is really hard to make general conclusions.

    And you are simply looking at the wrong issue. Even if Python is 20
    times slower than it's current implementation, it would still serve my
    purposes. Do you believe that you need more speed? Tell us what is it
    exactly that you are building and we will tell you what to do. Fetishes
    with Speed and executable size are very common for young newbies. I
    know because I had been there myself several years ago.

    Python has been more than ready as far as speed goes. Real people, real
    enterprises have been using Python in high load applications for quite
    a while now and there is nothing really left to proove. People have
    written entire application servers and databases in Python.

    I taught myself atleast half a dozen ways to write native extensions
    for Python, just in case. In the past 4 yrs or so that I have been
    using Python as my main language, I did not need to speed up my Python
    program even once with a custom extension. And I process multi giga
    byte data sets. Why? Because, if your program is slow, chances really
    are that your algorithm is slow, not the language. And most of the
    Python modules that are available that need speed (GUIs, image
    processing etc), are already written in C so that you, as a user, don't
    have to worry.

    Just get over your imaginary need for speed and learn to use Python for
    what it is intended. Once again, post your actual application need, not
    vague requirements with artificial conditions (I don't want C modules).

    You said, elsewhere that you are writing a web application. People have
    been using CGI, which has a terrible performace record for decades on
    very slow machines compared to modern PCs. My point is, web
    applications, "generally" aren't exactly the kind of applications that
    have a lot of computational overhead, atleast not from the logic that
    runs your site and is likely to be written in Python.
    James, Aug 25, 2005
    #13
  14. James enlightened us with:
    > One does not compare speed when they use Perl/Python/Ruby/Tcl. They
    > are all more or less in the same performance ball park.


    I don't want to offend you or anything, but doesn't the second
    sentence mean that someone DID do a speed comparison?

    Sybren
    --
    The problem with the world is stupidity. Not saying there should be a
    capital punishment for stupidity, but why don't we just take the
    safety labels off of everything and let the problem solve itself?
    Frank Zappa
    Sybren Stuvel, Aug 25, 2005
    #14
  15. Peter Hansen

    Peter Hansen Guest

    Sybren Stuvel wrote:
    > James enlightened us with:
    >
    >>One does not compare speed when they use Perl/Python/Ruby/Tcl. They
    >>are all more or less in the same performance ball park.

    >
    >
    > I don't want to offend you or anything, but doesn't the second
    > sentence mean that someone DID do a speed comparison?


    Yes, and has shown that they are in the same ballpark, and therefore one
    does not _need_ to compare speed any more. At least, that's how I read
    what James posted.

    -Peter
    Peter Hansen, Aug 25, 2005
    #15
  16. Peter Hansen enlightened us with:
    > Yes, and has shown that they are in the same ballpark, and therefore
    > one does not _need_ to compare speed any more.


    Ok. I'd worded it as "there have been tests already, so there is no
    need to do your own", instead of "one does not test".

    Sybren
    --
    The problem with the world is stupidity. Not saying there should be a
    capital punishment for stupidity, but why don't we just take the
    safety labels off of everything and let the problem solve itself?
    Frank Zappa
    Sybren Stuvel, Aug 25, 2005
    #16
  17. Peter Hansen

    James Guest

    >> I don't want to offend you or anything, but doesn't the second sentence mean that someone DID do a speed comparison?

    I did provide Language Shootout link in the next paragraph of the post
    you referred to along with an obligatory caution about interpreting
    benchmarks. The Language Shootout is a general enough benchmark to
    cover all popular languages not just comparing between
    Perl/Python/Ruby/Tcl (which I said was pointless). So the statements
    are not in conflict.
    James, Aug 25, 2005
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?VGltOjouLg==?=

    Loop the loop...

    =?Utf-8?B?VGltOjouLg==?=, Feb 16, 2005, in forum: ASP .Net
    Replies:
    2
    Views:
    1,370
    Karl Seguin
    Feb 16, 2005
  2. Steven

    while loop in a while loop

    Steven, Mar 24, 2005, in forum: Java
    Replies:
    5
    Views:
    2,213
    Tim Slattery
    Mar 30, 2005
  3. -
    Replies:
    12
    Views:
    675
    Remon van Vliet
    Jun 15, 2005
  4. Byte
    Replies:
    4
    Views:
    401
  5. Isaac Won
    Replies:
    9
    Views:
    349
    Ulrich Eckhardt
    Mar 4, 2013
Loading...

Share This Page