RE: I'm missing something here with range vs. xrange

Discussion in 'Python' started by Joe Goldthwaite, Dec 7, 2007.

  1. Here's the simple benchmark;

    start = time.time()
    for x in xrange(3):
    for y in xrange(10000000):
    pass
    print 'xRange %s' % (time.time() - start)

    start = time.time()
    for x in range(3):
    for y in range(10000000):
    pass
    print 'Range %s' % (time.time() - start)

    Here's what I get;

    xRange 92.5529999733
    Range 95.2669999599

    Not a lot of difference. Range is slower but not by much. I know that range
    builds
    a list then iterates through it. I thought that xrange just kept a counter
    that was
    incremented and returned with each call. No list was ever created. If that's
    true
    (and I guess it's not), xrange would be much faster than range. It seems
    almost
    identical. Given the amount of performance difference, I don't see why
    xrange even
    exists.

    P.S. I searched google again to try and find out more about range vs.
    xrange and
    this thread came up first on the list. That was strange. I thought I'd
    found
    someone else asking the same question as me until I clicked on the link and
    found
    out it actually was me.

    -----Original Message-----
    From: python-list-bounces+joe=
    [mailto:python-list-bounces+joe=]On Behalf Of
    Bjoern Schliessmann
    Sent: Thursday, December 06, 2007 3:33 PM
    To:
    Subject: Re: I'm missing something here with range vs. xrange


    Joe Goldthwaite wrote:
    > I read that the range function builds a list and that xrange
    > returns an iterator and is therefore more efficient.


    This is generally not true.

    > In my testing, they both come out to almost exactly the same
    > performance wise. Did something get changed in Python 2.4 to make
    > them identical?


    No. Try again with a list of length 10000000.

    Regards,


    Björn

    --
    BOFH excuse #359:

    YOU HAVE AN I/O ERROR -> Incompetent Operator error

    --
    http://mail.python.org/mailman/listinfo/python-list
     
    Joe Goldthwaite, Dec 7, 2007
    #1
    1. Advertisements

  2. Joe Goldthwaite

    Guest

    On Dec 7, 3:08 pm, "Joe Goldthwaite" <> wrote:
    > Here's the simple benchmark;
    >
    > start = time.time()
    > for x in xrange(3):
    > for y in xrange(10000000):
    > pass
    > print 'xRange %s' % (time.time() - start)
    >
    > start = time.time()
    > for x in range(3):
    > for y in range(10000000):
    > pass
    > print 'Range %s' % (time.time() - start)
    >
    > Here's what I get;
    >
    > xRange 92.5529999733
    > Range 95.2669999599
    >
    > Not a lot of difference. Range is slower but not by much. I know that range
    > builds
    > a list then iterates through it. I thought that xrange just kept a counter
    > that was
    > incremented and returned with each call. No list was ever created. If that's
    > true
    > (and I guess it's not), xrange would be much faster than range. It seems
    > almost
    > identical. Given the amount of performance difference, I don't see why
    > xrange even
    > exists.


    Try tracking your memory usage during the benchmark and
    it will become very clear why xrange exists.

    >
    > P.S. I searched google again to try and find out more about range vs.
    > xrange and
    > this thread came up first on the list. That was strange. I thought I'd
    > found
    > someone else asking the same question as me until I clicked on the link and
    > found
    > out it actually was me.
    >
    >
    >
    > -----Original Message-----
    > From: python-list-bounces+joe=
    >
    > [mailto:python-list-bounces+joe=]On Behalf Of
    > Bjoern Schliessmann
    > Sent: Thursday, December 06, 2007 3:33 PM
    > To:
    > Subject: Re: I'm missing something here with range vs. xrange
    >
    > Joe Goldthwaite wrote:
    > > I read that the range function builds a list and that xrange
    > > returns an iterator and is therefore more efficient.

    >
    > This is generally not true.
    >
    > > In my testing, they both come out to almost exactly the same
    > > performance wise. Did something get changed in Python 2.4 to make
    > > them identical?

    >
    > No. Try again with a list of length 10000000.
    >
    > Regards,
    >
    > Björn
    >
    > --
    > BOFH excuse #359:
    >
    > YOU HAVE AN I/O ERROR -> Incompetent Operator error
    >
    > --http://mail.python.org/mailman/listinfo/python-list- Hide quoted text -
    >
    > - Show quoted text -
     
    , Dec 7, 2007
    #2
    1. Advertisements

  3. Joe Goldthwaite

    John Machin Guest

    On Dec 8, 8:08 am, "Joe Goldthwaite" <> wrote:
    > Here's the simple benchmark;
    >
    > start = time.time()
    > for x in xrange(3):
    > for y in xrange(10000000):
    > pass
    > print 'xRange %s' % (time.time() - start)
    >
    > start = time.time()
    > for x in range(3):
    > for y in range(10000000):
    > pass
    > print 'Range %s' % (time.time() - start)
    >
    > Here's what I get;
    >
    > xRange 92.5529999733
    > Range 95.2669999599
    >
    > Not a lot of difference. Range is slower but not by much.


    90+ seconds?? What hardware, OS, and Python version? What else was
    running in the background?

    With this kit:
    OS Name: Microsoft Windows XP Professional
    Version: 5.1.2600 Service Pack 2 Build 2600
    Processor: x86 Family 15 Model 36 Stepping 2 AuthenticAMD ~1995 Mhz
    Python: Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310
    32 bit (Intel)] on win32

    and your exact code, I get:
    xRange 4.0
    Range 5.0

    After changing time.time to time.clock, I get:
    xRange 4.00560127055
    Range 4.8927366467

    After making the "benchmark" somewhat more realistic by executing it
    inside a function, I get:
    xRange 1.86865816745
    Range 3.31902658019

    By "inside a function", I mean changing
    <script>
    to

    def foo():
    <script>
    foo()
     
    John Machin, Dec 7, 2007
    #3
  4. Joe Goldthwaite

    Tim Chase Guest

    >> Here's what I get;
    >>
    >> xRange 92.5529999733
    >> Range 95.2669999599

    >
    > Try tracking your memory usage during the benchmark and
    > it will become very clear why xrange exists.


    Or, when memory-constrained and this extra memory usage pushes
    your machine to pound on your swap...not a pretty sight.

    -tkc (from his box with a meager 96 megs of memory, though
    <geezer>back when I worked on a machine with a copious 48k of
    memory and enjoyed it...</geezer>)
     
    Tim Chase, Dec 7, 2007
    #4
  5. Joe Goldthwaite

    Carl Banks Guest

    On Dec 7, 4:08 pm, "Joe Goldthwaite" <> wrote:
    > Here's the simple benchmark;
    >
    > start = time.time()
    > for x in xrange(3):
    > for y in xrange(10000000):
    > pass
    > print 'xRange %s' % (time.time() - start)
    >
    > start = time.time()
    > for x in range(3):
    > for y in range(10000000):
    > pass
    > print 'Range %s' % (time.time() - start)
    >
    > Here's what I get;
    >
    > xRange 92.5529999733
    > Range 95.2669999599
    >
    > Not a lot of difference.



    Try this:

    start = time.time()
    for x in xrange(3):
    for y in xrange(10000000):
    if y == 10:
    break
    print 'xRange %s' % (time.time() - start)

    start = time.time()
    for x in range(3):
    for y in range(10000000):
    if y == 10:
    break
    print 'Range %s' % (time.time() - start)


    > Range is slower but not by much. I know that range
    > builds
    > a list then iterates through it. I thought that xrange just kept a counter
    > that was
    > incremented and returned with each call. No list was ever created. If that's
    > true
    > (and I guess it's not), xrange would be much faster than range. It seems
    > almost
    > identical.


    xrange doesn't merely return a counter. It creates an int object and
    returns it, which is somewhat more expensive. The time required to
    create ten million integer objects dwarfs the time required to
    allocate a single list with ten million entries, and this cost occurs
    with both range and xrange, unless you break early.

    > Given the amount of performance difference, I don't see why
    > xrange even
    > exists.


    To keep memory use down. It's pointless and wasteful to allocate
    about 160 megabytes (or 1 kilobyte) of data to store a list when you
    only need 36 or so of those bytes at any given time.



    Carl Banks
     
    Carl Banks, Dec 8, 2007
    #5
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. JezB

    Am I missing something here ??

    JezB, Dec 4, 2003, in forum: ASP .Net
    Replies:
    3
    Views:
    405
  2. =?Utf-8?B?RGVscGhpbg==?=

    Am I missing something here?

    =?Utf-8?B?RGVscGhpbg==?=, Sep 8, 2005, in forum: ASP .Net
    Replies:
    5
    Views:
    585
    S. Justin Gengo
    Sep 8, 2005
  3. Willie

    Missing something here...

    Willie, Nov 1, 2003, in forum: C++
    Replies:
    1
    Views:
    390
    =?iso-8859-1?Q?Juli=E1n?= Albo
    Nov 1, 2003
  4. Steve R. Hastings

    efficiency of range() and xrange() in for loops

    Steve R. Hastings, Apr 5, 2006, in forum: Python
    Replies:
    29
    Views:
    1,026
    Fredrik Lundh
    Apr 9, 2006
  5. stdazi

    why I don't like range/xrange

    stdazi, Feb 16, 2007, in forum: Python
    Replies:
    17
    Views:
    681
    Bruno Desthuilliers
    Feb 18, 2007
  6. Joe Goldthwaite
    Replies:
    2
    Views:
    347
    Chris Mellon
    Dec 7, 2007
  7. harrismh777
    Replies:
    11
    Views:
    1,644
    Steven D'Aprano
    Aug 4, 2011
  8. Wayne  Wengert

    Missing Something Obvious Here!

    Wayne Wengert, Aug 30, 2003, in forum: ASP General
    Replies:
    0
    Views:
    185
    Wayne Wengert
    Aug 30, 2003
Loading...