RE: I'm missing something here with range vs. xrange

Discussion in 'Python' started by Joe Goldthwaite, Dec 7, 2007.

  1. Here's the simple benchmark;

    start = time.time()
    for x in xrange(3):
    for y in xrange(10000000):
    pass
    print 'xRange %s' % (time.time() - start)

    start = time.time()
    for x in range(3):
    for y in range(10000000):
    pass
    print 'Range %s' % (time.time() - start)

    Here's what I get;

    xRange 92.5529999733
    Range 95.2669999599

    Not a lot of difference. Range is slower but not by much. I know that range
    builds
    a list then iterates through it. I thought that xrange just kept a counter
    that was
    incremented and returned with each call. No list was ever created. If that's
    true
    (and I guess it's not), xrange would be much faster than range. It seems
    almost
    identical. Given the amount of performance difference, I don't see why
    xrange even
    exists.

    P.S. I searched google again to try and find out more about range vs.
    xrange and
    this thread came up first on the list. That was strange. I thought I'd
    found
    someone else asking the same question as me until I clicked on the link and
    found
    out it actually was me.

    -----Original Message-----
    From: python-list-bounces+joe=
    [mailto:python-list-bounces+joe=]On Behalf Of
    Bjoern Schliessmann
    Sent: Thursday, December 06, 2007 3:33 PM
    To:
    Subject: Re: I'm missing something here with range vs. xrange


    Joe Goldthwaite wrote:
    > I read that the range function builds a list and that xrange
    > returns an iterator and is therefore more efficient.


    This is generally not true.

    > In my testing, they both come out to almost exactly the same
    > performance wise. Did something get changed in Python 2.4 to make
    > them identical?


    No. Try again with a list of length 10000000.

    Regards,


    Björn

    --
    BOFH excuse #359:

    YOU HAVE AN I/O ERROR -> Incompetent Operator error

    --
    http://mail.python.org/mailman/listinfo/python-list
     
    Joe Goldthwaite, Dec 7, 2007
    #1
    1. Advertising

  2. Joe Goldthwaite

    Guest

    On Dec 7, 3:08 pm, "Joe Goldthwaite" <> wrote:
    > Here's the simple benchmark;
    >
    > start = time.time()
    > for x in xrange(3):
    > for y in xrange(10000000):
    > pass
    > print 'xRange %s' % (time.time() - start)
    >
    > start = time.time()
    > for x in range(3):
    > for y in range(10000000):
    > pass
    > print 'Range %s' % (time.time() - start)
    >
    > Here's what I get;
    >
    > xRange 92.5529999733
    > Range 95.2669999599
    >
    > Not a lot of difference. Range is slower but not by much. I know that range
    > builds
    > a list then iterates through it. I thought that xrange just kept a counter
    > that was
    > incremented and returned with each call. No list was ever created. If that's
    > true
    > (and I guess it's not), xrange would be much faster than range. It seems
    > almost
    > identical. Given the amount of performance difference, I don't see why
    > xrange even
    > exists.


    Try tracking your memory usage during the benchmark and
    it will become very clear why xrange exists.

    >
    > P.S. I searched google again to try and find out more about range vs.
    > xrange and
    > this thread came up first on the list. That was strange. I thought I'd
    > found
    > someone else asking the same question as me until I clicked on the link and
    > found
    > out it actually was me.
    >
    >
    >
    > -----Original Message-----
    > From: python-list-bounces+joe=
    >
    > [mailto:python-list-bounces+joe=]On Behalf Of
    > Bjoern Schliessmann
    > Sent: Thursday, December 06, 2007 3:33 PM
    > To:
    > Subject: Re: I'm missing something here with range vs. xrange
    >
    > Joe Goldthwaite wrote:
    > > I read that the range function builds a list and that xrange
    > > returns an iterator and is therefore more efficient.

    >
    > This is generally not true.
    >
    > > In my testing, they both come out to almost exactly the same
    > > performance wise. Did something get changed in Python 2.4 to make
    > > them identical?

    >
    > No. Try again with a list of length 10000000.
    >
    > Regards,
    >
    > Björn
    >
    > --
    > BOFH excuse #359:
    >
    > YOU HAVE AN I/O ERROR -> Incompetent Operator error
    >
    > --http://mail.python.org/mailman/listinfo/python-list- Hide quoted text -
    >
    > - Show quoted text -
     
    , Dec 7, 2007
    #2
    1. Advertising

  3. Joe Goldthwaite

    John Machin Guest

    On Dec 8, 8:08 am, "Joe Goldthwaite" <> wrote:
    > Here's the simple benchmark;
    >
    > start = time.time()
    > for x in xrange(3):
    > for y in xrange(10000000):
    > pass
    > print 'xRange %s' % (time.time() - start)
    >
    > start = time.time()
    > for x in range(3):
    > for y in range(10000000):
    > pass
    > print 'Range %s' % (time.time() - start)
    >
    > Here's what I get;
    >
    > xRange 92.5529999733
    > Range 95.2669999599
    >
    > Not a lot of difference. Range is slower but not by much.


    90+ seconds?? What hardware, OS, and Python version? What else was
    running in the background?

    With this kit:
    OS Name: Microsoft Windows XP Professional
    Version: 5.1.2600 Service Pack 2 Build 2600
    Processor: x86 Family 15 Model 36 Stepping 2 AuthenticAMD ~1995 Mhz
    Python: Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310
    32 bit (Intel)] on win32

    and your exact code, I get:
    xRange 4.0
    Range 5.0

    After changing time.time to time.clock, I get:
    xRange 4.00560127055
    Range 4.8927366467

    After making the "benchmark" somewhat more realistic by executing it
    inside a function, I get:
    xRange 1.86865816745
    Range 3.31902658019

    By "inside a function", I mean changing
    <script>
    to

    def foo():
    <script>
    foo()
     
    John Machin, Dec 7, 2007
    #3
  4. Joe Goldthwaite

    Tim Chase Guest

    >> Here's what I get;
    >>
    >> xRange 92.5529999733
    >> Range 95.2669999599

    >
    > Try tracking your memory usage during the benchmark and
    > it will become very clear why xrange exists.


    Or, when memory-constrained and this extra memory usage pushes
    your machine to pound on your swap...not a pretty sight.

    -tkc (from his box with a meager 96 megs of memory, though
    <geezer>back when I worked on a machine with a copious 48k of
    memory and enjoyed it...</geezer>)
     
    Tim Chase, Dec 7, 2007
    #4
  5. Joe Goldthwaite

    Carl Banks Guest

    On Dec 7, 4:08 pm, "Joe Goldthwaite" <> wrote:
    > Here's the simple benchmark;
    >
    > start = time.time()
    > for x in xrange(3):
    > for y in xrange(10000000):
    > pass
    > print 'xRange %s' % (time.time() - start)
    >
    > start = time.time()
    > for x in range(3):
    > for y in range(10000000):
    > pass
    > print 'Range %s' % (time.time() - start)
    >
    > Here's what I get;
    >
    > xRange 92.5529999733
    > Range 95.2669999599
    >
    > Not a lot of difference.



    Try this:

    start = time.time()
    for x in xrange(3):
    for y in xrange(10000000):
    if y == 10:
    break
    print 'xRange %s' % (time.time() - start)

    start = time.time()
    for x in range(3):
    for y in range(10000000):
    if y == 10:
    break
    print 'Range %s' % (time.time() - start)


    > Range is slower but not by much. I know that range
    > builds
    > a list then iterates through it. I thought that xrange just kept a counter
    > that was
    > incremented and returned with each call. No list was ever created. If that's
    > true
    > (and I guess it's not), xrange would be much faster than range. It seems
    > almost
    > identical.


    xrange doesn't merely return a counter. It creates an int object and
    returns it, which is somewhat more expensive. The time required to
    create ten million integer objects dwarfs the time required to
    allocate a single list with ten million entries, and this cost occurs
    with both range and xrange, unless you break early.

    > Given the amount of performance difference, I don't see why
    > xrange even
    > exists.


    To keep memory use down. It's pointless and wasteful to allocate
    about 160 megabytes (or 1 kilobyte) of data to store a list when you
    only need 36 or so of those bytes at any given time.



    Carl Banks
     
    Carl Banks, Dec 8, 2007
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steve R. Hastings

    efficiency of range() and xrange() in for loops

    Steve R. Hastings, Apr 5, 2006, in forum: Python
    Replies:
    29
    Views:
    795
    Fredrik Lundh
    Apr 9, 2006
  2. stdazi

    why I don't like range/xrange

    stdazi, Feb 16, 2007, in forum: Python
    Replies:
    17
    Views:
    546
    Bruno Desthuilliers
    Feb 18, 2007
  3. Joe Goldthwaite
    Replies:
    2
    Views:
    261
    Chris Mellon
    Dec 7, 2007
  4. harrismh777
    Replies:
    11
    Views:
    1,440
    Steven D'Aprano
    Aug 4, 2011
  5. Terry Reedy
    Replies:
    0
    Views:
    203
    Terry Reedy
    Nov 5, 2012
Loading...

Share This Page