Long integers and ways around xrange

[Martin Manns]

Hi

As stated in the manual, xrange raises an OverflowError for
long integer parameters. Looking for a xrange like generator
for long integers, I found this in the manual
(http://docs.python.org/library/functions.html):

CPython implementation detail: xrange() is intended to be simple and
fast. Implementations may impose restrictions to achieve this. The C
implementation of Python restricts all arguments to native C longs
(“short†Python integers), and also requires that the number of
elements fit in a native C long. If a larger range is needed, an
alternate version can be crafted using the itertools module:
islice(count(start, step), (stop-start+step-1)//step).

However, count only accepts one parameter, so that this solution does
not work. Furthermore, islice only accepts positive values for step.

I came up with a solution that I have pasted below.

Is there a standard long integer replacement for xrange?
Do you have ideas for improving the code?

Best Regards

Martin

--------------------
File irange.py:


#! /usr/bin/env python
# -*- coding: utf-8 -*-

from itertools import count, imap, islice

def scount(n=0, step=1):
"""Count that supports a step attribute"""

return imap(lambda x: x * step + n, count())

def irange(start, stop=None, step=1):
"""Range for long integers

Usage: irange([start], stop, [step])

Parameters
----------
start: Integer, defaults to 0
stop: Integer
step: Integer, defaults to 1

Note on long integers
---------------------

Each of the three parameters can be long integers.
If stop < start: start(stop-start+step-1) // step) must be a short integer.
If stop > start: start(stop-start+step+1) // step) must be a short integer.

"""

if start is None:
raise TypeError, "range() integer argument expected, got NoneType"

if stop is None:
stop = start
start = 0

if step is None:
step = 1

if step > 0:
if stop < start:
return (_ for _ in [])
return islice(scount(start, step), (stop-start+step-1) // step)

elif step < 0:
if stop > start:
return (_ for _ in [])
return islice(scount(start, step), (stop-start+step+1) // step)

else:
raise ValueError, "irange() step argument must not be zero"





--------------------
File test_irange.py


#! /usr/bin/env python
# -*- coding: utf-8 -*-

from irange import scount, irange

# Unit test for irange.py (py.test)

class TestIrange(object):
'Unit test for MainGridBase'
__module__ = __name__

def setup_method(self, method):
pass

def test_irange(self):
test_values = [ \
[2],
[10000],
[-23],
(0, 1),
(0, 2),
(0, 10000),
(1, 2),
(100, 1000),
(4, 0),
(4, -1),
(2**65-512, 2**65),
(-1, 1),
(-100, 0),
(0, 1, 1),
(0, 2, 1),
(0, 10, 1),
(0, 100, 1),
(0, -1, 1),
(0, 2, 2),
(0, 10, 3),
(0, 100, 4),
(0, 10000000, 600000),
(0, 2**65, 2**60),
(1, 0, -1),
(10, 0, -1),
(12312, 0, -1),
(2**65, 0, -2**61),
(1, 0, -2),
(10, 0, -3),
(12312, 0, -4),
(2**65, 0, -2**60),
(2**67, 0, -2**67+1000),
(-10, 2, 1),
(-2, 10, 1),
(3, 100, 1),
(10000000-1000, 10000000, 1),
(2**65, 2**66, 2**63),
(-120, 2**65, 2**63),
(1, -2334234, -10000),
(1, 10, 100),
(10, 1, 100),
(1, 10, -100),
(1, 2, 2**65),
(1, -2, 2**65),
]

for val in test_values:
assert list(irange(*val)) == range(*val)

 

[Steven D'Aprano]

Hi

As stated in the manual, xrange raises an OverflowError for long integer
parameters. Looking for a xrange like generator for long integers, I
found this in the manual
(http://docs.python.org/library/functions.html):


However, count only accepts one parameter, so that this solution does
not work. Furthermore, islice only accepts positive values for step.

This should be reported as a documentation bug.

I came up with a solution that I have pasted below.

Is there a standard long integer replacement for xrange?

Not really. There are many ways of implementing it, and it really depends
on what you care most about: speed, memory consumption, simplicity,
economy of code, or something else. Chances are that only memory
consumption is critical for any real application.

This is probably the simplest you can get:

def myxrange(start, end, step):
n = start
while n < end:
yield n
n += step

Adding error checking, useful defaults, and support for negative step
values is left as a exercise *wink*

Do you have ideas for improving the code?

Only quibbles.

E.g. you set the module encoding to utf-8, but don't use any non-ASCII
characters.

In the lambda in scount, you use x for an argument which is an int not a
float. I would use i rather than x to make that clear.

As I said, quibbles.

 

[Martin Manns]

This should be reported as a documentation bug.

I added a bug report [issue7721].

Thanks for the code review.

Martin