# Random string of digits?

Discussion in 'Python' started by Roy Smith, Dec 25, 2011.

1. ### Roy SmithGuest

I want to create a string of 20 random digits (I'm OK with leading
zeros). The best I came up with is:

''.join(str(random.randint(0, 9)) for i in range(20))

Is there something better?

Roy Smith, Dec 25, 2011

2. ### Steven D'ApranoGuest

On Sun, 25 Dec 2011 08:30:46 -0500, Roy Smith wrote:

> I want to create a string of 20 random digits (I'm OK with leading
> zeros). The best I came up with is:
>
> ''.join(str(random.randint(0, 9)) for i in range(20))
>
> Is there something better?

'%20d' % random.randint(0, 10**20-1)

--
Steven

Steven D'Aprano, Dec 25, 2011

3. ### Chris AngelicoGuest

On Mon, Dec 26, 2011 at 12:30 AM, Roy Smith <> wrote:
> I want to create a string of 20 random digits (I'm OK with leading
> zeros).  The best I came up with is:
>
> ''.join(str(random.randint(0, 9)) for i in range(20))
>
> Is there something better?

The simple option is:
random.randint(0,99999999999999999999)
or
"%020d"%random.randint(0,99999999999999999999)
(the latter gives you a string, padded with leading zeroes). But I'm
assuming that you discarded that option due to lack of entropy (ie you
can't trust randint() over that huge a range).

The way I'd do it would be in chunks. The simple option is one chunk;
your original technique is twenty. We can go somewhere in between.
First thing to do though: ascertain how far randint() is properly
random. The Python 2 docs [1] say that the underlying random()
function uses 53-bit floats, so you can probably rely on about that
much randomness; for argument's sake, let's say it's safe for up to
10,000 but no further (although 53 bits give you about 15 decimal
digits).

''.join('%04d'%random.randint(0,9999) for i in range(5))

For your actual task, I'd be inclined to take ten digits, twice, and
not bother with join():

'%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))

Looks a little ugly, but it works! And only two random number calls
(which can be expensive).

ChrisA

Chris Angelico, Dec 25, 2011
4. ### Chris AngelicoGuest

Chris Angelico, Dec 25, 2011
5. ### Chris AngelicoGuest

On Mon, Dec 26, 2011 at 12:48 AM, Steven D'Aprano
<> wrote:
> On Sun, 25 Dec 2011 08:30:46 -0500, Roy Smith wrote:
>
>> I want to create a string of 20 random digits (I'm OK with leading
>> zeros).  The best I came up with is:
>>
>> ''.join(str(random.randint(0, 9)) for i in range(20))
>>
>> Is there something better?

>
> '%20d' % random.randint(0, 10**20-1)

I should mention that this sort of thing is absolutely acceptable in
situations where you don't actually need that level of randomness; if
you want to assign 20-digit numbers to each of ten million objects,
you can happily use something that's only really random to fifteen. I
use this technique with base 36 numbers to generate arbitrary
alphanumeric strings to use as message tags, for instance; fill out a
specified field size, even though not every combination will actually
be used.

ChrisA

Chris Angelico, Dec 25, 2011
6. ### Roy SmithGuest

In article <>,
Chris Angelico <> wrote:

> "%020d"%random.randint(0,99999999999999999999)
> (the latter gives you a string, padded with leading zeroes). But I'm
> assuming that you discarded that option due to lack of entropy (ie you
> can't trust randint() over that huge a range).

Actually, the only entropy involved here is the ever increasing amount
of it between my ears. It never occurred to me to try that

> For your actual task, I'd be inclined to take ten digits, twice, and
> not bother with join():
>
> '%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))
>
> Looks a little ugly, but it works! And only two random number calls
> (which can be expensive).

Hmmm. In my case, I was looking more to optimize clarity of code, not
speed. This is being used during account creation on a web site, so it
doesn't get run very often.

It turns out, I don't really need 20 digits. If I can count on

>>> "%020d" % random.randint(0,999999999999999)

to give me 15-ish digits, that's good enough for my needs and I'll
probably go with that. Thanks.

Roy Smith, Dec 25, 2011
7. ### Chris AngelicoGuest

On Mon, Dec 26, 2011 at 1:21 AM, Roy Smith <> wrote:
> It turns out, I don't really need 20 digits.  If I can count on
>
>>>> "%020d" % random.randint(0,999999999999999)

>
> to give me 15-ish digits, that's good enough for my needs and I'll
> probably go with that.  Thanks.

I'd say you can. The information about the Mersenne Twister underlying
the module's functions isn't marked as "CPython Implementation Note",
so I would expect that every Python implementation will provide a
minimum of 53-bit precision. (It does have the feeling of an
implementation detail though; is there a guarantee that two Pythons
will generate the same sequence of numbers from the same seed?)

ChrisA

Chris Angelico, Dec 25, 2011
8. ### Peter OttenGuest

Chris Angelico wrote:

> On Mon, Dec 26, 2011 at 12:30 AM, Roy Smith <> wrote:
>> I want to create a string of 20 random digits (I'm OK with leading
>> zeros). The best I came up with is:
>>
>> ''.join(str(random.randint(0, 9)) for i in range(20))
>>
>> Is there something better?

>
> The simple option is:
> random.randint(0,99999999999999999999)
> or
> "%020d"%random.randint(0,99999999999999999999)
> (the latter gives you a string, padded with leading zeroes). But I'm
> assuming that you discarded that option due to lack of entropy (ie you
> can't trust randint() over that huge a range).
>
> The way I'd do it would be in chunks. The simple option is one chunk;
> your original technique is twenty. We can go somewhere in between.
> First thing to do though: ascertain how far randint() is properly
> random. The Python 2 docs [1] say that the underlying random()
> function uses 53-bit floats, so you can probably rely on about that
> much randomness; for argument's sake, let's say it's safe for up to
> 10,000 but no further (although 53 bits give you about 15 decimal
> digits).
>
> ''.join('%04d'%random.randint(0,9999) for i in range(5))
>
> For your actual task, I'd be inclined to take ten digits, twice, and
> not bother with join():
>
> '%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))
>
> Looks a little ugly, but it works! And only two random number calls
> (which can be expensive).

Judging from a quick look into the code (method Random._randbelow()) I'd say
you don't need to do that unless you override Random.random() and not
Random.getrandbits(). Even if you roll your own random() you'll get a
warning when you run into the limit:

>>> import random
>>> random.randrange(10**20)

27709407700486201379L
>>> class R(random.Random):

.... def random(self): return 4 # *
....
>>> R().randrange(10**20)

/usr/lib/python2.6/random.py:253: UserWarning: Underlying random() generator
does not supply
enough bits to choose from a population range this large
_warn("Underlying random() generator does not supply \n"
400000000000000000000L

(*) According to the literature 4 is the best random number, 9 being the
runner-up:

A quick sanity check:

>>> from collections import Counter
>>> import random
>>> Counter(str(random.randrange(10**10000)))

Counter({'9': 1060, '6': 1039, '3': 1036, '8': 1007, '7': 997, '4': 977,
'1': 976, '5': 976, '2': 970, '0': 962})

Peter Otten, Dec 25, 2011
9. ### Steven D'ApranoGuest

On Mon, 26 Dec 2011 01:51:30 +1100, Chris Angelico wrote:

> On Mon, Dec 26, 2011 at 1:21 AM, Roy Smith <> wrote:
>> It turns out, I don't really need 20 digits. Â If I can count on
>>
>>>>> "%020d" % random.randint(0,999999999999999)

>>
>> to give me 15-ish digits, that's good enough for my needs and I'll
>> probably go with that. Â Thanks.

>
> I'd say you can. The information about the Mersenne Twister underlying
> the module's functions isn't marked as "CPython Implementation Note", so
> I would expect that every Python implementation will provide a minimum
> of 53-bit precision. (It does have the feeling of an implementation
> detail though; is there a guarantee that two Pythons will generate the
> same sequence of numbers from the same seed?)

Yes.

http://docs.python.org/dev/library/random.html#notes-on-reproducibility

I think that's a new guarantee. Previously, the answer was only yes-ish:
in theory, no guarantee was made, but in practice, you could normally
rely on it. For example, when the Mersenne Twister became the default
random number generator, the old generator, Wichman-Hill, was moved into
its own module whrandom (deprecated in 2.4; now gone) for those who
needed backwards compatibility.

--
Steven

Steven D'Aprano, Dec 25, 2011
10. ### Steven D'ApranoGuest

On Mon, 26 Dec 2011 00:54:40 +1100, Chris Angelico wrote:

> On Mon, Dec 26, 2011 at 12:48 AM, Steven D'Aprano
> <> wrote:
>> On Sun, 25 Dec 2011 08:30:46 -0500, Roy Smith wrote:
>>
>>> I want to create a string of 20 random digits (I'm OK with leading
>>> zeros). Â The best I came up with is:
>>>
>>> ''.join(str(random.randint(0, 9)) for i in range(20))
>>>
>>> Is there something better?

>>
>> '%20d' % random.randint(0, 10**20-1)

>
> I should mention that this sort of thing is absolutely acceptable in
> situations where you don't actually need that level of randomness;

Use the Source, Luke, er, Chris

If I've read the source correctly, randint() will generate sufficient
bits of randomness to ensure that the entire int is random.

http://hg.python.org/cpython/file/default/Lib/random.py

--
Steven

Steven D'Aprano, Dec 25, 2011
11. ### Chris AngelicoGuest

On Mon, Dec 26, 2011 at 2:46 AM, Steven D'Aprano
<> wrote:
> Use the Source, Luke, er, Chris
>
> If I've read the source correctly, randint() will generate sufficient
> bits of randomness to ensure that the entire int is random.
>
> http://hg.python.org/cpython/file/default/Lib/random.py

I prefer not to rely on the source. That tells me what happens, not
what's guaranteed to happen. However... bit of poking around can't
hurt. That file doesn't actually justify anything, because
random.Random() does not define getrandbits() - that, it seems, comes
from _random(); turns out that getrandbits is actually doing pretty
much the same thing I suggested:

http://hg.python.org/cpython/file/745f9fd9856d/Modules/_randommodule.c#l371

Need a 64-bit random number? Take two 32-bit numbers and concatenate.
So, it's going to be easier and clearer to just take the simple
option, since it's actually doing the same thing underneath anyway.

ChrisA

Chris Angelico, Dec 25, 2011
12. ### Steven D'ApranoGuest

On Mon, 26 Dec 2011 03:11:56 +1100, Chris Angelico wrote:

> On Mon, Dec 26, 2011 at 2:46 AM, Steven D'Aprano
> <> wrote:
>> Use the Source, Luke, er, Chris
>>
>> If I've read the source correctly, randint() will generate sufficient
>> bits of randomness to ensure that the entire int is random.
>>
>> http://hg.python.org/cpython/file/default/Lib/random.py

>
> I prefer not to rely on the source. That tells me what happens, not
> what's guaranteed to happen.

In this case, the source explicitly tells you that the API includes
support for arbitrary large ranges if you include a getrandbits() method:

Optionally, implement a getrandbits() method so that randrange()
can cover arbitrarily large ranges.

I call that a pretty strong guarantee.

> However... bit of poking around can't hurt.
> That file doesn't actually justify anything, because random.Random()
> does not define getrandbits() - that, it seems, comes from _random();
> turns out that getrandbits is actually doing pretty much the same thing
> I suggested:
>
> http://hg.python.org/cpython/file/745f9fd9856d/Modules/

_randommodule.c#l371
>
> Need a 64-bit random number? Take two 32-bit numbers and concatenate.
> So, it's going to be easier and clearer to just take the simple option,
> since it's actually doing the same thing underneath anyway.

Um, I'm not sure what you consider "the simple option" in this context. I
would hope you mean to use the high level API of randint:

# need a random number with exactly 20 decimal digits
random.randint(10**20, 10**21-1)

rather than manually assembling a 20 digit number from smaller pieces.

--
Steven

Steven D'Aprano, Dec 25, 2011
13. ### Serhiy StorchakaGuest

25.12.11 15:48, Steven D'Aprano Ð½Ð°Ð¿Ð¸ÑÐ°Ð²(Ð»Ð°):
> On Sun, 25 Dec 2011 08:30:46 -0500, Roy Smith wrote:
>> I want to create a string of 20 random digits (I'm OK with leading
>> zeros). The best I came up with is:
>> ''.join(str(random.randint(0, 9)) for i in range(20))
>> Is there something better?

> '%20d' % random.randint(0, 10**20-1)

'%020d' % random.randrange(10**20)

Serhiy Storchaka, Dec 25, 2011
14. ### Roy SmithGuest

On Mon, 26 Dec 2011 03:11:56 +1100, Chris Angelico wrote:
> > I prefer not to rely on the source. That tells me what happens, not
> > what's guaranteed to happen.

Steven D'Aprano <> wrote:
> In this case, the source explicitly tells you that the API includes
> support for arbitrary large ranges if you include a getrandbits() method:
>
> Optionally, implement a getrandbits() method so that randrange()
> can cover arbitrarily large ranges.
>
> I call that a pretty strong guarantee.

I think you mis-understood Chris's point. The documentation is the
specification of how something behaves. If the documentation doesn't
say it, you can't rely on it. The user should never have to read the
source to know how to use a function, or what they can depend on. Now,
I'm not saying that reading the source isn't useful for a deeper
understanding, but it should be understood that any insights you glean
from doing that are strictly implementation details.

If you're saying that there are guarantees made by the implementation of
getrandbits() which are not documented, then one of two things are true:

1) It is intended that users can depend on that behavior, in which case
it's a bug in the docs, and the docs should be updated.

or

2) It is not intended that users can depend on that behavior, in which
case they would be foolish to do so.

Roy Smith, Dec 25, 2011
15. ### 88888 DihedralGuest

Roy Smithæ–¼ 2011å¹´12æœˆ26æ—¥æ˜ŸæœŸä¸€UTC+8ä¸Šåˆ1æ™‚41åˆ†29ç§’å¯«é“ï¼š
> On Mon, 26 Dec 2011 03:11:56 +1100, Chris Angelico wrote:
> > > I prefer not to rely on the source. That tells me what happens, not
> > > what's guaranteed to happen.

>
> Steven D'Aprano <> wrote:
> > In this case, the source explicitly tells you that the API includes
> > support for arbitrary large ranges if you include a getrandbits() method:
> >
> > Optionally, implement a getrandbits() method so that randrange()
> > can cover arbitrarily large ranges.
> >
> > I call that a pretty strong guarantee.

>
> I think you mis-understood Chris's point. The documentation is the
> specification of how something behaves. If the documentation doesn't
> say it, you can't rely on it. The user should never have to read the
> source to know how to use a function, or what they can depend on. Now,
> I'm not saying that reading the source isn't useful for a deeper
> understanding, but it should be understood that any insights you glean
> from doing that are strictly implementation details.
>
> If you're saying that there are guarantees made by the implementation of
> getrandbits() which are not documented, then one of two things are true:
>
> 1) It is intended that users can depend on that behavior, in which case
> it's a bug in the docs, and the docs should be updated.
>
> or
>
> 2) It is not intended that users can depend on that behavior, in which
> case they would be foolish to do so.

Random bit generations for RSA2048 encoding and cryptography applications
in python is simple and elegant.

88888 Dihedral, Dec 25, 2011
16. ### Steven D'ApranoGuest

On Sun, 25 Dec 2011 12:41:29 -0500, Roy Smith wrote:

> On Mon, 26 Dec 2011 03:11:56 +1100, Chris Angelico wrote:
>> > I prefer not to rely on the source. That tells me what happens, not
>> > what's guaranteed to happen.

>
> Steven D'Aprano <> wrote:
>> In this case, the source explicitly tells you that the API includes
>> support for arbitrary large ranges if you include a getrandbits()
>> method:
>>
>> Optionally, implement a getrandbits() method so that randrange()
>> can cover arbitrarily large ranges.
>>
>> I call that a pretty strong guarantee.

>
> I think you mis-understood Chris's point.

And I'm afraid that you have missed my point. The above comment from the
source is from a docstring: it *is* public, official documentation. See
help(random.Random).

> The documentation is the
> specification of how something behaves. If the documentation doesn't
> say it, you can't rely on it.

A nice platitude, but not true. Documentation is often incomplete or even
inaccurate. We rely on many things that aren't documented anywhere. For
example, we can rely on the fact that

x = 3917
print x+1

will print 3918, even though that specific fact isn't documented
anywhere. Nevertheless, we can absolutely bank on it -- if it happened to
do something else, we would report it as a bug and not expect to be told
"implementation detail, will not fix". We make a number of undocumented
assumptions:

numbers, it means the standard mathematical definition of addition
and not some other meaning;

* we assume that the result of such addition must be *correct*,
without that assumption being guaranteed anywhere;

* we assume that addition of two ints will return an int, as opposed
to some other numerically equal value such as a float or Fraction;

* we assume that not only will it be an int, but it will be *exactly*
an int, and not a subclass of int;

and no doubt there are others.

And by the way, in case you think I'm being ridiculously pedantic,
consider the first assumption listed above: the standard mathematical
definition of addition. That assumption is violated for floats.

py> 0.7 + 0.1 == 0.8
False

It is rather amusing the trust we put in documentation, when
documentation can be changed just as easily as code. Just because the
Fine Manual says that x+1 performs addition today, doesn't mean that it
will continue to say the same thing tomorrow. So if you trust the
language designers not to arbitrarily change the documentation, why not
trust them not to arbitrarily change the code?

Hint: as far as I can tell, nowhere does Python promise that printing
3918 will show those exact digits, but it's a pretty safe bet that Python
won't suddenly change to displaying integers in balanced-ternary notation

> The user should never have to read the
> source to know how to use a function, or what they can depend on.

Well, that's one school of thought. Another school of thought is that
documentation and comments lie, and the only thing that you can trust is
the code itself.

At the end of the day, the only thing that matters is the mindset of the
developer(s) of the software. You must try to get into their head and
determine whether they are the sort of people whose API promises can be
believed.

> Now,
> I'm not saying that reading the source isn't useful for a deeper
> understanding, but it should be understood that any insights you glean
> from doing that are strictly implementation details.

I couldn't care less about the specific details of *how* getrandbits is
used to put together an arbitrarily large random number. That "how" is
what people mean when they talk about "mere implementation details".

But the fact that getrandbits is used by randrange (as opposed to how it
is used) is not a mere implementation detail, but a public part of the
interface.

> If you're saying that there are guarantees made by the implementation of
> getrandbits() which are not documented, then one of two things are true:

The implementation of getrandbits is not documented at all. You would
have to read the C source of the _random module to find out how the
default pseudo-random number generator produces random bits. But note the
leading underscore: _random is a private implementation detail.

However the existence and use of random.Random.getrandbits is public,
documented, and guaranteed.

--
Steven

Steven D'Aprano, Dec 26, 2011
17. ### Chris AngelicoGuest

On Mon, Dec 26, 2011 at 2:00 PM, Steven D'Aprano
<> wrote:
> The implementation of getrandbits is not documented at all. You would
> have to read the C source of the _random module to find out how the
> default pseudo-random number generator produces random bits. But note the
> leading underscore: _random is a private implementation detail.
>
> However the existence and use of random.Random.getrandbits is public,
> documented, and guaranteed.

It's that last line where I find disagreement between documentation
and source code.

"""Class Random can also be subclassed if you want to use a different
basic generator of your own devising: in that case, override the
random(), seed(), getstate(), and setstate() methods. Optionally, a
new generator can supply a getrandbits() method — this allows
randrange() to produce selections over an arbitrarily large range."""

My reading of this is that, if I were to write my own generator, I
could provide that method and gain perfection. It's not until you dig
somewhat that you find out that the default generator actually does
provide getrandbits, meaning that the default randrange can actually
be used for arbitrarily large ranges. Yes, it IS documented, but in
separate places and somewhat as an afterthought; in the 2.x
documentation it's shown as "new in 2.4", which explains this.

Unfortunately my brain isn't really running on all cylinders at the
moment and I can't come up with a better wording, but is there some
way this could be stated up in the same top paragraph that explains
about 53-bit precision and 2**19937-1 period? That's where I got my
original information from; I didn't go down to the individual function
definitions, which is the only place that it's stated that the
Mersenne Twister does include getrandbits (and which assumes that
you've read the top section that states that Mersenne Twister is the
default implementation).

ChrisA

Chris Angelico, Dec 26, 2011
18. ### Roy SmithGuest

In article <4ef7e337\$0\$29973\$c3e8da3\$>,
Steven D'Aprano <> wrote:

> And I'm afraid that you have missed my point. The above comment from the
> source is from a docstring: it *is* public, official documentation. See
> help(random.Random).

When you wrote, "the source explicitly tells you..." the natural
assumption I made was "something in the source that's not part of the
documentation" (i.e. some comment).

> > The documentation is the specification of how something behaves.
> > If the documentation doesn't say it, you can't rely on it.

> A nice platitude, but not true. Documentation is often incomplete or even
> inaccurate.

Of course it is. Those things constitute doc bugs which need to get
fixed. The fact that the specification is flawed does not change the
fact that it *is* the specification.

Roy Smith, Dec 26, 2011
19. ### Chris AngelicoGuest

On Mon, Dec 26, 2011 at 3:17 PM, Roy Smith <> wrote:
> Of course it is.  Those things constitute doc bugs which need to get
> fixed.  The fact that the specification is flawed does not change the
> fact that it *is* the specification.

Also, the specification is what can be trusted across multiple
implementations of Python - the source can't. I don't intend to get
the source code to five Pythons to ascertain whether getrandbits() is
provided in the default random.Random() of each of them. Now, in this
particular instance, it IS stated in the docs - just down in the