Random string of digits?

R

Roy Smith

I want to create a string of 20 random digits (I'm OK with leading
zeros). The best I came up with is:

''.join(str(random.randint(0, 9)) for i in range(20))

Is there something better?
 
S

Steven D'Aprano

I want to create a string of 20 random digits (I'm OK with leading
zeros). The best I came up with is:

''.join(str(random.randint(0, 9)) for i in range(20))

Is there something better?

'%20d' % random.randint(0, 10**20-1)
 
C

Chris Angelico

I want to create a string of 20 random digits (I'm OK with leading
zeros).  The best I came up with is:

''.join(str(random.randint(0, 9)) for i in range(20))

Is there something better?

The simple option is:
random.randint(0,99999999999999999999)
or
"%020d"%random.randint(0,99999999999999999999)
(the latter gives you a string, padded with leading zeroes). But I'm
assuming that you discarded that option due to lack of entropy (ie you
can't trust randint() over that huge a range).

The way I'd do it would be in chunks. The simple option is one chunk;
your original technique is twenty. We can go somewhere in between.
First thing to do though: ascertain how far randint() is properly
random. The Python 2 docs [1] say that the underlying random()
function uses 53-bit floats, so you can probably rely on about that
much randomness; for argument's sake, let's say it's safe for up to
10,000 but no further (although 53 bits give you about 15 decimal
digits).

''.join('%04d'%random.randint(0,9999) for i in range(5))

For your actual task, I'd be inclined to take ten digits, twice, and
not bother with join():

'%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))

Looks a little ugly, but it works! And only two random number calls
(which can be expensive).

ChrisA
 
C

Chris Angelico

'%20d' % random.randint(0, 10**20-1)

I should mention that this sort of thing is absolutely acceptable in
situations where you don't actually need that level of randomness; if
you want to assign 20-digit numbers to each of ten million objects,
you can happily use something that's only really random to fifteen. I
use this technique with base 36 numbers to generate arbitrary
alphanumeric strings to use as message tags, for instance; fill out a
specified field size, even though not every combination will actually
be used.

ChrisA
 
R

Roy Smith

Chris Angelico said:
"%020d"%random.randint(0,99999999999999999999)
(the latter gives you a string, padded with leading zeroes). But I'm
assuming that you discarded that option due to lack of entropy (ie you
can't trust randint() over that huge a range).

Actually, the only entropy involved here is the ever increasing amount
of it between my ears. It never occurred to me to try that :)
For your actual task, I'd be inclined to take ten digits, twice, and
not bother with join():

'%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))

Looks a little ugly, but it works! And only two random number calls
(which can be expensive).

Hmmm. In my case, I was looking more to optimize clarity of code, not
speed. This is being used during account creation on a web site, so it
doesn't get run very often.

It turns out, I don't really need 20 digits. If I can count on

to give me 15-ish digits, that's good enough for my needs and I'll
probably go with that. Thanks.
 
C

Chris Angelico

It turns out, I don't really need 20 digits.  If I can count on


to give me 15-ish digits, that's good enough for my needs and I'll
probably go with that.  Thanks.

I'd say you can. The information about the Mersenne Twister underlying
the module's functions isn't marked as "CPython Implementation Note",
so I would expect that every Python implementation will provide a
minimum of 53-bit precision. (It does have the feeling of an
implementation detail though; is there a guarantee that two Pythons
will generate the same sequence of numbers from the same seed?)

ChrisA
 
P

Peter Otten

Chris said:
I want to create a string of 20 random digits (I'm OK with leading
zeros). The best I came up with is:

''.join(str(random.randint(0, 9)) for i in range(20))

Is there something better?

The simple option is:
random.randint(0,99999999999999999999)
or
"%020d"%random.randint(0,99999999999999999999)
(the latter gives you a string, padded with leading zeroes). But I'm
assuming that you discarded that option due to lack of entropy (ie you
can't trust randint() over that huge a range).

The way I'd do it would be in chunks. The simple option is one chunk;
your original technique is twenty. We can go somewhere in between.
First thing to do though: ascertain how far randint() is properly
random. The Python 2 docs [1] say that the underlying random()
function uses 53-bit floats, so you can probably rely on about that
much randomness; for argument's sake, let's say it's safe for up to
10,000 but no further (although 53 bits give you about 15 decimal
digits).

''.join('%04d'%random.randint(0,9999) for i in range(5))

For your actual task, I'd be inclined to take ten digits, twice, and
not bother with join():

'%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))

Looks a little ugly, but it works! And only two random number calls
(which can be expensive).

Judging from a quick look into the code (method Random._randbelow()) I'd say
you don't need to do that unless you override Random.random() and not
Random.getrandbits(). Even if you roll your own random() you'll get a
warning when you run into the limit:
.... def random(self): return 4 # *
..../usr/lib/python2.6/random.py:253: UserWarning: Underlying random() generator
does not supply
enough bits to choose from a population range this large
_warn("Underlying random() generator does not supply \n"
400000000000000000000L

(*) According to the literature 4 is the best random number, 9 being the
runner-up:
http://www.googlefight.com/index.php?lang=en_GB&word1=random+number+dilbert&word2=random+number+xkcd

A quick sanity check:
Counter({'9': 1060, '6': 1039, '3': 1036, '8': 1007, '7': 997, '4': 977,
'1': 976, '5': 976, '2': 970, '0': 962})
 
S

Steven D'Aprano

I'd say you can. The information about the Mersenne Twister underlying
the module's functions isn't marked as "CPython Implementation Note", so
I would expect that every Python implementation will provide a minimum
of 53-bit precision. (It does have the feeling of an implementation
detail though; is there a guarantee that two Pythons will generate the
same sequence of numbers from the same seed?)

Yes.

http://docs.python.org/dev/library/random.html#notes-on-reproducibility

I think that's a new guarantee. Previously, the answer was only yes-ish:
in theory, no guarantee was made, but in practice, you could normally
rely on it. For example, when the Mersenne Twister became the default
random number generator, the old generator, Wichman-Hill, was moved into
its own module whrandom (deprecated in 2.4; now gone) for those who
needed backwards compatibility.
 
C

Chris Angelico

Use the Source, Luke, er, Chris :)

If I've read the source correctly, randint() will generate sufficient
bits of randomness to ensure that the entire int is random.

http://hg.python.org/cpython/file/default/Lib/random.py

I prefer not to rely on the source. That tells me what happens, not
what's guaranteed to happen. However... bit of poking around can't
hurt. That file doesn't actually justify anything, because
random.Random() does not define getrandbits() - that, it seems, comes
from _random(); turns out that getrandbits is actually doing pretty
much the same thing I suggested:

http://hg.python.org/cpython/file/745f9fd9856d/Modules/_randommodule.c#l371

Need a 64-bit random number? Take two 32-bit numbers and concatenate.
So, it's going to be easier and clearer to just take the simple
option, since it's actually doing the same thing underneath anyway.

ChrisA
 
S

Steven D'Aprano

I prefer not to rely on the source. That tells me what happens, not
what's guaranteed to happen.

In this case, the source explicitly tells you that the API includes
support for arbitrary large ranges if you include a getrandbits() method:

Optionally, implement a getrandbits() method so that randrange()
can cover arbitrarily large ranges.

I call that a pretty strong guarantee.

However... bit of poking around can't hurt.
That file doesn't actually justify anything, because random.Random()
does not define getrandbits() - that, it seems, comes from _random();
turns out that getrandbits is actually doing pretty much the same thing
I suggested:

http://hg.python.org/cpython/file/745f9fd9856d/Modules/ _randommodule.c#l371

Need a 64-bit random number? Take two 32-bit numbers and concatenate.
So, it's going to be easier and clearer to just take the simple option,
since it's actually doing the same thing underneath anyway.

Um, I'm not sure what you consider "the simple option" in this context. I
would hope you mean to use the high level API of randint:

# need a random number with exactly 20 decimal digits
random.randint(10**20, 10**21-1)

rather than manually assembling a 20 digit number from smaller pieces.
 
R

Roy Smith

Steven D'Aprano said:
In this case, the source explicitly tells you that the API includes
support for arbitrary large ranges if you include a getrandbits() method:

Optionally, implement a getrandbits() method so that randrange()
can cover arbitrarily large ranges.

I call that a pretty strong guarantee.

I think you mis-understood Chris's point. The documentation is the
specification of how something behaves. If the documentation doesn't
say it, you can't rely on it. The user should never have to read the
source to know how to use a function, or what they can depend on. Now,
I'm not saying that reading the source isn't useful for a deeper
understanding, but it should be understood that any insights you glean
from doing that are strictly implementation details.

If you're saying that there are guarantees made by the implementation of
getrandbits() which are not documented, then one of two things are true:

1) It is intended that users can depend on that behavior, in which case
it's a bug in the docs, and the docs should be updated.

or

2) It is not intended that users can depend on that behavior, in which
case they would be foolish to do so.
 
8

88888 Dihedral

Roy Smithæ–¼ 2011å¹´12月26日星期一UTC+8上åˆ1時41分29秒寫é“:
I think you mis-understood Chris's point. The documentation is the
specification of how something behaves. If the documentation doesn't
say it, you can't rely on it. The user should never have to read the
source to know how to use a function, or what they can depend on. Now,
I'm not saying that reading the source isn't useful for a deeper
understanding, but it should be understood that any insights you glean
from doing that are strictly implementation details.

If you're saying that there are guarantees made by the implementation of
getrandbits() which are not documented, then one of two things are true:

1) It is intended that users can depend on that behavior, in which case
it's a bug in the docs, and the docs should be updated.

or

2) It is not intended that users can depend on that behavior, in which
case they would be foolish to do so.

Random bit generations for RSA2048 encoding and cryptography applications
in python is simple and elegant.
 
S

Steven D'Aprano

I think you mis-understood Chris's point.

And I'm afraid that you have missed my point. The above comment from the
source is from a docstring: it *is* public, official documentation. See
help(random.Random).

The documentation is the
specification of how something behaves. If the documentation doesn't
say it, you can't rely on it.

A nice platitude, but not true. Documentation is often incomplete or even
inaccurate. We rely on many things that aren't documented anywhere. For
example, we can rely on the fact that

x = 3917
print x+1

will print 3918, even though that specific fact isn't documented
anywhere. Nevertheless, we can absolutely bank on it -- if it happened to
do something else, we would report it as a bug and not expect to be told
"implementation detail, will not fix". We make a number of undocumented
assumptions:

* we assume that when the documentation talks about "adding" two
numbers, it means the standard mathematical definition of addition
and not some other meaning;

* we assume that the result of such addition must be *correct*,
without that assumption being guaranteed anywhere;

* we assume that addition of two ints will return an int, as opposed
to some other numerically equal value such as a float or Fraction;

* we assume that not only will it be an int, but it will be *exactly*
an int, and not a subclass of int;

and no doubt there are others.

And by the way, in case you think I'm being ridiculously pedantic,
consider the first assumption listed above: the standard mathematical
definition of addition. That assumption is violated for floats.

py> 0.7 + 0.1 == 0.8
False


It is rather amusing the trust we put in documentation, when
documentation can be changed just as easily as code. Just because the
Fine Manual says that x+1 performs addition today, doesn't mean that it
will continue to say the same thing tomorrow. So if you trust the
language designers not to arbitrarily change the documentation, why not
trust them not to arbitrarily change the code?

Hint: as far as I can tell, nowhere does Python promise that printing
3918 will show those exact digits, but it's a pretty safe bet that Python
won't suddenly change to displaying integers in balanced-ternary notation
instead of decimal.


The user should never have to read the
source to know how to use a function, or what they can depend on.

Well, that's one school of thought. Another school of thought is that
documentation and comments lie, and the only thing that you can trust is
the code itself.

At the end of the day, the only thing that matters is the mindset of the
developer(s) of the software. You must try to get into their head and
determine whether they are the sort of people whose API promises can be
believed.

Now,
I'm not saying that reading the source isn't useful for a deeper
understanding, but it should be understood that any insights you glean
from doing that are strictly implementation details.

I couldn't care less about the specific details of *how* getrandbits is
used to put together an arbitrarily large random number. That "how" is
what people mean when they talk about "mere implementation details".

But the fact that getrandbits is used by randrange (as opposed to how it
is used) is not a mere implementation detail, but a public part of the
interface.

If you're saying that there are guarantees made by the implementation of
getrandbits() which are not documented, then one of two things are true:

The implementation of getrandbits is not documented at all. You would
have to read the C source of the _random module to find out how the
default pseudo-random number generator produces random bits. But note the
leading underscore: _random is a private implementation detail.

However the existence and use of random.Random.getrandbits is public,
documented, and guaranteed.
 
C

Chris Angelico

The implementation of getrandbits is not documented at all. You would
have to read the C source of the _random module to find out how the
default pseudo-random number generator produces random bits. But note the
leading underscore: _random is a private implementation detail.

However the existence and use of random.Random.getrandbits is public,
documented, and guaranteed.

It's that last line where I find disagreement between documentation
and source code.

"""Class Random can also be subclassed if you want to use a different
basic generator of your own devising: in that case, override the
random(), seed(), getstate(), and setstate() methods. Optionally, a
new generator can supply a getrandbits() method — this allows
randrange() to produce selections over an arbitrarily large range."""

My reading of this is that, if I were to write my own generator, I
could provide that method and gain perfection. It's not until you dig
somewhat that you find out that the default generator actually does
provide getrandbits, meaning that the default randrange can actually
be used for arbitrarily large ranges. Yes, it IS documented, but in
separate places and somewhat as an afterthought; in the 2.x
documentation it's shown as "new in 2.4", which explains this.

Unfortunately my brain isn't really running on all cylinders at the
moment and I can't come up with a better wording, but is there some
way this could be stated up in the same top paragraph that explains
about 53-bit precision and 2**19937-1 period? That's where I got my
original information from; I didn't go down to the individual function
definitions, which is the only place that it's stated that the
Mersenne Twister does include getrandbits (and which assumes that
you've read the top section that states that Mersenne Twister is the
default implementation).

ChrisA
 
R

Roy Smith

Steven D'Aprano said:
And I'm afraid that you have missed my point. The above comment from the
source is from a docstring: it *is* public, official documentation. See
help(random.Random).

When you wrote, "the source explicitly tells you..." the natural
assumption I made was "something in the source that's not part of the
documentation" (i.e. some comment).
A nice platitude, but not true. Documentation is often incomplete or even
inaccurate.

Of course it is. Those things constitute doc bugs which need to get
fixed. The fact that the specification is flawed does not change the
fact that it *is* the specification.
 
C

Chris Angelico

Of course it is.  Those things constitute doc bugs which need to get
fixed.  The fact that the specification is flawed does not change the
fact that it *is* the specification.

Also, the specification is what can be trusted across multiple
implementations of Python - the source can't. I don't intend to get
the source code to five Pythons to ascertain whether getrandbits() is
provided in the default random.Random() of each of them. Now, in this
particular instance, it IS stated in the docs - just down in the
individual function descriptions, which I hadn't read - so I almost
certainly CAN trust that across all platforms. But in the general
case, I would not want to make firm statements on the basis of
implementation details.

ChrisA
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,009
Latest member
GidgetGamb

Latest Threads

Top