Random string of digits?

Discussion in 'Python' started by Roy Smith, Dec 25, 2011.

  1. Roy Smith

    Roy Smith Guest

    I want to create a string of 20 random digits (I'm OK with leading
    zeros). The best I came up with is:

    ''.join(str(random.randint(0, 9)) for i in range(20))

    Is there something better?
    Roy Smith, Dec 25, 2011
    #1
    1. Advertising

  2. On Sun, 25 Dec 2011 08:30:46 -0500, Roy Smith wrote:

    > I want to create a string of 20 random digits (I'm OK with leading
    > zeros). The best I came up with is:
    >
    > ''.join(str(random.randint(0, 9)) for i in range(20))
    >
    > Is there something better?


    '%20d' % random.randint(0, 10**20-1)


    --
    Steven
    Steven D'Aprano, Dec 25, 2011
    #2
    1. Advertising

  3. On Mon, Dec 26, 2011 at 12:30 AM, Roy Smith <> wrote:
    > I want to create a string of 20 random digits (I'm OK with leading
    > zeros).  The best I came up with is:
    >
    > ''.join(str(random.randint(0, 9)) for i in range(20))
    >
    > Is there something better?


    The simple option is:
    random.randint(0,99999999999999999999)
    or
    "%020d"%random.randint(0,99999999999999999999)
    (the latter gives you a string, padded with leading zeroes). But I'm
    assuming that you discarded that option due to lack of entropy (ie you
    can't trust randint() over that huge a range).

    The way I'd do it would be in chunks. The simple option is one chunk;
    your original technique is twenty. We can go somewhere in between.
    First thing to do though: ascertain how far randint() is properly
    random. The Python 2 docs [1] say that the underlying random()
    function uses 53-bit floats, so you can probably rely on about that
    much randomness; for argument's sake, let's say it's safe for up to
    10,000 but no further (although 53 bits give you about 15 decimal
    digits).

    ''.join('%04d'%random.randint(0,9999) for i in range(5))

    For your actual task, I'd be inclined to take ten digits, twice, and
    not bother with join():

    '%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))

    Looks a little ugly, but it works! And only two random number calls
    (which can be expensive).

    ChrisA
    Chris Angelico, Dec 25, 2011
    #3
  4. Chris Angelico, Dec 25, 2011
    #4
  5. On Mon, Dec 26, 2011 at 12:48 AM, Steven D'Aprano
    <> wrote:
    > On Sun, 25 Dec 2011 08:30:46 -0500, Roy Smith wrote:
    >
    >> I want to create a string of 20 random digits (I'm OK with leading
    >> zeros).  The best I came up with is:
    >>
    >> ''.join(str(random.randint(0, 9)) for i in range(20))
    >>
    >> Is there something better?

    >
    > '%20d' % random.randint(0, 10**20-1)


    I should mention that this sort of thing is absolutely acceptable in
    situations where you don't actually need that level of randomness; if
    you want to assign 20-digit numbers to each of ten million objects,
    you can happily use something that's only really random to fifteen. I
    use this technique with base 36 numbers to generate arbitrary
    alphanumeric strings to use as message tags, for instance; fill out a
    specified field size, even though not every combination will actually
    be used.

    ChrisA
    Chris Angelico, Dec 25, 2011
    #5
  6. Roy Smith

    Roy Smith Guest

    In article <>,
    Chris Angelico <> wrote:

    > "%020d"%random.randint(0,99999999999999999999)
    > (the latter gives you a string, padded with leading zeroes). But I'm
    > assuming that you discarded that option due to lack of entropy (ie you
    > can't trust randint() over that huge a range).


    Actually, the only entropy involved here is the ever increasing amount
    of it between my ears. It never occurred to me to try that :)

    > For your actual task, I'd be inclined to take ten digits, twice, and
    > not bother with join():
    >
    > '%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))
    >
    > Looks a little ugly, but it works! And only two random number calls
    > (which can be expensive).


    Hmmm. In my case, I was looking more to optimize clarity of code, not
    speed. This is being used during account creation on a web site, so it
    doesn't get run very often.

    It turns out, I don't really need 20 digits. If I can count on

    >>> "%020d" % random.randint(0,999999999999999)


    to give me 15-ish digits, that's good enough for my needs and I'll
    probably go with that. Thanks.
    Roy Smith, Dec 25, 2011
    #6
  7. On Mon, Dec 26, 2011 at 1:21 AM, Roy Smith <> wrote:
    > It turns out, I don't really need 20 digits.  If I can count on
    >
    >>>> "%020d" % random.randint(0,999999999999999)

    >
    > to give me 15-ish digits, that's good enough for my needs and I'll
    > probably go with that.  Thanks.


    I'd say you can. The information about the Mersenne Twister underlying
    the module's functions isn't marked as "CPython Implementation Note",
    so I would expect that every Python implementation will provide a
    minimum of 53-bit precision. (It does have the feeling of an
    implementation detail though; is there a guarantee that two Pythons
    will generate the same sequence of numbers from the same seed?)

    ChrisA
    Chris Angelico, Dec 25, 2011
    #7
  8. Roy Smith

    Peter Otten Guest

    Chris Angelico wrote:

    > On Mon, Dec 26, 2011 at 12:30 AM, Roy Smith <> wrote:
    >> I want to create a string of 20 random digits (I'm OK with leading
    >> zeros). The best I came up with is:
    >>
    >> ''.join(str(random.randint(0, 9)) for i in range(20))
    >>
    >> Is there something better?

    >
    > The simple option is:
    > random.randint(0,99999999999999999999)
    > or
    > "%020d"%random.randint(0,99999999999999999999)
    > (the latter gives you a string, padded with leading zeroes). But I'm
    > assuming that you discarded that option due to lack of entropy (ie you
    > can't trust randint() over that huge a range).
    >
    > The way I'd do it would be in chunks. The simple option is one chunk;
    > your original technique is twenty. We can go somewhere in between.
    > First thing to do though: ascertain how far randint() is properly
    > random. The Python 2 docs [1] say that the underlying random()
    > function uses 53-bit floats, so you can probably rely on about that
    > much randomness; for argument's sake, let's say it's safe for up to
    > 10,000 but no further (although 53 bits give you about 15 decimal
    > digits).
    >
    > ''.join('%04d'%random.randint(0,9999) for i in range(5))
    >
    > For your actual task, I'd be inclined to take ten digits, twice, and
    > not bother with join():
    >
    > '%010d%010d'%(random.randint(0,9999999999),random.randint(0,9999999999))
    >
    > Looks a little ugly, but it works! And only two random number calls
    > (which can be expensive).


    Judging from a quick look into the code (method Random._randbelow()) I'd say
    you don't need to do that unless you override Random.random() and not
    Random.getrandbits(). Even if you roll your own random() you'll get a
    warning when you run into the limit:

    >>> import random
    >>> random.randrange(10**20)

    27709407700486201379L
    >>> class R(random.Random):

    .... def random(self): return 4 # *
    ....
    >>> R().randrange(10**20)

    /usr/lib/python2.6/random.py:253: UserWarning: Underlying random() generator
    does not supply
    enough bits to choose from a population range this large
    _warn("Underlying random() generator does not supply \n"
    400000000000000000000L

    (*) According to the literature 4 is the best random number, 9 being the
    runner-up:
    http://www.googlefight.com/index.php?lang=en_GB&word1=random number dilbert&word2=random number xkcd

    A quick sanity check:

    >>> from collections import Counter
    >>> import random
    >>> Counter(str(random.randrange(10**10000)))

    Counter({'9': 1060, '6': 1039, '3': 1036, '8': 1007, '7': 997, '4': 977,
    '1': 976, '5': 976, '2': 970, '0': 962})
    Peter Otten, Dec 25, 2011
    #8
  9. On Mon, 26 Dec 2011 01:51:30 +1100, Chris Angelico wrote:

    > On Mon, Dec 26, 2011 at 1:21 AM, Roy Smith <> wrote:
    >> It turns out, I don't really need 20 digits.  If I can count on
    >>
    >>>>> "%020d" % random.randint(0,999999999999999)

    >>
    >> to give me 15-ish digits, that's good enough for my needs and I'll
    >> probably go with that.  Thanks.

    >
    > I'd say you can. The information about the Mersenne Twister underlying
    > the module's functions isn't marked as "CPython Implementation Note", so
    > I would expect that every Python implementation will provide a minimum
    > of 53-bit precision. (It does have the feeling of an implementation
    > detail though; is there a guarantee that two Pythons will generate the
    > same sequence of numbers from the same seed?)


    Yes.

    http://docs.python.org/dev/library/random.html#notes-on-reproducibility

    I think that's a new guarantee. Previously, the answer was only yes-ish:
    in theory, no guarantee was made, but in practice, you could normally
    rely on it. For example, when the Mersenne Twister became the default
    random number generator, the old generator, Wichman-Hill, was moved into
    its own module whrandom (deprecated in 2.4; now gone) for those who
    needed backwards compatibility.


    --
    Steven
    Steven D'Aprano, Dec 25, 2011
    #9
  10. On Mon, 26 Dec 2011 00:54:40 +1100, Chris Angelico wrote:

    > On Mon, Dec 26, 2011 at 12:48 AM, Steven D'Aprano
    > <> wrote:
    >> On Sun, 25 Dec 2011 08:30:46 -0500, Roy Smith wrote:
    >>
    >>> I want to create a string of 20 random digits (I'm OK with leading
    >>> zeros).  The best I came up with is:
    >>>
    >>> ''.join(str(random.randint(0, 9)) for i in range(20))
    >>>
    >>> Is there something better?

    >>
    >> '%20d' % random.randint(0, 10**20-1)

    >
    > I should mention that this sort of thing is absolutely acceptable in
    > situations where you don't actually need that level of randomness;


    Use the Source, Luke, er, Chris :)

    If I've read the source correctly, randint() will generate sufficient
    bits of randomness to ensure that the entire int is random.

    http://hg.python.org/cpython/file/default/Lib/random.py



    --
    Steven
    Steven D'Aprano, Dec 25, 2011
    #10
  11. On Mon, Dec 26, 2011 at 2:46 AM, Steven D'Aprano
    <> wrote:
    > Use the Source, Luke, er, Chris :)
    >
    > If I've read the source correctly, randint() will generate sufficient
    > bits of randomness to ensure that the entire int is random.
    >
    > http://hg.python.org/cpython/file/default/Lib/random.py


    I prefer not to rely on the source. That tells me what happens, not
    what's guaranteed to happen. However... bit of poking around can't
    hurt. That file doesn't actually justify anything, because
    random.Random() does not define getrandbits() - that, it seems, comes
    from _random(); turns out that getrandbits is actually doing pretty
    much the same thing I suggested:

    http://hg.python.org/cpython/file/745f9fd9856d/Modules/_randommodule.c#l371

    Need a 64-bit random number? Take two 32-bit numbers and concatenate.
    So, it's going to be easier and clearer to just take the simple
    option, since it's actually doing the same thing underneath anyway.

    ChrisA
    Chris Angelico, Dec 25, 2011
    #11
  12. On Mon, 26 Dec 2011 03:11:56 +1100, Chris Angelico wrote:

    > On Mon, Dec 26, 2011 at 2:46 AM, Steven D'Aprano
    > <> wrote:
    >> Use the Source, Luke, er, Chris :)
    >>
    >> If I've read the source correctly, randint() will generate sufficient
    >> bits of randomness to ensure that the entire int is random.
    >>
    >> http://hg.python.org/cpython/file/default/Lib/random.py

    >
    > I prefer not to rely on the source. That tells me what happens, not
    > what's guaranteed to happen.


    In this case, the source explicitly tells you that the API includes
    support for arbitrary large ranges if you include a getrandbits() method:

    Optionally, implement a getrandbits() method so that randrange()
    can cover arbitrarily large ranges.

    I call that a pretty strong guarantee.


    > However... bit of poking around can't hurt.
    > That file doesn't actually justify anything, because random.Random()
    > does not define getrandbits() - that, it seems, comes from _random();
    > turns out that getrandbits is actually doing pretty much the same thing
    > I suggested:
    >
    > http://hg.python.org/cpython/file/745f9fd9856d/Modules/

    _randommodule.c#l371
    >
    > Need a 64-bit random number? Take two 32-bit numbers and concatenate.
    > So, it's going to be easier and clearer to just take the simple option,
    > since it's actually doing the same thing underneath anyway.


    Um, I'm not sure what you consider "the simple option" in this context. I
    would hope you mean to use the high level API of randint:

    # need a random number with exactly 20 decimal digits
    random.randint(10**20, 10**21-1)

    rather than manually assembling a 20 digit number from smaller pieces.


    --
    Steven
    Steven D'Aprano, Dec 25, 2011
    #12
  13. 25.12.11 15:48, Steven D'Aprano напиÑав(ла):
    > On Sun, 25 Dec 2011 08:30:46 -0500, Roy Smith wrote:
    >> I want to create a string of 20 random digits (I'm OK with leading
    >> zeros). The best I came up with is:
    >> ''.join(str(random.randint(0, 9)) for i in range(20))
    >> Is there something better?

    > '%20d' % random.randint(0, 10**20-1)


    '%020d' % random.randrange(10**20)
    Serhiy Storchaka, Dec 25, 2011
    #13
  14. Roy Smith

    Roy Smith Guest

    On Mon, 26 Dec 2011 03:11:56 +1100, Chris Angelico wrote:
    > > I prefer not to rely on the source. That tells me what happens, not
    > > what's guaranteed to happen.


    Steven D'Aprano <> wrote:
    > In this case, the source explicitly tells you that the API includes
    > support for arbitrary large ranges if you include a getrandbits() method:
    >
    > Optionally, implement a getrandbits() method so that randrange()
    > can cover arbitrarily large ranges.
    >
    > I call that a pretty strong guarantee.


    I think you mis-understood Chris's point. The documentation is the
    specification of how something behaves. If the documentation doesn't
    say it, you can't rely on it. The user should never have to read the
    source to know how to use a function, or what they can depend on. Now,
    I'm not saying that reading the source isn't useful for a deeper
    understanding, but it should be understood that any insights you glean
    from doing that are strictly implementation details.

    If you're saying that there are guarantees made by the implementation of
    getrandbits() which are not documented, then one of two things are true:

    1) It is intended that users can depend on that behavior, in which case
    it's a bug in the docs, and the docs should be updated.

    or

    2) It is not intended that users can depend on that behavior, in which
    case they would be foolish to do so.
    Roy Smith, Dec 25, 2011
    #14
  15. Roy Smithæ–¼ 2011å¹´12月26日星期一UTC+8上åˆ1時41分29秒寫é“:
    > On Mon, 26 Dec 2011 03:11:56 +1100, Chris Angelico wrote:
    > > > I prefer not to rely on the source. That tells me what happens, not
    > > > what's guaranteed to happen.

    >
    > Steven D'Aprano <> wrote:
    > > In this case, the source explicitly tells you that the API includes
    > > support for arbitrary large ranges if you include a getrandbits() method:
    > >
    > > Optionally, implement a getrandbits() method so that randrange()
    > > can cover arbitrarily large ranges.
    > >
    > > I call that a pretty strong guarantee.

    >
    > I think you mis-understood Chris's point. The documentation is the
    > specification of how something behaves. If the documentation doesn't
    > say it, you can't rely on it. The user should never have to read the
    > source to know how to use a function, or what they can depend on. Now,
    > I'm not saying that reading the source isn't useful for a deeper
    > understanding, but it should be understood that any insights you glean
    > from doing that are strictly implementation details.
    >
    > If you're saying that there are guarantees made by the implementation of
    > getrandbits() which are not documented, then one of two things are true:
    >
    > 1) It is intended that users can depend on that behavior, in which case
    > it's a bug in the docs, and the docs should be updated.
    >
    > or
    >
    > 2) It is not intended that users can depend on that behavior, in which
    > case they would be foolish to do so.


    Random bit generations for RSA2048 encoding and cryptography applications
    in python is simple and elegant.
    88888 Dihedral, Dec 25, 2011
    #15
  16. On Sun, 25 Dec 2011 12:41:29 -0500, Roy Smith wrote:

    > On Mon, 26 Dec 2011 03:11:56 +1100, Chris Angelico wrote:
    >> > I prefer not to rely on the source. That tells me what happens, not
    >> > what's guaranteed to happen.

    >
    > Steven D'Aprano <> wrote:
    >> In this case, the source explicitly tells you that the API includes
    >> support for arbitrary large ranges if you include a getrandbits()
    >> method:
    >>
    >> Optionally, implement a getrandbits() method so that randrange()
    >> can cover arbitrarily large ranges.
    >>
    >> I call that a pretty strong guarantee.

    >
    > I think you mis-understood Chris's point.


    And I'm afraid that you have missed my point. The above comment from the
    source is from a docstring: it *is* public, official documentation. See
    help(random.Random).


    > The documentation is the
    > specification of how something behaves. If the documentation doesn't
    > say it, you can't rely on it.


    A nice platitude, but not true. Documentation is often incomplete or even
    inaccurate. We rely on many things that aren't documented anywhere. For
    example, we can rely on the fact that

    x = 3917
    print x+1

    will print 3918, even though that specific fact isn't documented
    anywhere. Nevertheless, we can absolutely bank on it -- if it happened to
    do something else, we would report it as a bug and not expect to be told
    "implementation detail, will not fix". We make a number of undocumented
    assumptions:

    * we assume that when the documentation talks about "adding" two
    numbers, it means the standard mathematical definition of addition
    and not some other meaning;

    * we assume that the result of such addition must be *correct*,
    without that assumption being guaranteed anywhere;

    * we assume that addition of two ints will return an int, as opposed
    to some other numerically equal value such as a float or Fraction;

    * we assume that not only will it be an int, but it will be *exactly*
    an int, and not a subclass of int;

    and no doubt there are others.

    And by the way, in case you think I'm being ridiculously pedantic,
    consider the first assumption listed above: the standard mathematical
    definition of addition. That assumption is violated for floats.

    py> 0.7 + 0.1 == 0.8
    False


    It is rather amusing the trust we put in documentation, when
    documentation can be changed just as easily as code. Just because the
    Fine Manual says that x+1 performs addition today, doesn't mean that it
    will continue to say the same thing tomorrow. So if you trust the
    language designers not to arbitrarily change the documentation, why not
    trust them not to arbitrarily change the code?

    Hint: as far as I can tell, nowhere does Python promise that printing
    3918 will show those exact digits, but it's a pretty safe bet that Python
    won't suddenly change to displaying integers in balanced-ternary notation
    instead of decimal.



    > The user should never have to read the
    > source to know how to use a function, or what they can depend on.


    Well, that's one school of thought. Another school of thought is that
    documentation and comments lie, and the only thing that you can trust is
    the code itself.

    At the end of the day, the only thing that matters is the mindset of the
    developer(s) of the software. You must try to get into their head and
    determine whether they are the sort of people whose API promises can be
    believed.


    > Now,
    > I'm not saying that reading the source isn't useful for a deeper
    > understanding, but it should be understood that any insights you glean
    > from doing that are strictly implementation details.


    I couldn't care less about the specific details of *how* getrandbits is
    used to put together an arbitrarily large random number. That "how" is
    what people mean when they talk about "mere implementation details".

    But the fact that getrandbits is used by randrange (as opposed to how it
    is used) is not a mere implementation detail, but a public part of the
    interface.


    > If you're saying that there are guarantees made by the implementation of
    > getrandbits() which are not documented, then one of two things are true:


    The implementation of getrandbits is not documented at all. You would
    have to read the C source of the _random module to find out how the
    default pseudo-random number generator produces random bits. But note the
    leading underscore: _random is a private implementation detail.

    However the existence and use of random.Random.getrandbits is public,
    documented, and guaranteed.


    --
    Steven
    Steven D'Aprano, Dec 26, 2011
    #16
  17. On Mon, Dec 26, 2011 at 2:00 PM, Steven D'Aprano
    <> wrote:
    > The implementation of getrandbits is not documented at all. You would
    > have to read the C source of the _random module to find out how the
    > default pseudo-random number generator produces random bits. But note the
    > leading underscore: _random is a private implementation detail.
    >
    > However the existence and use of random.Random.getrandbits is public,
    > documented, and guaranteed.


    It's that last line where I find disagreement between documentation
    and source code.

    """Class Random can also be subclassed if you want to use a different
    basic generator of your own devising: in that case, override the
    random(), seed(), getstate(), and setstate() methods. Optionally, a
    new generator can supply a getrandbits() method — this allows
    randrange() to produce selections over an arbitrarily large range."""

    My reading of this is that, if I were to write my own generator, I
    could provide that method and gain perfection. It's not until you dig
    somewhat that you find out that the default generator actually does
    provide getrandbits, meaning that the default randrange can actually
    be used for arbitrarily large ranges. Yes, it IS documented, but in
    separate places and somewhat as an afterthought; in the 2.x
    documentation it's shown as "new in 2.4", which explains this.

    Unfortunately my brain isn't really running on all cylinders at the
    moment and I can't come up with a better wording, but is there some
    way this could be stated up in the same top paragraph that explains
    about 53-bit precision and 2**19937-1 period? That's where I got my
    original information from; I didn't go down to the individual function
    definitions, which is the only place that it's stated that the
    Mersenne Twister does include getrandbits (and which assumes that
    you've read the top section that states that Mersenne Twister is the
    default implementation).

    ChrisA
    Chris Angelico, Dec 26, 2011
    #17
  18. Roy Smith

    Roy Smith Guest

    In article <4ef7e337$0$29973$c3e8da3$>,
    Steven D'Aprano <> wrote:

    > And I'm afraid that you have missed my point. The above comment from the
    > source is from a docstring: it *is* public, official documentation. See
    > help(random.Random).


    When you wrote, "the source explicitly tells you..." the natural
    assumption I made was "something in the source that's not part of the
    documentation" (i.e. some comment).

    > > The documentation is the specification of how something behaves.
    > > If the documentation doesn't say it, you can't rely on it.


    > A nice platitude, but not true. Documentation is often incomplete or even
    > inaccurate.


    Of course it is. Those things constitute doc bugs which need to get
    fixed. The fact that the specification is flawed does not change the
    fact that it *is* the specification.
    Roy Smith, Dec 26, 2011
    #18
  19. On Mon, Dec 26, 2011 at 3:17 PM, Roy Smith <> wrote:
    > Of course it is.  Those things constitute doc bugs which need to get
    > fixed.  The fact that the specification is flawed does not change the
    > fact that it *is* the specification.


    Also, the specification is what can be trusted across multiple
    implementations of Python - the source can't. I don't intend to get
    the source code to five Pythons to ascertain whether getrandbits() is
    provided in the default random.Random() of each of them. Now, in this
    particular instance, it IS stated in the docs - just down in the
    individual function descriptions, which I hadn't read - so I almost
    certainly CAN trust that across all platforms. But in the general
    case, I would not want to make firm statements on the basis of
    implementation details.

    ChrisA
    Chris Angelico, Dec 26, 2011
    #19
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Anoop
    Replies:
    4
    Views:
    378
    Anoop
    Jul 26, 2006
  2. Anoop

    String.digits help!!!

    Anoop, Aug 8, 2006, in forum: Python
    Replies:
    5
    Views:
    487
    Ron Adam
    Aug 9, 2006
  3. globalrev
    Replies:
    4
    Views:
    742
    Gabriel Genellina
    Apr 20, 2008
  4. sotirac
    Replies:
    16
    Views:
    521
    sotirac
    Sep 29, 2008
  5. VK
    Replies:
    15
    Views:
    1,120
    Dr J R Stockton
    May 2, 2010
Loading...

Share This Page