random number generator thread safety

[Mike Brown]

I have questions about thread safety in the 'random' module.

When using the random.Random class (be it Mersenne Twister or Wichmann-Hill
based), is it sufficiently thread-safe (preserving entropy and guarding
against attack) to just have each thread work with its own random.Random
instance? Or do I need to wrap my calls to each instance's methods to use
locks? Wichmann-Hill in particular has the warning in its .random()
vulnerability; do I need to make an exception for that case?

What about when using the random.SystemRandom class (be it based on
/dev/urandom or CryptGenRandom based)? Should I be doing anything in my code
to ensure that /dev/urandom, for example, is polled serially? (I thought I
read somewhere that this was advisable). SystemRandom claims it has no state,
which may be true at the Python class level, but is that guaranteed to be true
for the underlying RNG?

Also, side note: The doc string for random.SystemRandom says "Not available on
all systems (see os.urandom() for details)" ...but os.urandom()'s doc string
provides no details. Actually, os.urandom() and random.SystemRandom are
"available" on all systems (they exist and can be called/instantiated), but
os.urandom() can raise a NotImplementedError (any time it is called, in
theory), therefore random.SystemRandom's methods can fail with a bubbled-up
NotImplementedError (on non-win32 OSes, at least). At the very least, this
should be documented. I'm not in a position to submit a patch, but will submit
a documentation bug report if so advised.

Thanks,
Mike

 

[Raymond Hettinger]

I have questions about thread safety in the 'random' module.

When using the random.Random class (be it Mersenne Twister or Wichmann-Hill
based), is it sufficiently thread-safe (preserving entropy and guarding
against attack) to just have each thread work with its own random.Random
instance? Or do I need to wrap my calls to each instance's methods to use
locks? Wichmann-Hill in particular has the warning in its .random()
vulnerability; do I need to make an exception for that case?

Thread-safety has nothing to do with preserving entropy or guarding
against attack. All of the entropy in an MT sequence is contained in
the seed (upto 624 bytes) and that entropy is preserved through all
subsequent calls.

Create unique instances of MT generators when you want to be able to
reproduce the sequence later.

Nothing in the random module provides cryptographic guarantees. If you
want crypto-strength, then use real encryption. Search SourceForge for
patches that show how to use MD5 and other cryptographic hash functions
as a basis for random number generation.

As far as I'm concerned, there is no reason other than backwards
compatability to use the Wichmann-Hill generator -- the Mersenne
Twister is a much better generator.



Raymond

 

[Paul Rubin]

Thread-safety has nothing to do with preserving entropy or guarding
against attack. All of the entropy in an MT sequence is contained in
the seed (upto 624 bytes) and that entropy is preserved through all
subsequent calls.

I think the concern is that there can be a thread switch after some
intermediate result is computed but before the state is updated. That
would mean two threads can get random numbers that are identical or
anyway correlated. Whether that can happen with Python's MT, I don't
know.

Nothing in the random module provides cryptographic guarantees. If you
want crypto-strength, then use real encryption. Search SourceForge for
patches that show how to use MD5 and other cryptographic hash functions
as a basis for random number generation.

Or use os.urandom.

 

[Raymond Hettinger]

Thread-safety has nothing to do with preserving entropy or guarding

I think the concern is that there can be a thread switch after some
intermediate result is computed but before the state is updated.

The concern expressed by the OP was "preserving entropy or guarding
against attack".

That
would mean two threads can get random numbers that are identical or
anyway correlated. Whether that can happen with Python's MT, I don't
know.

It can't. That is what the docs mean when they say that MT is
thread-safe.