/dev/urandom vs. /dev/random

R

Ron Peterson

In the following piece of code, which simply generates a sequence of
(random) octal codes, I'm surprised by the apparent non-randomness of
/dev/random. It's not noticeable unless RAND_LENGTH is largish. I was
under the assumption that /dev/random was "more random" than
/dev/urandom, and that it would block if it ran out of entropy until it
got more. Why am I seeing so many zeroes in my output?

#include <stdio.h>
#include <fcntl.h>
#define RAND_LEN 1024

void
read_random( const char* dev ) {
int i, fd;
char dat[RAND_LEN];

fd = open( dev, O_RDONLY );

dat[RAND_LEN] = '\0';

if( fd != -1 ) {
read( fd, dat, RAND_LEN );
for( i = 0; i < RAND_LEN; i++ ) {
dat = (dat >> 4 & 0x07) + 48;
}
printf( "%s: %s\n\n", dev, dat );
} else {
exit( 1 );
}

close( fd );
}

int
main( void ) {
read_random( "/dev/random" );
read_random( "/dev/urandom" );
return( 0 );
}

506$ ./test

/dev/random: 2000506170635264376025103466550724740317674263066050462223206040017306150002434274350703035033674561207541735104000400000000500000000000077372045773000017735604050445002204340400003404177361046404640400000000000000000000000000000000000000000000000000000000000000000000000000006404234610000000000000000000000000000000000000000000240400000000410070000000634610000000000000000000000000000000000000000000000000000000000000000000000056040000000000003404377304003773050430000000477300000000000000000000000000003000100000000000000000000000000000000000000000000000000000000000000000000004500450040000000020007004130433040704000017043404170405042773070405040030160457732404000067040000000000000000000070042000640400003404000064046773000450040000000057732000700413043304070400001704340464044000477307044000077000007773240400005704000000000000000000000000000034047100070477732204170467040000000000007773000000000000000007700773540405041604630407045773377302047000200040044773577311046414641437736504641457733404000054041773220464045704

/dev/urandom: 1031370027607513304714532275473174471533054030554455772411741125544374737071744665152471472301655103216730326121470443763613364711034254266017775642152753750650676053204675567754500234362613154550113060140067534031123241466461750705230751645572440071577444255364702743523324151641314351151255731014427455654471777705452015235176424166620757100770012235623563012311717741547124617262165454244161061517742016510632004461761354716624024114244125322455631154514247622366470217200055365261224751155042451472612104741103553240132614710710775524572432011176156426106214363467646401702717200545253373673060222036745552163576536757361575312361516251036006403026113642124742630634422243216772330177360713406462475345366453547703614053407702174514415307013537256732255776760340240651131657563766750630561770243035222250564070154307150656704155003661646277201332465230456417603435662664427316673133650166660764302367321031007354535664756417052425166306626013450370701275067537603265673560312322526077022157223404211323047413633213326054
 
J

Joona I Palaste

Ron Peterson said:
In the following piece of code, which simply generates a sequence of
(random) octal codes, I'm surprised by the apparent non-randomness of
/dev/random. It's not noticeable unless RAND_LENGTH is largish. I was
under the assumption that /dev/random was "more random" than
/dev/urandom, and that it would block if it ran out of entropy until it
got more. Why am I seeing so many zeroes in my output?

Your question is about the Unix devices /dev/random and /dev/urandom,
not about C. Please continue discussion about this on
comp.unix.programmer.
#include <stdio.h>
#include <fcntl.h>
#define RAND_LEN 1024
void
read_random( const char* dev ) {
int i, fd;
char dat[RAND_LEN];
fd = open( dev, O_RDONLY );
dat[RAND_LEN] = '\0';
if( fd != -1 ) {
read( fd, dat, RAND_LEN );
for( i = 0; i < RAND_LEN; i++ ) {
dat = (dat >> 4 & 0x07) + 48;
}
printf( "%s: %s\n\n", dev, dat );
} else {
exit( 1 );
}

close( fd );
}
int
main( void ) {
read_random( "/dev/random" );
read_random( "/dev/urandom" );
return( 0 );
}
506$ ./test
/dev/random: 2000506170635264376025103466550724740317674263066050462223206040017306150002434274350703035033674561207541735104000400000000500000000000077372045773000017735604050445002204340400003404177361046404640400000000000000000000000000000000000000000000000000000000000000000000000000006404234610000000000000000000000000000000000000000000240400000000410070000000634610000000000000000000000000000000000000000000000000000000000000000000000056040000000000003404377304003773050430000000477300000000000000000000 0000000030001000000000000000000000000000000000000000000000000000000000000000000000045004500400000000200070041304330407040000170434041704050427730704050400301604577324040000670400000000000000000000700420006404000034040000640467730004500400000000577320007004130433040704000017043404640440004773070440000770000077732404000057040000000000000000000000000000340471000704777322041704670400000000000077730000000000000000077007735404050416046304070457733773020470002000400447735773110464146414377365046414577334040000540
41773220464045704

/dev/urandom: 103137002760751330471453227547317447153305403055445577241174112554437473707174466515247147230165510321673032612147044376361336471103425426601777564215275375065067605320467556775450023436261315455011306014006753403112324146646175070523075164557244007157744425536470274352332415164131435115125573101442745565447177770545201523517642416662075710077001223562356301231171774154712461726216545424416106151774201651063200446176135471662402411424412532245563115451424762236647021720005536526122475115504
2451472612104741103553240132614710710775524572432011176156426106214363467646401702717200545253373673060222036745552163576536757361575312361516251036006403026113642124742630634422243216772330177360713406462475345366453547703614053407702174514415307013537256732255776760340240651131657563766750630561770243035222250564070154307150656704155003661646277201332465230456417603435662664427316673133650166660764302367321031007354535664756417052425166306626013450370701275067537603265673560312322526077022157223404211323
047413633213326054
 
I

infobahn

Ron said:
In the following piece of code, which simply generates a sequence of
(random) octal codes, I'm surprised by the apparent non-randomness of
/dev/random. It's not noticeable unless RAND_LENGTH is largish. I was
under the assumption that /dev/random was "more random" than
/dev/urandom, and that it would block if it ran out of entropy until it
got more. Why am I seeing so many zeroes in my output?

#include <stdio.h>
#include <fcntl.h>
#define RAND_LEN 1024

void
read_random( const char* dev ) {
int i, fd;
char dat[RAND_LEN];

fd = open( dev, O_RDONLY );

dat[RAND_LEN] = '\0';

This assignment violates dat's bounds, invoking undefined behaviour.
So all bets are off.

Make it char dat[RAND_LEN + 1];
if( fd != -1 ) {
read( fd, dat, RAND_LEN );
for( i = 0; i < RAND_LEN; i++ ) {
dat = (dat >> 4 & 0x07) + 48;


Again, undefined behaviour. This time it's because you're reading
an indeterminate value, dat.
}
printf( "%s: %s\n\n", dev, dat );
} else {
exit( 1 );
}

close( fd );
}

int
main( void ) {
read_random( "/dev/random" );
read_random( "/dev/urandom" );

Leaving those bugs unfixed, does swapping these two calls
reverse the behaviour? That is, if you do this instead:

read_random( "/dev/urandom" );
read_random( "/dev/random" );

are the excess 0s now associated with /dev/urandom instead?

If so, you can be reasonably sure that the problem lies with
the problems I've mentioned.
 
R

Ron Peterson

I was copying bits and pieces from other code, and made a couple of
errors. I should have typed:

char dat[RAND_LEN + 1]; (so I'm not going beyond end of array)

and

dat = (dat & 0x07) + '0'; (I don't need to bit shift, I just want
three bits).

Anyway, when I do these things, I still get long strings of zeroes from
/dev/random. ?

Not exactly a C question, I know. I could direct this to the kernel
list, as the kernel provides random and urandom, but I was thinking the
behaviour should be consistent between kernels, so it's not really a
kernel question. If someone can suggest a more appropriate list, that's
helpful too.

BTW, I'm using linux 2.6.x. If anyone running a BSD variant could
compile this and compare the results, that might be interesting.

-Ron-


In the following piece of code, which simply generates a sequence of
(random) octal codes, I'm surprised by the apparent non-randomness of
/dev/random. It's not noticeable unless RAND_LENGTH is largish. I was
under the assumption that /dev/random was "more random" than
/dev/urandom, and that it would block if it ran out of entropy until it
got more. Why am I seeing so many zeroes in my output?

#include <stdio.h>
#include <fcntl.h>
#define RAND_LEN 1024

void
read_random( const char* dev ) {
int i, fd;
char dat[RAND_LEN];

fd = open( dev, O_RDONLY );

dat[RAND_LEN] = '\0';

if( fd != -1 ) {
read( fd, dat, RAND_LEN );
for( i = 0; i < RAND_LEN; i++ ) {
dat = (dat >> 4 & 0x07) + 48;
}
printf( "%s: %s\n\n", dev, dat );
} else {
exit( 1 );
}

close( fd );
}

int
main( void ) {
read_random( "/dev/random" );
read_random( "/dev/urandom" );
return( 0 );
}

506$ ./test

/dev/random: 2000506170635264376025103466550724740317674263066050462223206040017306150002434274350703035033674561207541735104000400000000500000000000077372045773000017735604050445002204340400003404177361046404640400000000000000000000000000000000000000000000000000000000000000000000000000006404234610000000000000000000000000000000000000000000240400000000410070000000634610000000000000000000000000000000000000000000000000000000000000000000000056040000000000003404377304003773050430000000477300000000000000000000000000003000100000000000000000000000000000000000000000000000000000000000000000000004500450040000000020007004130433040704000017043404170405042773070405040030160457732404000067040000000000000000000070042000640400003404000064046773000450040000000057732000700413043304070400001704340464044000477307044000077000007773240400005704000000000000000000000000000034047100070477732204170467040000000000007773000000000000000007700773540405041604630407045773377302047000200040044773577311046414641437736504641457733404000054041773220464045704

/dev/urandom: 1031370027607513304714532275473174471533054030554455772411741125544374737071744665152471472301655103216730326121470443763613364711034254266017775642152753750650676053204675567754500234362613154550113060140067534031123241466461750705230751645572440071577444255364702743523324151641314351151255731014427455654471777705452015235176424166620757100770012235623563012311717741547124617262165454244161061517742016510632004461761354716624024114244125322455631154514247622366470217200055365261224751155042451472612104741103553240132614710710775524572432011176156426106214363467646401702717200545253373673060222036745552163576536757361575312361516251036006403026113642124742630634422243216772330177360713406462475345366453547703614053407702174514415307013537256732255776760340240651131657563766750630561770243035222250564070154307150656704155003661646277201332465230456417603435662664427316673133650166660764302367321031007354535664756417052425166306626013450370701275067537603265673560312322526077022157223404211323047413633213326054
 
J

Jens.Toerring

Ron Peterson said:
I was copying bits and pieces from other code, and made a couple of
errors. I should have typed:
char dat[RAND_LEN + 1]; (so I'm not going beyond end of array)

dat = (dat & 0x07) + '0'; (I don't need to bit shift, I just want
three bits).

Anyway, when I do these things, I still get long strings of zeroes from
/dev/random. ?
Not exactly a C question, I know. I could direct this to the kernel
list, as the kernel provides random and urandom, but I was thinking the
behaviour should be consistent between kernels, so it's not really a
kernel question. If someone can suggest a more appropriate list, that's
helpful too.

Joona set the follow-up to comp.unix.programmer where it's more
on-topic and where I tried to answer the question...

Regards, Jens
 
C

Christopher Benson-Manica

infobahn said:
if( fd != -1 ) {
read( fd, dat, RAND_LEN );
for( i = 0; i < RAND_LEN; i++ ) {
dat = (dat >> 4 & 0x07) + 48;

Again, undefined behaviour. This time it's because you're reading
an indeterminate value, dat.


It isn't necessarily undefined behavior, if RAND_LEN bytes happen to
have been read. Unless I'm missing something, the code could be
corrected with

if( fd != EOF ) { /* right? */
int bytes=read( fd, dat, RAND_LEN );
if( bytes == -1 ) {
/* error */
}
if( bytes ) { /* bytes == 0 implies end of file */
for( i=0; i < bytes; i++ ) {
/* ... */
}
}
/* ... */
}
 
I

infobahn

Ron said:
I was copying bits and pieces from other code, and made a couple of
errors. I should have typed:

Oh dear. We'd prefer to debug your program errors, not your
typographical errors. :-(
char dat[RAND_LEN + 1]; (so I'm not going beyond end of array)

Fair enough.
and

dat = (dat & 0x07) + '0'; (I don't need to bit shift, I just want
three bits).


This is still broken. Please refer to my earlier reply.
 
C

Chris Croughton

In the following piece of code, which simply generates a sequence of
(random) octal codes, I'm surprised by the apparent non-randomness of
/dev/random. It's not noticeable unless RAND_LENGTH is largish. I was
under the assumption that /dev/random was "more random" than
/dev/urandom, and that it would block if it ran out of entropy until it
got more. Why am I seeing so many zeroes in my output?

For a start, your question is not really C related, it could happen in
any language. Not only is /dev/random device specific but you are also
using POSIX I/O functions (open/read etc.) rather than the C ones.
read( fd, dat, RAND_LEN );

What is the length read? It is quite possible that read() is returning
when it gets the first byte or few bytes, and the rest of the array is
undefined.
for( i = 0; i < RAND_LEN; i++ ) {

You are assuming that you know that the whole array was filled. I
suggest that it wasn't.

(I have a reason for suggesting this. I used dd(1) to read a block from
/dev/random into a file. The first time it got 1024 bytes. The next
time it got two, the one after around 30. I'd be pretty sure that dd(1)
is using the same mechanism to read data. That was on Linux kernel
2.2.19...)

Which does make it a sort-of C question, because if you used the C I/O
(fopen, fread etc.) you might find different behaviour. Or perhaps
not...

Chris C
 
I

infobahn

Christopher said:
infobahn said:
if( fd != -1 ) {
read( fd, dat, RAND_LEN );
for( i = 0; i < RAND_LEN; i++ ) {
dat = (dat >> 4 & 0x07) + 48;

Again, undefined behaviour. This time it's because you're reading
an indeterminate value, dat.


It isn't necessarily undefined behavior, if RAND_LEN bytes happen to
have been read.


My apologies to the OP. I didn't spot the read()! (Duh...)

In that case, I am not sure that anything remains for comp.lang.c
to contribute here. Perhaps comp.unix.programmer could help.
 
R

Ron Peterson

Anyway, when I do these things, I still get long strings of zeroes from
/dev/random. ?

Not exactly a C question, I know. I could direct this to the kernel
list, as the kernel provides random and urandom, but I was thinking the
behaviour should be consistent between kernels, so it's not really a
kernel question.

Pencils down. The LKML list pinned it down for me.

It was in fact a C programming error. I wasn't checking the return
value of 'read', to see how much data was actually read.

Best.
 
C

Christopher Benson-Manica

Ron Peterson said:
Pencils down. The LKML list pinned it down for me.
It was in fact a C programming error. I wasn't checking the return
value of 'read', to see how much data was actually read.

Not to boast, but note my response elsethread :)
 
M

Michael Wojcik

It was in fact a C programming error. I wasn't checking the return
value of 'read', to see how much data was actually read.

This is a POSIX programming error, not a C programming error. You
could make precisely the same mistake with any language capable of
making system calls on a POSIX system. read is not a C function;
it's a POSIX function.

And that is precisely why this question was OT for comp.lang.c, and
topical for comp.unix.programmer.

--
Michael Wojcik (e-mail address removed)

Painful lark, labouring to rise!
The solemn mallet says:
In the grave's slot
he lies. We rot. -- Basil Bunting
 
M

Michael Wojcik

if( fd != EOF ) { /* right? */

Wrong, I'm afraid, and yet another example of why we shouldn't try
to answer off-topic questions here.

fd is the return value from the POSIX open function. open returns
a file descriptor, which is a non-negative integer, or -1 on failure.
Since EOF may not be -1, that's not the appropriate value to compare
it to.
int bytes=read( fd, dat, RAND_LEN );
if( bytes == -1 ) {
/* error */
}
if( bytes ) { /* bytes == 0 implies end of file */
for( i=0; i < bytes; i++ ) {
/* ... */
}
}

This does fix part of the error in the OP's code (it appears that
he expects the read_random function to populate the entire buffer,
so it really needs to loop around calls to read until it's done so),
but note that the test "if( bytes )" is extraneous, since the for
loop does nothing if bytes == 0.
 
C

Christopher Benson-Manica

Michael Wojcik said:
Wrong, I'm afraid, and yet another example of why we shouldn't try
to answer off-topic questions here.

Well, it wasn't totally off-topic, since we agree that OP failed to
use the standard read() function correctly.
 
M

Martin Ambuhl

Christopher said:
Well, it wasn't totally off-topic, since we agree that OP failed to
use the standard read() function correctly.

There is no standard function in C called "read". If you are referring
to the POSIX function called "read", then you know that it is topical in
newsgroups addressing POSIX issues and off-topic in newsgroups
addressing C issues. Your "reason" for the question's not being
"totally off-topic" is completely bogus.
 
K

Keith Thompson

Christopher Benson-Manica said:
Well, it wasn't totally off-topic, since we agree that OP failed to
use the standard read() function correctly.

What standard read() function would that be? read() is POSIX, not
ISO C. fread() is ISO C.
 
I

infobahn

Christopher said:
Well, it wasn't totally off-topic, since we agree that OP failed to
use the standard read() function correctly.

No, that's not why it wasn't off-topic (for reasons which have been
explained already). It wasn't off-topic because it was a question
about C. The fact that the OP used read() turned out to be relevant,
at which point it would have been perfectly reasonable to redirect
him to another newsgroup if it weren't for the fact that his
question had by that stage been answered already.

We cannot expect everyone asking questions here to know everything
about C, and particularly what is and what is not C. That does not
mean that all questions are topical here if they contain the letter
C! A detailed explanation of the read() function and its part in
the OP's downfall would have been off-topic here, but the question
itself was fine. Consider this question:

Q: As my example code shows, I'm struggling with fork(). Could
someone tell me what I'm doing wrong?

which is a perfectly sensible question to ask in this newsgroup.
Of course, the only topical /answer/ would be to refer the OP to a
*nix group, but that's beside the point.
 
M

Michael Wojcik

No, that's not why it wasn't off-topic (for reasons which have been
explained already). It wasn't off-topic because it was a question
about C. The fact that the OP used read() turned out to be relevant,
at which point it would have been perfectly reasonable to redirect
him to another newsgroup if it weren't for the fact that his
question had by that stage been answered already.

Agreed, but note that by "answer off-topic questions" above I meant
specifically "try to provide a direct answer, rather than a
redirection to an appropriate group" (usage which is frequently
employed in topicality discussions here).

Of course when confronted with questions that the questioner believes
are about C, but actually turn on an implementation dependency, we
should "answer" them in the sense of providing a redirection. What
we should avoid doing is trying to answer the substance of the
question, for the reasons commonly cited: we may be mistaken about
the implementation, and our responses will not be vetted by the
community dedicated to that implementation.
A detailed explanation of the read() function and its part in
the OP's downfall would have been off-topic here, but the question
itself was fine.

That was indeed my point (except that I referred to Christopher's
comment regarding the return value of the open() function, not the
read() function).

--
Michael Wojcik (e-mail address removed)

Auden often writes like Disney. Like Disney, he knows the shape of beasts --
(& incidently he, too, might have a company of artists producing his lines) --
unlike Lawrence, he does not know what shapes or motivates these beasts.
-- Dylan Thomas
 
C

Christopher Benson-Manica

Keith Thompson said:
What standard read() function would that be?

The one that I thought existed after looking it up in K&R; I should
have understood the difference between "library function" (standard)
and "system call" (not standard). Sorry.
 
K

Keith Thompson

Christopher Benson-Manica said:
The one that I thought existed after looking it up in K&R; I should
have understood the difference between "library function" (standard)
and "system call" (not standard). Sorry.

The distinction between library functions and system calls isn't the
same as the distinction between standard and non-standard functions.
System calls are an implementation detail; a function in the C
standard library could easily be implemented as a system call. (The
time() function typically is, for example.) Similarly, there are
plenty of functions defined by POSIX but not by ISO C that are
typically implemented as library functions rather than as system
calls.

<OT>
A system call generally works by invoking code within the OS kernel
rather than in a user-mode library. On a Unix-like system, system
calls are documented by man pages in section 2, library functions in
section 3.
</OT>
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,731
Messages
2,569,432
Members
44,832
Latest member
GlennSmall

Latest Threads

Top