EOF and reading /proc

[Mark Seger]

I have a script I've been running for some time that reads process data
from /proc/pid/stat. My outer loop open /proc and reads each entry with
readdir, opening that appropriate stat file and reading though it in
an inner loop as follow - the reason for the '99' is the same routine is
used to read other /proc structures beside process data:

for ($i=0; $i<99; $i++)
{
last if !($line=<PROC>);
print $line;
}
close PROC;

In any event, this should (and does) read a single line and then fall
through the loop. However, if I run this for a long time, it
occasionaly reads a second record which is not right! the contents of
that partial line look like the end of 'stat':

Opened /proc/8887/stat
8887 (sshd) R 1761 1761 1761 0 -1 64 153 54 231 43 19 62 0 0 15 0 0 0
484375023 6979584 449 4294967295 134512640 134784984 3221221808
3221202756 4294959106 0 0 4096 73728 0 0 0 17 0 0 0 19 62 0 0
9 62 0 0

thoughts?

-mark

 

[Big and Blue]

In any event, this should (and does) read a single line and then fall
through the loop. However, if I run this for a long time, it
occasionaly reads a second record which is not right! the contents of
that partial line look like the end of 'stat':

Opened /proc/8887/stat
8887 (sshd) R 1761 1761 1761 0 -1 64 153 54 231 43 19 62 0 0 15 0 0 0
484375023 6979584 449 4294967295 134512640 134784984 3221221808
3221202756 4294959106 0 0 4096 73728 0 0 0 17 0 0 0 19 62 0 0
9 62 0 0

You haven't said which OS, but it doesn't matter (looks like Linux?).

I suspect the problem goes like this:

a) you read the line (so your file-handle is now at position x == eol)

b) The process stats change. The line length is now longer.

c) You read from the same file handle, starting at position x. Because
of b there are now data beyond this point, so you get them.

The solution is trivial.

You know the file will only contain one line, so only read one line.
Don't use a generic loop unless you can control it to do what you actually
need. You *don't* want to read <n> lines until EOF - you want to read 1
line then close the file.

 

[Mark Seger]

yes, it's linux. your explanation sounds reasonable...
actually I had thought of just reading one line too, but wanted to
understand why this was happening rather than just changing some code -
something that happens all too often and ends up biting one if one isn't
careful.

I guess part of my problem is I don't know the semantics of how /proc
works since these aren't really files. Are you saying it's possible for
things to change in the middle of reading them? Wouldn't this make all
system utilities that read /proc subject to errors? I have looked at a
random set of utilities, like iostat and sar and they all seem to just
read /proc and not worry about corrupted data.

-mark

 

[Big and Blue]

I guess part of my problem is I don't know the semantics of how /proc
works since these aren't really files.

Define "file".

Are you saying it's possible for
things to change in the middle of reading them?

It is possible for the contents of *any* (unlocked) file to change
while your read it.

Wouldn't this make all
system utilities that read /proc subject to errors?

It depends on the format of the file. The stat file is 1 line, so if
you read 1 line you won't have a problem. Your problem is that having read
one line you look to see whether there is any more. In that gap between
the two reads the file contents may change.

I have looked at a
random set of utilities, like iostat and sar and they all seem to just
read /proc and not worry about corrupted data.

If they read it all in one go (perhaps indirectly) then parse it they
won't have a problem. Or the file may have a fixed format anyway.

 

[Michele Dondi]

things to change in the middle of reading them? Wouldn't this make all
system utilities that read /proc subject to errors? I have looked at a

This may well be OT, but actually I and a friend of mine noticed that
'w' occasionally gives erroneous output under heavy load. (On a Linux
machine, that is, or better: on quite a few Linux machines we tried
this thing on...)


Michele