pid from startet process

Discussion in 'Perl Misc' started by carlo.maier@netcologne.de, Jul 19, 2007.

  1. Guest

    hi,

    i am starting an exectuable on hp-ux (ActivePerl Build 817) from a
    perl script and need to know the childs pid.

    my $cmd="myexecutable -P parmfile "

    my $pid=open(H, "${cmd} |");
    print "\ttest: $_" while (<H>);
    close(H);

    The executable starts, but the pid i am getting is not right. The
    executable writes a logfile with the pid as part of the logfile name.
    After the executable returns, i want to check the logfile:

    my $logfile="${logdir}/program_${pid}.log";
    open(H,"< $logfile") or die "Can't open logfile '${logfile}': ($!)\n";
    while (<H>) {
    print if (/ENTRY/);
    }
    close (H);

    For the reason that the pid, that i am trying to catch with the first
    open-statement, is not correct, i have a problem to check the logfile.

    What kind of reasons can be imagined, that the method "$pid=open(H, "$
    {cmd} |");" doesn't supply the correct pid?
    Does anybody know about a method that gives me the correct pid?

    Thanks in adance,
    carlo
     
    , Jul 19, 2007
    #1
    1. Advertising

  2. Mumia W. Guest

    On 07/19/2007 07:51 AM, wrote:
    > [...]
    > What kind of reasons can be imagined, that the method "$pid=open(H, "$
    > {cmd} |");" doesn't supply the correct pid?
    > Does anybody know about a method that gives me the correct pid?
    >
    > Thanks in adance,
    > carlo
    >


    Perhaps "myexecutable" is a script that starts the real executable.

    Although it's just guessing, the real PID might be $pid+1.
     
    Mumia W., Jul 19, 2007
    #2
    1. Advertising

  3. Guest

    wrote:
    > hi,
    >
    > i am starting an exectuable on hp-ux (ActivePerl Build 817) from a
    > perl script and need to know the childs pid.
    >
    > my $cmd="myexecutable -P parmfile "
    >
    > my $pid=open(H, "${cmd} |");
    > print "\ttest: $_" while (<H>);
    > close(H);
    >
    > The executable starts, but the pid i am getting is not right.


    I don't see that problem.

    $ perl -le 'print open my $fh, q{perl -le "print \$$;" |}; print <$fh>'
    11050
    11050

    Maybe your $cmd isn't doing what you think it is doing.

    Also, you might want to try using the 3-or-more argument form
    of open:

    open my $fh, "-|", "$cmd1", @cmd_args;

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Jul 19, 2007
    #3
  4. On Jul 19, 5:51 am, wrote:
    > ...
    > What kind of reasons can be imagined, that the method "$pid=open(H, "$
    > {cmd} |");" doesn't supply the correct pid?
    > Does anybody know about a method that gives me the correct pid?
    >


    If 'myexecutable' starts another script as
    suggested and you're able to tinker with
    myexecutable, launching the secondary script
    via 'exec' will preserve the $pid.

    hth,
    --
    Charles DeRykus
     
    comp.llang.perl.moderated, Jul 19, 2007
    #4
  5. In article <20070719123834.995$>,
    writes:
    > wrote:
    > > i am starting an exectuable on hp-ux (ActivePerl Build 817) from a
    > > perl script and need to know the childs pid.
    > >
    > > my $cmd="myexecutable -P parmfile "
    > >
    > > my $pid=open(H, "${cmd} |");
    > > print "\ttest: $_" while (<H>);
    > > close(H);
    > >
    > > The executable starts, but the pid i am getting is not right.

    >
    > I don't see that problem.
    >
    > $ perl -le 'print open my $fh, q{perl -le "print \$$;" |}; print <$fh>'
    > 11050
    > 11050


    You don't see the problem because you're not doing the same thing
    as the original poster.

    The original poster's basic problem is that this form of open
    (using '|') almost certainly invokes the system shell in order to
    execute the command. That shell then creates a new process to
    execute the command named in the $cmd parameter. The Perl open
    function returns the PID of the shell, but the child (and the
    relevant program) will have a different PID.

    (And in response to another comment in this thread, the PID of
    child will almost certainly not be the shell's PID plus 1. Any
    system that does that presents a major security risk and should
    be trashed immediately -- because PIDs would then be guessable.)

    What you have done is to present to the shell a command (print)
    that it can execute internally. Thus, it does not need to create
    a child to execute the command. Even if that weren't the case,
    you have used a construct ($$) that gets expanded at the shell
    level before the subprocess is invoked. (Note you have actually
    quoted that construct in several ways, but you have only protected
    it against expansion in the shell that invoked perl and within
    perl itself. To protect it against the shell that is created by
    the open function would require several more backslashes and lots
    of experimentation.)

    > Maybe your $cmd isn't doing what you think it is doing.
    >
    > Also, you might want to try using the 3-or-more argument form
    > of open:
    >
    > open my $fh, "-|", "$cmd1", @cmd_args;


    That will exhibit the same problem.

    > Xho


    To the OP: What you are trying to do is rather difficult and can
    be system-dependent. There are several approaches that are often
    used:

    - Many subsystems avoid this situation by allowing you to specify
    the name of the logfile as a parameter, either in the command
    line or in the parameter file.

    - You might be able to avoid the creation of the shell by using
    the perl fork and exec built-in functions. This will require
    knowledge about the underlying POSIX functions.

    - You might perform some shell trickery whereby the newly
    created shell would create the child asynchronously, store the
    PID someplace that you could read it (e.g., in a file or as
    the first returned line in the output). That would require
    a command something like "($cmd & echo PID=$$ ; wait) |" -- but
    that would require a higher comfort level with shell programming.
    (And don't forget about the need for quoting within Perl.)

    - This is a fairly common problem. Someone may have actually
    written a package for this, but I am not familiar with any
    (having normally avoided the problem via the above solutions).
    It would probably be profitable for you to do some searching.

    Good luck,

    - dmw

    --
    .. Douglas Wells . Connection Technologies .
    .. Internet: -sp9804- -at - contek.com- .
     
    Douglas Wells, Jul 19, 2007
    #5
  6. -berlin.de Guest

    Douglas Wells <> wrote in comp.lang.perl.misc:
    > In article <20070719123834.995$>,
    > writes:
    > > wrote:


    > (And in response to another comment in this thread, the PID of
    > child will almost certainly not be the shell's PID plus 1. Any
    > system that does that presents a major security risk and should
    > be trashed immediately -- because PIDs would then be guessable.)


    It's the other way around. Anything that relies on the non-predict-
    ability of PIDs is insecure, unless the system guarantees that property
    in a quantifiable way. I've never heard of that.

    Many Unix systems assign PIDs sequentially (usually skipping those
    that have been used recently, in some sense).

    Anno
     
    -berlin.de, Jul 20, 2007
    #6
  7. -berlin.de Guest

    <> wrote in comp.lang.perl.misc:
    > hi,
    >
    > i am starting an exectuable on hp-ux (ActivePerl Build 817) from a
    > perl script and need to know the childs pid.
    >
    > my $cmd="myexecutable -P parmfile "
    >
    > my $pid=open(H, "${cmd} |");
    > print "\ttest: $_" while (<H>);
    > close(H);
    >
    > The executable starts, but the pid i am getting is not right. The
    > executable writes a logfile with the pid as part of the logfile name.
    > After the executable returns, i want to check the logfile:
    >
    > my $logfile="${logdir}/program_${pid}.log";
    > open(H,"< $logfile") or die "Can't open logfile '${logfile}': ($!)\n";
    > while (<H>) {
    > print if (/ENTRY/);
    > }
    > close (H);
    >
    > For the reason that the pid, that i am trying to catch with the first
    > open-statement, is not correct, i have a problem to check the logfile.
    >
    > What kind of reasons can be imagined, that the method "$pid=open(H, "$
    > {cmd} |");" doesn't supply the correct pid?
    > Does anybody know about a method that gives me the correct pid?


    The problem (or one compounding problem) seems to be that Perl uses
    a shell or not to run the external program, depending on whether the
    string contains shell metacharacters. It may depend on other factors
    too.

    As has been suggested in the thread, exec() could be helpful here. For
    instance, *if* it's a perl-interpolated shell that is the problem, you
    can change the command as in $cmd = "exec $cmd". Then your command
    will run under the shell's PID.

    Anno
     
    -berlin.de, Jul 20, 2007
    #7
  8. Guest

    d (Douglas Wells) wrote:
    > In article <20070719123834.995$>,
    > writes:
    > > wrote:
    > > > i am starting an exectuable on hp-ux (ActivePerl Build 817) from a
    > > > perl script and need to know the childs pid.
    > > >
    > > > my $cmd="myexecutable -P parmfile "
    > > >
    > > > my $pid=open(H, "${cmd} |");
    > > > print "\ttest: $_" while (<H>);
    > > > close(H);
    > > >
    > > > The executable starts, but the pid i am getting is not right.

    > >
    > > I don't see that problem.
    > >
    > > $ perl -le 'print open my $fh, q{perl -le "print \$$;" |}; print
    > > <$fh>' 11050
    > > 11050

    >
    > You don't see the problem because you're not doing the same thing
    > as the original poster.


    Obviously I'm not doing the exact thing as he is, because I don't
    have access to his program "myexecutable -P parmfile". However, I am doing
    the same thing as far as Perl is concerned.

    >
    > The original poster's basic problem is that this form of open
    > (using '|') almost certainly invokes the system shell in order to
    > execute the command.


    Right. As does my example.

    And the shell executes the command. Specifically, it "exec"s the command,
    which means it doesn't change the pid. That is what my example shows, too.

    > (And in response to another comment in this thread, the PID of
    > child will almost certainly not be the shell's PID plus 1.


    I am far from certain of that. In fact, in 2 out of 3 OS I tested it on,
    after arranging for it to fork rather than exec so the pid changes, it was
    in fact the old pid plus 1.


    > Any
    > system that does that presents a major security risk and should
    > be trashed immediately -- because PIDs would then be guessable.)


    Sorry, but I have no intention of trashing my systems.

    > What you have done is to present to the shell a command (print)
    > that it can execute internally.


    No, I presented the shell with a one line Perl script. The "print" is a
    Perl print, not a shell print.

    > Thus, it does not need to create
    > a child to execute the command.


    Whether or not it *needs* to do so, that is what it actually does; strace
    verifies this.

    > Even if that weren't the case,
    > you have used a construct ($$) that gets expanded at the shell
    > level before the subprocess is invoked.


    I back-whacked the first $ for a reason. And strace verifies that
    the expansion does not occur in either of the shells, but is passed to the
    inner Perl as intended.

    > (Note you have actually
    > quoted that construct in several ways, but you have only protected
    > it against expansion in the shell that invoked perl and within
    > perl itself. To protect it against the shell that is created by
    > the open function would require several more backslashes and lots
    > of experimentation.)


    I did the experimentation. The escaping I supplied is appropriate.


    >
    > > Maybe your $cmd isn't doing what you think it is doing.
    > >
    > > Also, you might want to try using the 3-or-more argument form
    > > of open:
    > >
    > > open my $fh, "-|", "$cmd1", @cmd_args;

    >
    > That will exhibit the same problem.


    That would depend on what the actual problem is.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Jul 20, 2007
    #8
  9. In article <>, -berlin.de writes:
    > Douglas Wells <> wrote in comp.lang.perl.misc:
    >
    > > (And in response to another comment in this thread, the PID of
    > > child will almost certainly not be the shell's PID plus 1. Any
    > > system that does that presents a major security risk and should
    > > be trashed immediately -- because PIDs would then be guessable.)

    >
    > It's the other way around. Anything that relies on the non-predict-
    > ability of PIDs is insecure, unless the system guarantees that property
    > in a quantifiable way. I've never heard of that.
    >
    > Many Unix systems assign PIDs sequentially (usually skipping those
    > that have been used recently, in some sense).
    >
    > Anno


    You're partially right. Going back and checking I find that indeed
    a number, perhaps even most, POSIX/Linux systems do assign PIDs
    sequentially. So, I was wrong about that.

    But, it is still a potential security vulnerability, and high
    integrity systems don't do that. Let me explain (taking some
    liberties with terminology for purposes of simplification).

    Let's look at three levels of security-awareness in an OS environment.

    Basic Security: It's possible for a highly knowledgeable engineer
    to create an application that does not disclose information to
    unauthorized subjects.
    Example feature: Before the creation of the O_EXCL flag
    to open(2) (originally creat(2)), it was almost impossible
    to securely create a file in a shared directory.

    General Security: The standard system tools support the average
    programmer in the creation of secure applications.
    Example feature: The invention of the mkstemp(3) subroutine
    made it convenient to securely create a temporary file. It was
    possible to properly use the earlier mktemp(3) subroutine, but
    many programmers got it wrong.

    Proactive Security: The system provides mechanisms that protect
    against some design errors in applications.
    Example feature: The removal of execute access from the stack.
    This is not strictly necessary, and even disallows some
    desirable features, but it protects against a common programming
    error (buffer overflow).

    Now lets look at guessable PIDs. Note that some shell scripts
    (and even some programs) have a construct similar to:

    TMPFILE=/tmp/mycmd.$$
    (echo cmd1 ; echo cmd2 ) > $TMPFILE
    sh $TMPFILE

    If I find that a privileged process is running such a shell script,
    I can precreate the file (by guessing the PID that the shell will
    soon be using(*)). Once I have the privileged process executing
    commands from a file that I own or have write privileges I can
    easily compromise that level of privilege. While this particular
    example is fairly blatant, numerous less obvious constructs are
    equally vulnerable.

    That's why guessable PIDs are a security vulnerability. And I'll
    note that your statement that "Anything that relies on the
    non-predictability of PIDs is insecure ..." is also true. The two
    concepts are not contradictory once you accept that secureness is
    not a binary attribute.

    There's actually quite of bit of history of exploits using techniques
    similar to this. For example, the mkstemp subroutine was added
    to UNIX systems after a number of exploits or mktemp became known.
    I won't get into lots of details here as we're rapidly getting off
    topic -- although Perl scripts are subject to the same vulnerabilities
    as the shell.

    - dmw

    (*) Actually I would precreate a lot of files to increase my chances
    of guessing the right one. For example (in pseudo-code),
    for PIDnum = getpid () to getpid () + 100
    create /tmp/mycmd.PIDnum

    --
    .. Douglas Wells . Connection Technologies .
    .. Internet: -sp9804- -at - contek.com- .
     
    Douglas Wells, Jul 20, 2007
    #9
  10. -berlin.de Guest

    Douglas Wells <> wrote in comp.lang.perl.misc:
    > In article <>,
    > -berlin.de writes:
    > > Douglas Wells <> wrote in comp.lang.perl.misc:
    > >
    > > > (And in response to another comment in this thread, the PID of
    > > > child will almost certainly not be the shell's PID plus 1. Any
    > > > system that does that presents a major security risk and should
    > > > be trashed immediately -- because PIDs would then be guessable.)

    > >
    > > It's the other way around. Anything that relies on the non-predict-
    > > ability of PIDs is insecure, unless the system guarantees that property
    > > in a quantifiable way. I've never heard of that.
    > >
    > > Many Unix systems assign PIDs sequentially (usually skipping those
    > > that have been used recently, in some sense).
    > >
    > > Anno

    >
    > You're partially right. Going back and checking I find that indeed
    > a number, perhaps even most, POSIX/Linux systems do assign PIDs
    > sequentially. So, I was wrong about that.
    >
    > But, it is still a potential security vulnerability, and high
    > integrity systems don't do that. Let me explain (taking some
    > liberties with terminology for purposes of simplification).
    >
    > Let's look at three levels of security-awareness in an OS environment.


    [well known facts and concepts snipped]

    > Now lets look at guessable PIDs. Note that some shell scripts
    > (and even some programs) have a construct similar to:
    >
    > TMPFILE=/tmp/mycmd.$$
    > (echo cmd1 ; echo cmd2 ) > $TMPFILE
    > sh $TMPFILE


    Yes, that's widely known to be vulnerable. Using it means to
    rely on the un-guessability of PIDs, so it's out when security
    is a concern. There are ways to create temp files safely.

    Code that uses this construct must be considered unsafe. It doesn't
    become safe by stopping this particular hole by changing the PID
    assignment strategy. I'm not saying that PID randomization is a
    bad thing. To rely on it is.

    Anno
     
    -berlin.de, Jul 20, 2007
    #10
  11. On 2007-07-19 22:28, Douglas Wells <> wrote:
    > (And in response to another comment in this thread, the PID of
    > child will almost certainly not be the shell's PID plus 1. Any
    > system that does that presents a major security risk and should
    > be trashed immediately -- because PIDs would then be guessable.)


    PIDs are guessable on the majority of unix systems (the only exceptions
    I know are some BSD variants) - so this is something the average unix
    programmer expects.

    OTOH, a random pid is something the average unix programmer does not
    expect - which may lead to a different class of (possibly
    security-critical) errors.

    hp


    --
    _ | Peter J. Holzer | I know I'd be respectful of a pirate
    |_|_) | Sysadmin WSR | with an emu on his shoulder.
    | | | |
    __/ | http://www.hjp.at/ | -- Sam in "Freefall"
     
    Peter J. Holzer, Jul 26, 2007
    #11
  12. In article <>,
    "Peter J. Holzer" <> writes:
    > On 2007-07-19 22:28, Douglas Wells <> wrote:
    > > (And in response to another comment in this thread, the PID of
    > > child will almost certainly not be the shell's PID plus 1. Any
    > > system that does that presents a major security risk and should
    > > be trashed immediately -- because PIDs would then be guessable.)

    >
    > PIDs are guessable on the majority of unix systems (the only exceptions
    > I know are some BSD variants) - so this is something the average unix
    > programmer expects.


    Yes, elsewhere in this thread, I acknowledged that I made a wrong
    declaration about "most" POSIX-like system: Many of them do
    generate new candidates for PIDs by incrementing a counter. I
    have not, however, yielded on the claim of a security threat posed
    by this algorithm. Instead, I supplied a scenario, based on
    historical security incidents, that posed a threat in the presence
    of this algorithm.

    > OTOH, a random pid is something the average unix programmer does not
    > expect - which may lead to a different class of (possibly
    > security-critical) errors.


    I'm sorry, but I find that (the security-critical error possibility)
    preposterous. Can you provide a scenario that both leads to correct
    behavior in the presence of the incrementing PID algorithm and to
    the presence of a security threat in its absence?

    In fact, I'd like to hear of a scenario that leads to non-probabilistic
    correct behavior in the presence of an application-level program
    that predicts future PIDs based on any algorithm. Can you provide
    one?

    I can imagine a frazzled human debugging a multi-process program,
    operating in a benign environment, and choosing to attach the
    the "next" PID based on a mental calculation. But, I fail to
    understand how that could work reasonably in a deployed system.

    It appears to me any user-level algorithm for predicting the "next"
    PID, such as adding one to the PID of the current process, faces
    several difficulties:

    - The system could well have created a non-related process with
    the "next" PID in the meanwhile.

    - The current sequence may have reached the maximum PID value and
    wrap around. Neither the maximum value nor the initial value
    after wrap-around is specified by the POSIX standard (nor by
    the Linux standard, which seems to defer to POSIX in this
    instance).

    - Even if neither of those events has occurred, the "next" PID
    might not be usable due to the existence of a long-lived process
    with that ID, and the standard's requirement that PIDs be
    temporally unique.

    - Even if there isn't an existing process with the same PID,
    that PID value might not be usable due to the standard's
    "Process ID Reuse" prohibition on reuse of process group ids.

    Can you offer an user-level algorithm that alleviates the effect
    of those problems?

    Also, do you actually know of any interesting applications that
    would significantly benefit by knowing the PID of a process that
    has yet to be created?

    > hp


    - dmw

    --
    .. Douglas Wells . Connection Technologies .
    .. Internet: -sp9804- -at - contek.com- .
     
    Douglas Wells, Jul 26, 2007
    #12
  13. Peter J. Holzer wrote:

    > OTOH, a random pid is something the average unix programmer does not
    > expect - which may lead to a different class of (possibly
    > security-critical) errors.


    Sorry, but I have never seen a piece of (*x-) software that relies on
    some relation between PIDs. It will break as soon as you have a wrap-around.

    Getting your own PID is a very simple call, getting your child's pid a
    result of the fork() system call. I don't see how you can put any
    constraint on one or the other to make it appear non-random!

    Josef
    --
    These are my personal views and not those of Fujitsu Siemens Computers!
    Josef Möllers (Pinguinpfleger bei FSC)
    If failure had no penalty success would not be a prize (T. Pratchett)
    Company Details: http://www.fujitsu-siemens.com/imprint.html
     
    Josef Moellers, Jul 27, 2007
    #13
  14. On 2007-07-26 22:57, Douglas Wells <> wrote:
    > In article <>,
    > "Peter J. Holzer" <> writes:
    >> On 2007-07-19 22:28, Douglas Wells <> wrote:
    >> > (And in response to another comment in this thread, the PID of
    >> > child will almost certainly not be the shell's PID plus 1. Any
    >> > system that does that presents a major security risk and should
    >> > be trashed immediately -- because PIDs would then be guessable.)

    >>
    >> PIDs are guessable on the majority of unix systems (the only exceptions
    >> I know are some BSD variants) - so this is something the average unix
    >> programmer expects.

    >
    > Yes, elsewhere in this thread, I acknowledged that I made a wrong
    > declaration about "most" POSIX-like system: Many of them do
    > generate new candidates for PIDs by incrementing a counter. I
    > have not, however, yielded on the claim of a security threat posed
    > by this algorithm. Instead, I supplied a scenario, based on
    > historical security incidents, that posed a threat in the presence
    > of this algorithm.


    Yes. If you had yielded that claim I wouldn't have had any reason to
    answer. But since you haven't, I had to object.

    The pid has traditionally always been a simple wrap-around counter. Any
    unix programmer should know this. Using it in a context where a random
    number (much less a cryptographically strong random number) is required
    is just using the wrong tool for the job. Such an error may lead to a
    security problem, but that's the fault of the programmer, not the tool.
    (In your scenario, the real error is probably not using O_EXCL, btw, not
    using the pid, but that depends on the intended use).


    >> OTOH, a random pid is something the average unix programmer does not
    >> expect - which may lead to a different class of (possibly
    >> security-critical) errors.

    >
    > I'm sorry, but I find that (the security-critical error possibility)
    > preposterous. Can you provide a scenario that both leads to correct
    > behavior in the presence of the incrementing PID algorithm and to
    > the presence of a security threat in its absence?


    Yes. A linearly incrementing pid has some minimum time between the start
    time of two processes with the same pid. Typically, about 30000 forks
    are needed before the same pid can be reused.

    A programmer which knows this may assume that the start time of the
    process with suitably high resolution (for example 1 millisecond - 1
    second is already too grainy given current computer speeds) together
    with the pid is always unique: The system would need to be able to fork
    30000 processes within 1 ms, which is far beyond the capabilities of
    current systems and will stay impossible for some time.

    A randomized pid breaks this assumption: The pid may be (with some small
    probability, but nonetheless) reused immediately. Thus it is possible
    that two processes which are started in the same millisecond get the
    same "unique" id. What happens then depends on the application: Maybe
    nothing, maybe some data gets mixed up, maybe some data vanishes ...

    I know a number of applications which didn't work correctly on BSD
    systems after randomized pids were introduced. The results were usually
    lost data or data leaked to a different user, so that was at least
    potentially security-critical.

    Personally, I think it's a good thing that these applications were
    broken, because it alerted the maintainers that the assumption
    "timestamp + pid is unique" was faulty way before it became faulty on
    systems on which it may have been possible to systematically exploit the
    bug (IIRC all these applications used a one-second timestamp which is
    now getting too short).

    I'm not saying that random pids are bad /per se/. But the average unix
    programmer probably doesn't know that they exist so he cannot consider
    their consequences.

    (As an aside: Is the randomness of the pids on BSD systems
    cryptographically strong? If not, a programmer might assume they are and
    make the same error as a programmer who thinks that linearly incremented
    pids are "non-predictable". If they are, how about other systems with
    random pids?)


    [rest of posting snipped, as it was completely beside the point I was
    trying to make]

    hp


    --
    _ | Peter J. Holzer | I know I'd be respectful of a pirate
    |_|_) | Sysadmin WSR | with an emu on his shoulder.
    | | | |
    __/ | http://www.hjp.at/ | -- Sam in "Freefall"
     
    Peter J. Holzer, Jul 27, 2007
    #14
  15. On 2007-07-27 07:38, Josef Moellers <> wrote:
    > Peter J. Holzer wrote:
    >
    >> OTOH, a random pid is something the average unix programmer does not
    >> expect - which may lead to a different class of (possibly
    >> security-critical) errors.

    >
    > Sorry, but I have never seen a piece of (*x-) software that relies on
    > some relation between PIDs.


    See for example the original definition of the maildir format. Any
    software which still uses that format is broken on systems which can
    reuse the same pid within one second.

    > It will break as soon as you have a wrap-around.


    No. That is expected and handled. The relation which the application
    relies on is "two processes which are alive within the same second
    cannot have the same pid".

    This assumption is of course faulty anyway (there may already be systems
    which can fork 30000 times per second, and it may be possible to delay a
    process before generating the timestamp on slower systems), but the fact
    is that the first systems where this problem was noticed (with real lost
    mail, not just as a theoretical possibility) were systems with
    randomized pids - because it is many orders of magnitude more likely to
    happen on such systems.

    hp


    --
    _ | Peter J. Holzer | I know I'd be respectful of a pirate
    |_|_) | Sysadmin WSR | with an emu on his shoulder.
    | | | |
    __/ | http://www.hjp.at/ | -- Sam in "Freefall"
     
    Peter J. Holzer, Jul 27, 2007
    #15
  16. Peter J. Holzer wrote:
    > On 2007-07-27 07:38, Josef Moellers <> wrote:
    >
    >>Peter J. Holzer wrote:
    >>
    >>
    >>>OTOH, a random pid is something the average unix programmer does not
    >>>expect - which may lead to a different class of (possibly
    >>>security-critical) errors.

    >>
    >>Sorry, but I have never seen a piece of (*x-) software that relies on
    >>some relation between PIDs.

    >
    >
    > See for example the original definition of the maildir format. Any
    > software which still uses that format is broken on systems which can
    > reuse the same pid within one second.


    OK, I see. So it's not really the fact that PIDs are supposed to be
    monotonic increasing (apart from one place), but the fact that, when
    using random PIDs, a PID can be re-used before all the other PIDs were
    (re-)used. Indeed, with random PIDs, it would not be impossible for a
    process to get the PID of a process which has just died.
    OTOH it would work if one could guarantee that a recently deceased
    process' PID will be the last PID (re-)used of all available PIDs at
    that moment, e.g. by putting PIDs into a FIFO.

    I stand corrected,

    Josef
    --
    These are my personal views and not those of Fujitsu Siemens Computers!
    Josef Möllers (Pinguinpfleger bei FSC)
    If failure had no penalty success would not be a prize (T. Pratchett)
    Company Details: http://www.fujitsu-siemens.com/imprint.html
     
    Josef Moellers, Jul 27, 2007
    #16
  17. In article <>, "Peter J. Holzer" <> writes:
    > On 2007-07-26 22:57, Douglas Wells <> wrote:
    > > In article <>,
    > > "Peter J. Holzer" <> writes:
    > >> PIDs are guessable on the majority of unix systems (the only exceptions
    > >> I know are some BSD variants) - so this is something the average unix
    > >> programmer expects.

    > >
    > > Yes, elsewhere in this thread, I acknowledged that I made a wrong
    > > declaration about "most" POSIX-like system: Many of them do
    > > generate new candidates for PIDs by incrementing a counter. I
    > > have not, however, yielded on the claim of a security threat posed
    > > by this algorithm. Instead, I supplied a scenario, based on
    > > historical security incidents, that posed a threat in the presence
    > > of this algorithm.

    >
    > Yes. If you had yielded that claim I wouldn't have had any reason to
    > answer. But since you haven't, I had to object.


    And I still don't. You are describing a situation where an
    application programmer makes an erroneous assumption, and that
    assumption results in a security problem.

    > The pid has traditionally always been a simple wrap-around counter. Any
    > unix programmer should know this.


    That depends on your definition of "traditionally." Yes, the
    standard code base from AT&T did this, and many of the derivative
    systems also did this. However, that is irrelevant, as it was not
    guaranteed by either the system specification or a published
    standard. (And, I know of systems at least as far back as 1982 that
    used non-sequential PIDs.). A programmer may know that this is the
    way that things often work, but he/she shouldn't depend on something
    that isn't *documented* as a permanent characteristic of the
    system. And, if you are targeting a portable platform, such as
    POSIX, you need to be aware of possible differences across
    implementations of that platform.

    > Using it in a context where a random
    > number (much less a cryptographically strong random number) is required
    > is just using the wrong tool for the job. Such an error may lead to a
    > security problem, but that's the fault of the programmer, not the tool.
    > (In your scenario, the real error is probably not using O_EXCL, btw, not
    > using the pid, but that depends on the intended use).


    Yes, *one* underlying error was not using O_EXCL -- as I implied
    by my reference to O_EXCL in that post.

    As decoded in a parallel sub-thread, it would appear that a real
    problem underlying your concern was a dependence on a time delay
    before reuse of PIDs. That characteristic, of course, is not
    currently a guarantee of the POSIX/UNIX standard, and I can find
    no instances of any system documentation promising that.

    The question we have before us seems to be how to attribute the
    blame of being a security flaw. Consider the case of using an
    uninitialized variable in a program. It can be the case that that
    variable happens to inherit a value that causes the the program
    to behave reasonably. As time elapses that accidental value can
    change for any number of reasons: the level of optimization changes,
    the compiler changes, other code in the program changes and happens
    to leave a different value in that location, or the program is
    moved to a different platform.

    When that program later misbehaves, leaving a loss of data (and a
    security problem using your definition), one shouldn't point to
    the new compiler or the different platform as the security problem.
    The security flaw is the improper coding in the original program.

    In the case of the example that you provide in a parallel sub-thread,
    the basic security flaw is clearly at the hands of the designers
    of the maildir format (assuming your description is correct, which
    I don't doubt). They made an assumption that wasn't true across
    the targeted platforms.

    Security experts, for both computers and otherwise, note that
    vulnerability is the product of the existence of security
    vulnerabilities and the presence of security threats. In this
    case, the introduction of a system with randomized PIDs increased
    the level of security threat, but that doesn't mean that the use
    of randomized PIDs is itself a security threat, which is an
    implication, but not explicit claim, of your earlier message to
    which I was reacting.

    So, with that said, let me react to the rest of your message:

    > Yes. A linearly incrementing pid has some minimum time between the start
    > time of two processes with the same pid. Typically, about 30000 forks
    > are needed before the same pid can be reused.
    >
    > A programmer which knows this may assume that the start time of the
    > process with suitably high resolution (for example 1 millisecond - 1
    > second is already too grainy given current computer speeds) together
    > with the pid is always unique: The system would need to be able to fork
    > 30000 processes within 1 ms, which is far beyond the capabilities of
    > current systems and will stay impossible for some time.


    Right, and that programmer is creating a bug by depending on a
    characteristic that is not guaranteed by any spec.

    > I know a number of applications which didn't work correctly on BSD
    > systems after randomized pids were introduced. The results were usually
    > lost data or data leaked to a different user, so that was at least
    > potentially security-critical.


    Agreed, and the security problem was in the application. The
    introduction of the use of the "BSD" system (I'm presuming that
    this was likely OpenBSD) increased the security threat by altering
    more characteristics of the platform.

    > Personally, I think it's a good thing that these applications were
    > broken, because it alerted the maintainers that the assumption
    > "timestamp + pid is unique" was faulty way before it became faulty on
    > systems on which it may have been possible to systematically exploit the
    > bug (IIRC all these applications used a one-second timestamp which is
    > now getting too short).
    >
    > I'm not saying that random pids are bad /per se/. But the average unix
    > programmer probably doesn't know that they exist so he cannot consider
    > their consequences.


    OK, and the way to deal with that is to teach those programmers
    how to write portable applications by not depending on accidental
    characteristics.

    > (As an aside: Is the randomness of the pids on BSD systems
    > cryptographically strong? If not, a programmer might assume they are and
    > make the same error as a programmer who thinks that linearly incremented
    > pids are "non-predictable". If they are, how about other systems with
    > random pids?)


    I don't know about those particular systems, but often such algorithms
    are pluggable and can be replaced depending upon the security
    requirements of the deployed environment. Given that the rate of
    production of PIDs is slower, and the visibility of PIDs is lower
    than communication data, the tolerable level of strength would be
    lower.

    > hp


    - dmw

    --
    .. Douglas Wells . Connection Technologies .
    .. Internet: -sp9804- -at - contek.com- .
     
    Douglas Wells, Jul 27, 2007
    #17
  18. [I'm sorry, this posting grew too long and quite repetitive. But it's
    rather late and I'm too lazy now to trim it down. So bear with me or
    feel free to skip it]

    On 2007-07-27 17:19, Douglas Wells <> wrote:
    > In article <>, "Peter J. Holzer" <> writes:
    >> On 2007-07-26 22:57, Douglas Wells <> wrote:
    >> > In article <>,
    >> > "Peter J. Holzer" <> writes:
    >> >> PIDs are guessable on the majority of unix systems (the only exceptions
    >> >> I know are some BSD variants) - so this is something the average unix
    >> >> programmer expects.
    >> >
    >> > Yes, elsewhere in this thread, I acknowledged that I made a wrong
    >> > declaration about "most" POSIX-like system: Many of them do
    >> > generate new candidates for PIDs by incrementing a counter. I
    >> > have not, however, yielded on the claim of a security threat posed
    >> > by this algorithm. Instead, I supplied a scenario, based on
    >> > historical security incidents, that posed a threat in the presence
    >> > of this algorithm.

    >>
    >> Yes. If you had yielded that claim I wouldn't have had any reason to
    >> answer. But since you haven't, I had to object.

    >
    > And I still don't. You are describing a situation where an
    > application programmer makes an erroneous assumption, and that
    > assumption results in a security problem.


    Same as you did. The assumption that a pid is unguessable is just as
    erroneous, and it is very simple to refute on almost any Unix system. So
    it is a problem that the average Unix programmer should be aware of and
    he can take appropriate measures (which he still has to do even if his
    system uses randomized pids: a) because he cannot rely on this feature
    and b) because an attacker may still guess the pid because of its
    limited range (typically 30k). So I stand by my claim that sequentially
    allocated PIDs are not a "major security threat", and I would even guess
    that randomized pids add very little security. (The grsecurity people
    seem to agree - randomized PIDs were removed again)


    >> The pid has traditionally always been a simple wrap-around counter. Any
    >> unix programmer should know this.

    >
    > That depends on your definition of "traditionally." Yes, the
    > standard code base from AT&T did this, and many of the derivative
    > systems also did this. However, that is irrelevant, as it was not
    > guaranteed by either the system specification or a published
    > standard.


    You are right that the standard doesn't specify this, and the printed
    docs I still have for PC/IX (which are probably quite close to those of
    Version 7 Unix) don't specify it either.

    However, as you say, AT&T Unix did it this way and the vast majority of
    derivatives and clones followed this example. This behaviour was also
    documented in text books over and over (for example in Banahan and
    Rutter: UNIX - the Book (1982), which happens to be the book I learned
    Unix from together with the aforementioned PC/IX manpages 20 years ago).
    So if a programmer finds some behaviour documented and all systems he
    has access to behave that way, why should he suspect that his books are
    wrong? (Because all textbooks contain errors and should not be used as
    reference, you say, and you are of course entirely correct)


    > (And, I know of systems at least as far back as 1982 that
    > used non-sequential PIDs.).


    I don't doubt this, although the first system I was aware of using that
    feature was OpenBSD (since at least 1999). I do think that these systems
    were rather obscure, though, and most unix programmers were not aware of
    their existence.

    > A programmer may know that this is the way that things often work, but
    > he/she shouldn't depend on something that isn't *documented* as a
    > permanent characteristic of the system.


    Unfortunately, while it certainly wasn't documented in POSIX and
    probably wasn't documented in any specific system documentations
    (although it is quite possible that some unixes did document their pid
    generation scheme), it *was* "documented" in lots of secondary
    literature, and programmers would pick up the wrong assumption from
    there.


    >> Using it in a context where a random
    >> number (much less a cryptographically strong random number) is required
    >> is just using the wrong tool for the job. Such an error may lead to a
    >> security problem, but that's the fault of the programmer, not the tool.
    >> (In your scenario, the real error is probably not using O_EXCL, btw, not
    >> using the pid, but that depends on the intended use).

    >
    > Yes, *one* underlying error was not using O_EXCL -- as I implied
    > by my reference to O_EXCL in that post.
    >
    > As decoded in a parallel sub-thread, it would appear that a real
    > problem underlying your concern was a dependence on a time delay
    > before reuse of PIDs.


    My "concern" is not that randomized pids are somehow worse than
    sequential pids. My "concern" is your claim that sequential pids are a
    major security threat and that all systems using it should be trashed.

    I am pointing out that unix programmers know that pids can be guessed,
    so this is not a major security threat.

    I am further pointing out that unix programmers very often do not know
    that sequential pids are not universal. So the slight advantage of
    having pids which are slightly harder to guess is offset by behaviour
    which is not expected by the programmer. Of course it's the programmer's
    fault if he assumes that pids are assigned sequentially. But it is also
    his fault if he creates a file in /tmp without using O_EXCL and checking
    the the return value. I fail to see why one should be a "major security
    threat of the system" and the other the programmer's fault. Failure to
    deal correctly with sequential PIDs is just as much the programmer's
    fault as failure to deal with randomized PIDs. In fact one could argue
    that the former is more the programmers fault, since it is the usual
    case.


    >> Personally, I think it's a good thing that these applications were
    >> broken, because it alerted the maintainers that the assumption
    >> "timestamp + pid is unique" was faulty way before it became faulty on
    >> systems on which it may have been possible to systematically exploit the
    >> bug (IIRC all these applications used a one-second timestamp which is
    >> now getting too short).
    >>
    >> I'm not saying that random pids are bad /per se/. But the average unix
    >> programmer probably doesn't know that they exist so he cannot consider
    >> their consequences.

    >
    > OK, and the way to deal with that is to teach those programmers
    > how to write portable applications by not depending on accidental
    > characteristics.


    Right. Just as we have to teach programmers to use File::Temp for
    temporary files (or use lower level functions properly). The
    non-existence of "/tmp/$progname.$$" is an accidental characteristic
    which a programmer must not rely on. Not for portable applications and
    not even for non-portable ones.


    >> (As an aside: Is the randomness of the pids on BSD systems
    >> cryptographically strong? If not, a programmer might assume they are and
    >> make the same error as a programmer who thinks that linearly incremented
    >> pids are "non-predictable". If they are, how about other systems with
    >> random pids?)

    >
    > I don't know about those particular systems, but often such algorithms
    > are pluggable and can be replaced depending upon the security
    > requirements of the deployed environment. Given that the rate of
    > production of PIDs is slower, and the visibility of PIDs is lower
    > than communication data, the tolerable level of strength would be
    > lower.


    My point here is that a programmer cannot rely on PIDs being
    unguessable. A program which relies on unguessable PIDs is just as buggy
    on a system with random PIDs as it is on a system with sequential PIDs.
    A programmer may be lured into a false sense of security by apparently
    random pids which aren't. Even if the pids on his system are strong
    (lets for a moment assume that the system uses 64 bit numbers from a
    sufficiently large entropy pool), the program may be ported to another
    system which maybe uses only 15 bit numbers from a 32-bit linear
    congruental PRNG, or, worse, to a system with sequential PIDs.

    hp


    --
    _ | Peter J. Holzer | I know I'd be respectful of a pirate
    |_|_) | Sysadmin WSR | with an emu on his shoulder.
    | | | |
    __/ | http://www.hjp.at/ | -- Sam in "Freefall"
     
    Peter J. Holzer, Jul 28, 2007
    #18
  19. In article <>,
    "Peter J. Holzer" <> writes:
    >
    > [ ... ] The assumption that a pid is unguessable is just as
    > erroneous, and it is very simple to refute on almost any Unix system. So
    > it is a problem that the average Unix programmer should be aware of and
    > he can take appropriate measures (which he still has to do even if his
    > system uses randomized pids: a) because he cannot rely on this feature
    > and b) because an attacker may still guess the pid because of its
    > limited range (typically 30k). So I stand by my claim that sequentially
    > allocated PIDs are not a "major security threat", and I would even guess
    > that randomized pids add very little security. (The grsecurity people
    > seem to agree - randomized PIDs were removed again)


    Ahh, I see that you were reacting to the words "major security
    threat." I don't see where you made that point explicitly in the
    earlier messages, and I didn't understand that it was one of your
    contention points. So, let me back off on the "major" adjective.
    I over-reacted to the earlier proposal that a programmer should
    depend on the PID generation algorithm of a particular system.

    > You are right that the standard doesn't specify this, and the printed
    > docs I still have for PC/IX (which are probably quite close to those of
    > Version 7 Unix) don't specify it either.
    >
    > However, as you say, AT&T Unix did it this way and the vast majority of
    > derivatives and clones followed this example. This behaviour was also
    > documented in text books over and over (for example in Banahan and
    > Rutter: UNIX - the Book (1982), which happens to be the book I learned
    > Unix from together with the aforementioned PC/IX manpages 20 years ago).
    > So if a programmer finds some behaviour documented and all systems he
    > has access to behave that way, why should he suspect that his books are
    > wrong? (Because all textbooks contain errors and should not be used as
    > reference, you say, and you are of course entirely correct)


    OK. I think that we are in agreement on that. It seems to be a
    characteristic of many humans that it's easiest to learn from an
    example. Unfortunately, it also seems to be a trait that we make
    assumptions and improperly extrapolate the specific to the general.

    I have found 4 books in my library that explain the creation of
    PIDs (Lyons; Leffler, McKusick, et al.; Bach; and Tanenbaum). All
    explain the use of an incrementing counter; none provide any
    assurance that the algorithm in guaranteed. However, I can
    understand how someone relatively new to UNIX would assume that
    this was the only way. (I'll also that there are differences in
    the algorithms and none quite matches the use in current systems
    that I am familiar with.)

    > My "concern" is not that randomized pids are somehow worse than
    > sequential pids. My "concern" is your claim that sequential pids are a
    > major security threat and that all systems using it should be trashed.


    I have backed off on the particular claim of a "major threat. I
    also noted in my earlier explanation that programs were no vulnerable
    if they properly used O_EXCL in system calls, "no clobber" in the
    shell, and mkstemp in programs.

    > I am pointing out that unix programmers know that pids can be guessed,
    > so this is not a major security threat.


    Unfortunately, I have to disagree with this assertion. As an
    exemplar, I will point out that the Perl documentation for "open"
    contains the faulty construct, namely: open(EXTRACT, "|sort >Tmp$$")
    (Granted this example doesn't necessarily put the file into a
    shared directory, but it also doesn't warn about the situation.)

    > Right. Just as we have to teach programmers to use File::Temp for
    > temporary files (or use lower level functions properly). The
    > non-existence of "/tmp/$progname.$$" is an accidental characteristic
    > which a programmer must not rely on. Not for portable applications and
    > not even for non-portable ones.


    I have to agree with that.

    > My point here is that a programmer cannot rely on PIDs being
    > unguessable. A program which relies on unguessable PIDs is just as buggy
    > on a system with random PIDs as it is on a system with sequential PIDs.
    > A programmer may be lured into a false sense of security by apparently
    > random pids which aren't. Even if the pids on his system are strong
    > (lets for a moment assume that the system uses 64 bit numbers from a
    > sufficiently large entropy pool), the program may be ported to another
    > system which maybe uses only 15 bit numbers from a 32-bit linear
    > congruental PRNG, or, worse, to a system with sequential PIDs.


    I wasn't attempting to make that point, and, if fact, I was trying
    to make the same point as you are here, so I think that we agree
    on this.

    > hp


    So, let me try a refined assertion:

    - The use of guessable PIDs leads to a higher level of security
    threat on a system, most notably:

    - in the presence of faulty programs that create files in
    shared directories where the names depend on process ids.

    - in certain aspects of high-assurance systems that we haven't
    discussed here.

    - dmw

    --
    .. Douglas Wells . Connection Technologies .
    .. Internet: -sp9804- -at - contek.com- .
     
    Douglas Wells, Jul 28, 2007
    #19
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Sigal

    Process PID

    Sigal, Feb 18, 2004, in forum: Perl
    Replies:
    1
    Views:
    721
    Sigal
    Feb 18, 2004
  2. Replies:
    1
    Views:
    1,597
    Victor Bazarov
    Mar 3, 2008
  3. P.S.
    Replies:
    0
    Views:
    354
  4. Miki Tebeka
    Replies:
    2
    Views:
    634
    Chris Angelico
    Apr 8, 2011
  5. Jakobus Schuerz

    Problems with Popup-Windows startet with Java

    Jakobus Schuerz, Jul 25, 2006, in forum: Javascript
    Replies:
    0
    Views:
    102
    Jakobus Schuerz
    Jul 25, 2006
Loading...

Share This Page