java, multithreading, fork() - performance problem on Solaris

Discussion in 'Java' started by bugbear, Mar 25, 2008.

  1. bugbear

    bugbear Guest

    I'm using tomcat on Solaris with around 100 threads.
    This is perfectly normal and mundane, and works well.

    However, I have now noticed that when the JSP's
    call system() (either via runtime.exec or new processBuilder)
    the whole thing slows down a lot, and the kernel
    usage goes crazy. Idle time goes UP (which is odd).
    JSP response tanks.

    No, I can't stop shelling out from tomcat :-(

    I have traced this problem to fork(), to whit:

    The fork() and fork1() functions suspend
    all threads in the process before proceeding.

    (ref:
    http://docs.sun.com/app/docs/doc/816-0212/6m6nd4n9e?a=view
    )

    They're all started up again after the fork(), of course, and
    I have confirmed this diagnosis by crawling over an ENORMOUS
    truss output log.

    Now, with 100 tomcat threads, this is a bit of an overhead (*)

    It appears that multithreaded Java and fork()
    on Solaris have a rather deep incompatibility.

    Has any body else seen/experienced this, and
    has anybody found a solution/workround?

    BugBear

    (*) British understatement.
     
    bugbear, Mar 25, 2008
    #1
    1. Advertising

  2. bugbear

    Tim Bradshaw Guest

    Re: java, multithreading, fork() - performance problem on Solaris

    On Mar 25, 3:30 pm, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:

    > Has any body else seen/experienced this, and
    > has anybody found a solution/workround?


    It might be that posix_spawn has different behaviour, though this
    seems unlikely to me. Otherwise the obvious thing would be to have a
    separate, single-threaded, process which does the forks to which your
    huge JVM can talk using some IPC mechanism.
     
    Tim Bradshaw, Mar 26, 2008
    #2
    1. Advertising

  3. Re: java, multithreading, fork() - performance problem on Solaris

    Tim Bradshaw <> writes:

    >On Mar 25, 3:30 pm, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:


    >> Has any body else seen/experienced this, and
    >> has anybody found a solution/workround?


    >It might be that posix_spawn has different behaviour, though this
    >seems unlikely to me. Otherwise the obvious thing would be to have a
    >separate, single-threaded, process which does the forks to which your
    >huge JVM can talk using some IPC mechanism.


    System and posix_spawn both use vfork(); vfork() also needs to
    suspend all threads, pretty much by definition.

    Note that fork/fork1 are expensive in requiring all page
    table entries for the process to be marked "copy on write".

    vfork() is generally cheap because it only needs to suspend the
    parent (but that is quite different when the parent has 100s
    of threads). (While vfork() is documented as not being thread
    safe, the restrictions on its use are known and fixed so we
    do know how to safely use it in the implementation of system(),
    popen() and posix_spawn)


    The fork() process creation model and the POSIX threads in a single
    address space model don't really work well together and the
    combination should be avoided.

    Casper
    --
    Expressed in this posting are my opinions. They are in no way related
    to opinions held by my employer, Sun Microsystems.
    Statements on Sun products included here are not gospel and may
    be fiction rather than truth.
     
    Casper H.S. Dik, Mar 26, 2008
    #3
  4. bugbear

    bugbear Guest

    Re: java, multithreading, fork() - performance problem on Solaris

    Tim Bradshaw wrote:
    > On Mar 25, 3:30 pm, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:
    >
    >> Has any body else seen/experienced this, and
    >> has anybody found a solution/workround?

    >
    > It might be that posix_spawn has different behaviour, though this
    > seems unlikely to me. Otherwise the obvious thing would be to have a
    > separate, single-threaded, process which does the forks to which your
    > huge JVM can talk using some IPC mechanism.


    Yes, thank you.

    That solution is quite natural, but would complicate
    out software architecture (and hence deployment).

    I was hoping to avoid such a solution if possible...
    (which it may not be :-( )

    It appears that (in theory) the problem
    is independant of Java; any strongly multi threaded
    code would have "issues" with shell out.

    It's just that Java tends to *be* strongly
    multi-threaded.

    BugBear
     
    bugbear, Mar 26, 2008
    #4
  5. bugbear

    Arne Vajhøj Guest

    bugbear wrote:
    > I'm using tomcat on Solaris with around 100 threads.
    > This is perfectly normal and mundane, and works well.
    >
    > However, I have now noticed that when the JSP's
    > call system() (either via runtime.exec or new processBuilder)
    > the whole thing slows down a lot, and the kernel
    > usage goes crazy. Idle time goes UP (which is odd).
    > JSP response tanks.
    >
    > No, I can't stop shelling out from tomcat :-(


    [other have commented on the fork part]

    You may need to run external commands, but could
    you instead of starting new processes have a
    pool of permanent processes you just communicated
    with ?

    Arne
     
    Arne Vajhøj, Mar 28, 2008
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. thomhashi
    Replies:
    2
    Views:
    843
    Nils O. =?iso-8859-1?Q?Sel=E5sdal?=
    Oct 31, 2003
  2. Eric Snow

    os.fork and pty.fork

    Eric Snow, Jan 8, 2009, in forum: Python
    Replies:
    0
    Views:
    604
    Eric Snow
    Jan 8, 2009
  3. mk
    Replies:
    1
    Views:
    317
    Oktaka Com
    Jan 18, 2010
  4. Naveen Reddy

    MultiThreading and 'fork'

    Naveen Reddy, Jan 25, 2004, in forum: Perl Misc
    Replies:
    9
    Views:
    119
    Ilya Zakharevich
    Jan 27, 2004
  5. Replies:
    16
    Views:
    567
Loading...

Share This Page