reducing Java program startup time

Discussion in 'Java' started by neuneudr@yahoo.fr, Jan 25, 2007.

  1. Guest

    Hi all,

    I'm using small Java utilities called from scripts (mainly
    shell scripts on Unix platforms).

    One example would be something like:

    find somedirectory -iname "*txt" -exec java -jar myprog.jar {} \;

    Which could result in hundreds of call to the Java program.

    JVM startup times have kept going down... And Java 1.6 is
    continuing to show this (very good) trend.

    Still... The JVM startup time are killing the perfs here.

    Taking into account the fact that I want to continue using
    these Java utilities (that is: I won't rewrite them in any
    other language), is there any way to speed up JVM startup
    time?

    Particularly, I was thinking of the following: could I launch
    one daemon-like Java process that would always be running
    and that would take care of launching the various jars
    when needed?

    Would that help to prevent a new JVM startup everytime a
    new .jar is launched?

    So is it possible to have one (say one per user) JVM "always
    on" and run new jars inside that JVM?

    And the example would become, say:

    find somedirectory -iname "*txt" -exec
    littleUtilThatNotifiesMyJavaDaemon myprog.jar {} \;

    That 'littleUtilThatNotifiesMyJavaDaemon' being "something" (you tell
    me :)
    that tells the running JVM that it should execute a new .jar.

    I seems possible to do such a thing, but would this offer any
    significant speedup?

    I guess my question boils down to: if you run a .jar from another
    Java program, can this be made to be faster than simply doing
    a new "java -jar ...".

    Any info greatly appreciated,

    Driss
     
    , Jan 25, 2007
    #1
    1. Advertising

  2. <> wrote in message
    news:...
    > ...
    >
    > Particularly, I was thinking of the following: could I launch
    > one daemon-like Java process that would always be running
    > and that would take care of launching the various jars
    > when needed?


    Yes.

    > Would that help to prevent a new JVM startup everytime a
    > new .jar is launched?


    Class loading time is probably more important, and you will
    save that.

    > So is it possible to have one (say one per user) JVM "always
    > on" and run new jars inside that JVM?


    Yes.

    > I seems possible to do such a thing, but would this offer any
    > significant speedup?


    Yes.

    > I guess my question boils down to: if you run a .jar from another
    > Java program, can this be made to be faster than simply doing
    > a new "java -jar ...".


    You may not see much benefit the first time the classes
    are loaded, but after that you will.
     
    Larry Barowski, Jan 25, 2007
    #2
    1. Advertising

  3. Guest

    On 25 jan, 17:01, "Larry Barowski"
    <MElarrybar-AT-eng_DOT_auburnANOTHERDOTeduEND> wrote:
    ....
    > > Would that help to prevent a new JVM startup everytime a
    > > new .jar is launched?

    > Class loading time is probably more important, and you
    > will save that.


    Can you elaborate a little bit on this? Will I save on class loading
    time for the classes used by the Java utility I'll end up calling a
    lot?

    Or on class loading time for the usual classes like String, etc.?

    Or both?


    > > I guess my question boils down to: if you run a .jar from another
    > > Java program, can this be made to be faster than simply doing
    > > a new "java -jar ...".


    > You may not see much benefit the first time the classes
    > are loaded, but after that you will.


    Great, where do I start?

    :)

    Seriously though, do you seen any pitfalls I shouldn't fall if I start
    trying to program that? Would I be reinventing the wheel?

    Thanks for your super-fast answer,

    Driss
     
    , Jan 25, 2007
    #3
  4. <> wrote in message
    news:...
    > Can you elaborate a little bit on this? Will I save on class loading
    > time for the classes used by the Java utility I'll end up calling a
    > lot?
    >
    > Or on class loading time for the usual classes like String, etc.?
    >
    > Or both?


    Both.

    > Great, where do I start?


    First you might run some tests to see how much of a problem
    startup time really is. Of course, you may already know this.

    Single-instance mechanisms that use socket communication are
    easy to find. These are applications where the client just tells the
    server to pop the main window to the top or open a file. You
    could use this same method for client-server communication,
    or you could use platform-dependent methods through jni. I've
    done this both ways.

    > Seriously though, do you seen any pitfalls I shouldn't fall if I start
    > trying to program that? Would I be reinventing the wheel?


    Do some searching and you may be able to find such a system
    (and wait to see if anyone suggests one here). If you roll your
    own and have a small number of jars and commands that are
    known in advance, it may be simpler to hard-code them rather
    than creating a general-purpose system that uses reflection. In
    either case you may want to prime the pump by executing a
    few commonly used packages/methods when the server starts.
     
    Larry Barowski, Jan 25, 2007
    #4
  5. In article <>, writes:
    > I'm using small Java utilities called from scripts (mainly
    > shell scripts on Unix platforms).
    >
    > One example would be something like:
    > find somedirectory -iname "*txt" -exec java -jar myprog.jar {} \;
    >
    > Which could result in hundreds of call to the Java program.
    >
    > JVM startup times have kept going down... And Java 1.6 is
    > continuing to show this (very good) trend.
    >
    > Taking into account the fact that I want to continue using
    > these Java utilities (that is: I won't rewrite them in any
    > other language), is there any way to speed up JVM startup
    > time?
    >
    > Particularly, I was thinking of the following: could I launch
    > one daemon-like Java process that would always be running
    > and that would take care of launching the various jars
    > when needed?
    >
    > And the example would become, say:
    >
    > find somedirectory -iname "*txt" -exec
    > littleUtilThatNotifiesMyJavaDaemon myprog.jar {} \;
    >
    > I guess my question boils down to: if you run a .jar from another
    > Java program, can this be made to be faster than simply doing
    > a new "java -jar ...".
    >
    > Driss


    As noted elsewhere, you could certainly do that, but how about
    something a bit less invasive?

    Could you instead convert your programs to accept multiple parameters,
    a la. Convert:
    public static void main (String [] args)
    {
    String arg = args [1];
    dofunction (arg);
    }
    to:
    public static void main (String [] args)
    {
    for (String arg : args)
    dofunction (arg);
    }

    and then invoke it as:
    find somedirectory -iname "*txt" -print0 | xargs -0 java -jar myprog.jar

    or even:
    find somedirectory -iname "*txt" -print | xargs java -jar myprog.jar
    if your xargs doesn't support -0 and you don't have any funky file names.

    That would apportion the startup costs across hundreds to thousands of
    invocations (depending upon your system's tuning and options to xargs).

    - dmw

    --
    .. Douglas Wells . Connection Technologies .
    .. Internet: -sp9804- -at - contek.com- .
     
    Douglas Wells, Jan 25, 2007
    #5
  6. Nigel Wade Guest

    wrote:

    > Hi all,
    >
    > I'm using small Java utilities called from scripts (mainly
    > shell scripts on Unix platforms).
    >
    > One example would be something like:
    >
    > find somedirectory -iname "*txt" -exec java -jar myprog.jar {} \;
    >
    > Which could result in hundreds of call to the Java program.
    >


    Another approach entirely would be to change myprog.jar so that it accepts
    multiple arguments. This may not be applicable in your case, but if you can do
    this you could vastly improve efficiency by getting myprog.jar to process
    multiple files per invocation. All you then need to do is combine find with
    xargs:

    find somedirectory -iname "*txt" | xargs -n somenumber java -jar myprog.jar

    xargs will run "java -jar myprog.jar" with "somenumber" of args from find. Say
    you set this to 20, then for every 20 files found by find, xargs would run your
    jar file with those 20 files as the arguments. The normal trick is to tune
    "somenumber" so that it is large for efficiency, but not large enough to
    overflow the command line buffer.

    Basically, xargs reads a list from its standard input, and for every N elements
    of that list it executes the command with those elements as command line
    arguments. Combining find and xargs is a common way of improving efficiency
    over using -exec when processing a list of files from find.

    --
    Nigel Wade, System Administrator, Space Plasma Physics Group,
    University of Leicester, Leicester, LE1 7RH, UK
    E-mail :
    Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555
     
    Nigel Wade, Jan 26, 2007
    #6
  7. Daniel Pitts Guest

    On Jan 25, 7:43 am, wrote:
    > Hi all,
    >
    > I'm using small Java utilities called from scripts (mainly
    > shell scripts on Unix platforms).
    >
    > One example would be something like:
    >
    > find somedirectory -iname "*txt" -exec java -jar myprog.jar {} \;
    >
    > Which could result in hundreds of call to the Java program.
    >
    > JVMstartuptimes have kept going down... And Java 1.6 is
    > continuing to show this (very good) trend.
    >
    > Still... The JVMstartuptime are killing the perfs here.
    >
    > Taking into account the fact that I want to continue using
    > these Java utilities (that is: I won't rewrite them in any
    > other language), is there any way to speed up JVMstartup
    > time?
    >
    > Particularly, I was thinking of the following: could I launch
    > one daemon-like Java process that would always be running
    > and that would take care of launching the various jars
    > when needed?
    >
    > Would that help to prevent a new JVMstartupeverytime a
    > new .jar is launched?
    >
    > So is it possible to have one (say one per user) JVM "always
    > on" and run new jars inside that JVM?
    >
    > And the example would become, say:
    >
    > find somedirectory -iname "*txt" -exec
    > littleUtilThatNotifiesMyJavaDaemon myprog.jar {} \;
    >
    > That 'littleUtilThatNotifiesMyJavaDaemon' being "something" (you tell
    > me :)
    > that tells the running JVM that it should execute a new .jar.
    >
    > I seems possible to do such a thing, but would this offer any
    > significant speedup?
    >
    > I guess my question boils down to: if you run a .jar from another
    > Java program, can this be made to be faster than simply doing
    > a new "java -jar ...".
    >
    > Any info greatly appreciated,
    >
    > Driss


    It might be better to change the Java code to read from a list of
    files, rather than take one file as an argument.
    Then, you can have your find command simply output the file, redirect
    to a list.txt file, and then run the jar on the list.txt file.

    Hope this helps,
    Daniel.
     
    Daniel Pitts, Jan 26, 2007
    #7
  8. Guest

    On Jan 26, 10:08 am, Nigel Wade <> wrote:
    > wrote:
    > > Hi all,

    >
    > > I'm using small Java utilities called from scripts (mainly
    > > shell scripts on Unix platforms).

    >
    > > One example would be something like:

    >
    > > find somedirectory -iname "*txt" -exec java -jar myprog.jar {} \;



    I'd like to thanks everybody who answered here.

    Regarding the xargs solution: though I knew xargs I didn't think of
    it here, so it's good that you pointed it out.

    However I'm calling several little Java utilities and xargs won't work
    in all the cases. So I may end up coding a little Java program that
    will call other Java program and... remember to use xargs everywhere
    it's applicable.

    Thanks again to everybody,

    Driss
     
    , Jan 27, 2007
    #8
  9. Brian Palmer Guest

    lid (Douglas Wells) writes:

    > As noted elsewhere, you could certainly do that, but how about
    > something a bit less invasive?
    >
    > Could you instead convert your programs to accept multiple parameters,
    > a la.


    Of course, this could be done by creating a wrapper class that uses
    intros pection to call the original class's main once for each file on
    the command line.

    Then you'd do it as
    find ... | xargs -0rx java argsplitter com.example.MyProg

    --
    See comp.lang.java.announce for java-related announcements
     
    Brian Palmer, Jan 29, 2007
    #9
  10. david ullua Guest

    On Jan 29, 10:45 pm, Brian Palmer <>
    wrote:
    > (Douglas Wells) writes:
    > > As noted elsewhere, you could certainly do that, but how about
    > > something a bit less invasive?

    >
    > > Could you instead convert your programs to accept multiple parameters,
    > > a la.

    >
    > Of course, this could be done by creating a wrapper class that uses
    > intros pection to call the original class's main once for each file on
    > the command line.
    >
    > Then you'd do it as
    > find ... | xargs -0rx java argsplitter com.example.MyProg
    >
    > --
    > See comp.lang.java.announce for java-related announcements


    a good question and good anwsers. summary of these key points:
    1. combine several invokes args into one, and pass it to a args parser
    java program.
    2. use file to save the args. and read args from file in java program.
    3. can use wrapper to hide the origin java program if cannot be
    modified.
     
    david ullua, Feb 2, 2007
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Martin Altemark
    Replies:
    1
    Views:
    309
    Natty Gur
    Aug 21, 2003
  2. Joost Kraaijeveld
    Replies:
    4
    Views:
    5,460
  3. Arjan Dikhoff
    Replies:
    2
    Views:
    503
    Miki Tebeka
    Feb 11, 2004
  4. flamesrock
    Replies:
    8
    Views:
    519
    Hendrik van Rooyen
    Nov 24, 2006
  5. Shriphani
    Replies:
    2
    Views:
    569
    Ian Kelly
    Jun 1, 2008
Loading...

Share This Page