Re: Recommendations for Lightweight Threading?

Discussion in 'Java' started by Eric Sosman, Jun 16, 2012.

  1. Eric Sosman

    Eric Sosman Guest

    On 6/15/2012 6:33 PM, Aaron W. Hsu wrote:
    > I am considering moving one of my projects from C to Java, but I am
    > hoping to find a high-performance threading implementation, or something
    > along the lines of libqthread, which offers Fill-Empty bit blocking and
    > good cooperative lightweight threading as a library.
    >[...]


    Others have covered the essentials, but I'd like to raise an
    eyebrow at the word "cooperative." In the usual argot, "cooperative
    threading" refers to schemes where a thread runs until it chooses
    to relinguish the CPU, voluntarily. This may be contrasted to
    "preemptive" schemes, where an external scheduler decides which
    threads will run on which CPU's and when, and may choose to evict
    a running thread or start a stalled thread at any arbitrary moment.

    The "collegial" nature of cooperative threading offers a degree
    of calmness, and some freedom from worry about critical sections
    (if you're on a uniprocessor, you may be able to avoid taking an
    explicit lock to guard a critical section: Just don't yield the CPU
    while you're in it). But this cooperation is fragile in the extreme,
    and becomes almost completely untenable once you start assembling an
    application out of classes from disparate sources. Unless you're
    writing for a very tightly controlled environment where you have
    audited all the components or developed them yourself -- embedded
    software in a Mars rover, say -- I think "cooperative multithreading"
    is too brittle for most practical applications.

    If that's not what you meant by "cooperative," well, never mind.
    But if it is, my advice is Don't Do That.

    --
    Eric Sosman
    d
     
    Eric Sosman, Jun 16, 2012
    #1
    1. Advertising

  2. Eric Sosman

    Eric Sosman Guest

    On 6/15/2012 8:59 PM, Aaron W. Hsu wrote:
    > On Fri, 15 Jun 2012 20:19:12 -0400, Eric Sosman wrote:
    >
    >> Unless you're writing for a very tightly controlled environment where
    >> you have audited all the components or developed them yourself --
    >> embedded software in a Mars rover, say -- I think "cooperative
    >> multithreading"
    >> is too brittle for most practical applications.

    >
    > In some sense, this is the environment. I am building on top of non-
    > deterministic concurrency primitives to construct deterministic models.
    > In these models, I prefer to have more control over when thread
    > preemption happens, as well as what lightweight threads get associated
    > with what worker thread (thread pool) and similarly what worker threads
    > get associated with what CPUs. This is to allow me to make more fine
    > grained decisions about scheduling, which may affect cache behavior.
    >
    > Preemptive multi-threading is certainly the more reliable when dealing
    > with a general programming interface, but since these models specifically
    > constrain the problems in certain ways, I know the points at which I
    > would like to preempt a thread, and I also know that preempting earlier
    > or later than these points is relatively useless.
    >
    > Once I have a model written up in Java, I will see how things go. It is
    > likely that for small scale tests I will not notice a difference in
    > cooperative versus preemptive models.


    Okay. Keep in mind that in Java it is very difficult to avoid
    being preempted: Even if the thread you're interested in is careful
    not to create new object instances, object creation in other threads
    can trigger the garbage collector at pretty much any time. You may
    be able to get some scheduling benefits out of a cooperative scheme,
    but I doubt you'll get thread safety. Locking or the concurrency
    libraries will be all but unavoidable.

    --
    Eric Sosman
    d
     
    Eric Sosman, Jun 16, 2012
    #2
    1. Advertising

  3. Eric Sosman

    Roedy Green Guest

    On Fri, 15 Jun 2012 20:19:12 -0400, Eric Sosman
    <> wrote, quoted or indirectly quoted
    someone who said :

    > The "collegial" nature of cooperative threading offers a degree
    >of calmness, and some freedom from worry about critical sections
    >(if you're on a uniprocessor, you may be able to avoid taking an
    >explicit lock to guard a critical section:


    that takes me back. I wrote code for a Univac 1616 military mini to
    make it simulate an IBM front end communications processor. That is
    exactly how it worked.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    Controlling complexity is the essence of computer programming.
    ~ Brian W. Kernighan 1942-01-01
    ..
     
    Roedy Green, Jun 16, 2012
    #3
  4. Eric Sosman

    Roedy Green Guest

    On Sat, 16 Jun 2012 01:00:10 -0700, Roedy Green
    <> wrote, quoted or indirectly quoted
    someone who said :

    >
    >that takes me back. I wrote code for a Univac 1616 military mini to
    >make it simulate an IBM front end communications processor. That is
    >exactly how it worked.


    Circa 1990 I wrote a co-operative thread package for windows for C. It
    was surprisingly simple. On task switch I had to save the stack and
    registers and restore another thread's stack and registers. That was
    basically all there was to it.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    Controlling complexity is the essence of computer programming.
    ~ Brian W. Kernighan 1942-01-01
    ..
     
    Roedy Green, Jun 16, 2012
    #4
  5. Eric Sosman

    Eric Sosman Guest

    Re: Controlling the Garbage Collector

    On 6/16/2012 12:51 PM, Aaron W. Hsu wrote:
    > On Fri, 15 Jun 2012 21:37:32 -0400, Eric Sosman wrote:
    >
    >> Even if the thread you're interested in is careful not to create new
    >> object instances, object creation in other threads can trigger the
    >> garbage collector at pretty much any time.

    >
    > The garbage collector is one of the reasons that I have hesitated to move
    > to Java. In some languages (Chez Scheme) I can get quite explicit
    > control over when and how garbage collection occurs, which can make it
    > possible to do very fine grained things to avoid some of the corner case
    > problems that can manifest in garbage collected languages. Usually this
    > is not a problem at all, but I would like to have the control nonetheless.


    Then I'd suggest Java may be ill-suited to your needs.

    You may be able to get some level of GC control on a particular
    JVM implementation, but it won't be robust. Your control is likely
    to slip if you move your Java to another machine, or if you upgrade
    the JVM, or even if you do something as innocuous as adjusting the
    sizes of assorted memory pools.

    For good or ill, Java's "attitude" about garbage collection is
    that it's a service provided by the JVM -- in whatever way the JVM
    sees fit to provide it, and the JVM is allowed to be capricious.

    --
    Eric Sosman
    d
     
    Eric Sosman, Jun 16, 2012
    #5
  6. Eric Sosman

    markspace Guest

    Re: Controlling the Garbage Collector

    On 6/16/2012 11:24 AM, Eric Sosman wrote:
    > Then I'd suggest Java may be ill-suited to your needs.



    For us production coders, that's true. Depending on how experimental
    Aaron's project is however, hacking the JVM may not be out of the
    question. What he's talking about certainly sounds more academic to me,
    so I'm sort of curious what methods he's going to use to achieve his goals.
     
    markspace, Jun 16, 2012
    #6
  7. Eric Sosman

    Daniel Pitts Guest

    Re: Controlling the Garbage Collector

    On 6/16/12 12:24 PM, markspace wrote:
    > On 6/16/2012 11:24 AM, Eric Sosman wrote:
    >> Then I'd suggest Java may be ill-suited to your needs.

    >
    >
    > For us production coders, that's true. Depending on how experimental
    > Aaron's project is however, hacking the JVM may not be out of the
    > question. What he's talking about certainly sounds more academic to me,
    > so I'm sort of curious what methods he's going to use to achieve his goals.

    It almost sounds like the Aaron may be better off using Event Loop style
    architecture, with one event dispatch thread per CPU core. Allow the GC
    to do what it does best, and have the operations be smaller and more
    decoupled.
     
    Daniel Pitts, Jun 16, 2012
    #7
  8. Eric Sosman

    Eric Sosman Guest

    Re: Controlling the Garbage Collector

    On 6/16/2012 4:14 PM, Daniel Pitts wrote:
    > On 6/16/12 12:24 PM, markspace wrote:
    >> On 6/16/2012 11:24 AM, Eric Sosman wrote:
    >>> Then I'd suggest Java may be ill-suited to your needs.

    >>
    >>
    >> For us production coders, that's true. Depending on how experimental
    >> Aaron's project is however, hacking the JVM may not be out of the
    >> question. What he's talking about certainly sounds more academic to me,
    >> so I'm sort of curious what methods he's going to use to achieve his
    >> goals.

    > It almost sounds like the Aaron may be better off using Event Loop style
    > architecture, with one event dispatch thread per CPU core. Allow the GC
    > to do what it does best, and have the operations be smaller and more
    > decoupled.


    I, too, am mystified about what he's up to. It's hard to reconcile
    an interest in tight control over GC with "Something that scales up
    efficiently to distributed computing" -- the aims seem divergent.

    Aaron: Care to share?

    --
    Eric Sosman
    d
     
    Eric Sosman, Jun 16, 2012
    #8
  9. Eric Sosman

    markspace Guest

    Re: Controlling the Garbage Collector

    On 6/16/2012 1:14 PM, Daniel Pitts wrote:
    > On 6/16/12 12:24 PM, markspace wrote:
    >> On 6/16/2012 11:24 AM, Eric Sosman wrote:
    >>> Then I'd suggest Java may be ill-suited to your needs.

    >>
    >>
    >> For us production coders, that's true. Depending on how experimental
    >> Aaron's project is however, hacking the JVM may not be out of the
    >> question. What he's talking about certainly sounds more academic to me,
    >> so I'm sort of curious what methods he's going to use to achieve his
    >> goals.

    > It almost sounds like the Aaron may be better off using Event Loop style
    > architecture, with one event dispatch thread per CPU core. Allow the GC
    > to do what it does best, and have the operations be smaller and more
    > decoupled.
    >



    I dunno. Phase 1 of his little project might just be to implement a
    regular old multi-threaded program in Java (probably one with some
    specific characteristics that he's interested in).

    Phase 2 might be to modify his local JVM and show that his special
    GC/threading model produces some specific improvement(s), while running
    the same byte codes as phase 1.

    That's what I'd do, if I had the resources....
     
    markspace, Jun 16, 2012
    #9
  10. Re: Controlling the Garbage Collector

    On Saturday, June 16, 2012 6:51:13 PM UTC+2, Aaron W. Hsu wrote:
    > On Fri, 15 Jun 2012 21:37:32 -0400, Eric Sosman wrote:
    >
    > > Even if the thread you're interested in is careful not to create new
    > > object instances, object creation in other threads can trigger the
    > > garbage collector at pretty much any time.

    >
    > The garbage collector is one of the reasons that I have hesitated to move
    > to Java. In some languages (Chez Scheme) I can get quite explicit
    > control over when and how garbage collection occurs, which can make it
    > possible to do very fine grained things to avoid some of the corner case
    > problems that can manifest in garbage collected languages. Usually this
    > is not a problem at all, but I would like to have the control nonetheless.


    Be sure to try out G1 collector. In my brief tests so far it worked pretty well with regard to the target timings albeit at the expense of a bit of CPU overhead.

    Kind regards

    robert
     
    Robert Klemme, Jun 18, 2012
    #10
  11. Re: Controlling the Garbage Collector

    On 6/16/2012 12:51 PM, Aaron W. Hsu wrote:
    > On Fri, 15 Jun 2012 21:37:32 -0400, Eric Sosman wrote:
    >> Even if the thread you're interested in is careful not to create new
    >> object instances, object creation in other threads can trigger the
    >> garbage collector at pretty much any time.

    >
    > The garbage collector is one of the reasons that I have hesitated to move
    > to Java. In some languages (Chez Scheme) I can get quite explicit
    > control over when and how garbage collection occurs, which can make it
    > possible to do very fine grained things to avoid some of the corner case
    > problems that can manifest in garbage collected languages. Usually this
    > is not a problem at all, but I would like to have the control nonetheless.


    What do you mean by corner case?

    GC and real time does usually not fit well (*), but I would not consider
    real time a corner case. Some need it - many don't.

    Arne

    *) There is RTSJ (Real-Time Specification for Java) but it does not
    seem to have that much traction.
     
    Arne Vajhøj, Jun 21, 2012
    #11
  12. Eric Sosman

    Lew Guest

    Re: Controlling the Garbage Collector

    Aaron W. Hsu wrote:
    > Arne Vajhøj wrote:
    >> Aaron W. Hsu wrote:
    >>> In some languages (Chez Scheme) I can get quite explicit
    >>> control over when and how garbage collection occurs, which can make it
    >>> possible to do very fine grained things to avoid some of the corner
    >>> case problems that can manifest in garbage collected languages.

    >>
    >> What do you mean by corner case?

    >
    > So, you mentioned that you thought real-time was not a corner case, but I
    > guess I would consider it one compared to general purpose programming in
    > the majority. However, there are other, less broad things where I have
    > found it extremely helpful to have control of the collector.
    >
    > For instance, I implemented a number of weak pointer structures which


    WeakReference.

    > required me to be able to extend what happened during collections. This


    Object#finalize()
    WeakHashMap

    > can be done in user-space efficiently if there is an efficient way to
    > extend the garbage collector and trigger events to occur on certain
    > conditions. This would be an example of a relatively obscure data
    > structure that can be really useful in some situations.


    Java has the infrastructure for what you need.

    > On the other side, there are times when I know that I am going to be
    > doing a number of very small, little allocations all in a row, and I do
    > not want the GC to run during this tight loop, because it will cause a
    > noticeable hit. I would rather delay the collector, consuming more memory
    > until the end when I can explicitly trigger the collector at that point.


    Have you looked at the parameters to control GC?

    > Or, I may know that some structure is going to be extremely long lived
    > and I need to keep it out of the collector entirely.


    Are you absolutely certain you can do this better than the collector?
    I'm not.

    > Finally, there are times when I want to do a large bulk allocation
    > outside of the collector, and then selectively move certain things into
    > the collected space, but still have a checked, high-level way of
    > accessing data structures in the uncollected space.


    Not Java.

    > Now, all of these are corner cases to me because you can get by without
    > them in most situations, unless you are trying to eek out all of the
    > performance you can from your system.


    I'm not so sure that you could do what you want the way you say you want to..

    I feel highly confident that by appropriate choice of collectors and tuning
    their parameters you can achieve your performance goals.

    --
    Lew
     
    Lew, Jun 21, 2012
    #12
  13. Eric Sosman

    Eric Sosman Guest

    Re: Controlling the Garbage Collector

    On 6/21/2012 2:24 PM, Aaron W. Hsu wrote:
    > [...]
    > On the other side, there are times when I know that I am going to be
    > doing a number of very small, little allocations all in a row, and I do
    > not want the GC to run during this tight loop, because it will cause a
    > noticeable hit. I would rather delay the collector, consuming more memory
    > until the end when I can explicitly trigger the collector at that point.


    Sounds like you might be better off collecting first, when there
    are fewer objects to be discovered and analyzed. Also, since you've
    said you're not speaking of real-time applications, I don't see why
    you should be concerned about the precise timing of the "hit:" Before,
    after, during, what difference does it make? (If you're worried about
    GC roiling the caches, please provide actual measurements.)

    > Or, I may know that some structure is going to be extremely long lived
    > and I need to keep it out of the collector entirely.


    Generational collectors do a pretty good imitation of this. It's
    not perfect because it takes time for the GC to discover what's long-
    lived and what's not, but it's pretty effective. Also, it has the
    huge advantage of always being right; few programmers can match that
    accuracy. (See also "memory leak.")

    > Finally, there are times when I want to do a large bulk allocation
    > outside of the collector, and then selectively move certain things into
    > the collected space, but still have a checked, high-level way of
    > accessing data structures in the uncollected space.


    JNI, if you simply must.

    > Now, all of these are corner cases to me because you can get by without
    > them in most situations, unless you are trying to eek out all of the
    > performance you can from your system.


    "Eek" is right ...

    --
    Eric Sosman
    d
     
    Eric Sosman, Jun 21, 2012
    #13
  14. Eric Sosman

    Fred Greer Guest

    Re: Controlling the Garbage Collector

    On Thu, 21 Jun 2012 11:37:09 -0700, Lew wrote:

    > Aaron W. Hsu wrote:
    >> Arne Vajhøj wrote:
    >>> Aaron W. Hsu wrote:
    >>>> In some languages (Chez Scheme) I can get quite explicit control over
    >>>> when and how garbage collection occurs, which can make it possible to
    >>>> do very fine grained things to avoid some of the corner case problems
    >>>> that can manifest in garbage collected languages.
    >>>
    >>> What do you mean by corner case?

    >>
    >> So, you mentioned that you thought real-time was not a corner case, but
    >> I guess I would consider it one compared to general purpose programming
    >> in the majority. However, there are other, less broad things where I
    >> have found it extremely helpful to have control of the collector.
    >>
    >> For instance, I implemented a number of weak pointer structures which

    >
    > WeakReference.
    >
    >> required me to be able to extend what happened during collections. This

    >
    > Object#finalize()
    > WeakHashMap
    >
    >> can be done in user-space efficiently if there is an efficient way to
    >> extend the garbage collector and trigger events to occur on certain
    >> conditions. This would be an example of a relatively obscure data
    >> structure that can be really useful in some situations.


    ReferenceQueue
     
    Fred Greer, Jun 21, 2012
    #14
  15. Re: Controlling the Garbage Collector

    On 21.06.2012 20:24, Aaron W. Hsu wrote:
    > On Wed, 20 Jun 2012 21:19:01 -0400, Arne Vajhøj wrote:
    >
    >> On 6/16/2012 12:51 PM, Aaron W. Hsu wrote:

    >
    >>> In some languages (Chez Scheme) I can get quite explicit
    >>> control over when and how garbage collection occurs, which can make it
    >>> possible to do very fine grained things to avoid some of the corner
    >>> case problems that can manifest in garbage collected languages.

    >>
    >> What do you mean by corner case?

    >
    > So, you mentioned that you thought real-time was not a corner case, butI
    > guess I would consider it one compared to general purpose programming in
    > the majority. However, there are other, less broad things where I have
    > found it extremely helpful to have control of the collector.


    Your statement illustrates one of the biggest problems of Java (or maybe
    even any GC'ed language): GC should be automatic but humans still need
    control (just count the collectors and all the JVM options for GC)
    because the automatisms are not good enough (yet). Question is: will
    they ever be good enough that we can get by without settings - or at
    least a dramatic reduced number of settings? I do not see that coming
    soon. Sun took seven or eight years from the first experimental
    implementation of G1 to production ready. Go figure.

    > For instance, I implemented a number of weak pointer structures which
    > required me to be able to extend what happened during collections. This
    > can be done in user-space efficiently if there is an efficient way to
    > extend the garbage collector and trigger events to occur on certain
    > conditions. This would be an example of a relatively obscure data
    > structure that can be really useful in some situations.


    Can you explain what you did at collection time? I am asking because
    anytime I would prefer to do cleanup when I do not need an object any
    more. This is the typical resource deallocation in a finally block or
    other piece of code which is invoked after some operation has finished
    (hooks for example). Advantage is that you free your resources as early
    as possible and do not need to cope with the downsides of finalizers.
    Finalizers in Java have so many disadvantages (i.e. when making objects
    live again, effects on GC performance) that they are short of being
    deprecated - at least that's my impression from what I read.

    > On the other side, there are times when I know that I am going to be
    > doing a number of very small, little allocations all in a row, and I do
    > not want the GC to run during this tight loop, because it will cause a
    > noticeable hit. I would rather delay the collector, consuming more memory
    > until the end when I can explicitly trigger the collector at that point.


    But what do you gain if a GC pause occurs after this operation? The
    time is spent either way. Btw, depending on collector chosen you might
    not even notice that it is running because it works in separate threads.

    > Or, I may know that some structure is going to be extremely long lived
    > and I need to keep it out of the collector entirely.


    Long lived objects which live shorter than the application (i.e. not
    classes) are actually the Achilles heel of GC because it is very hard to
    tune the collector in a way that it does not visit those long living
    objects too often and yet run often enough to ensure enough free memory
    is available.

    > Finally, there are times when I want to do a large bulk allocation
    > outside of the collector, and then selectively move certain things into
    > the collected space, but still have a checked, high-level way of
    > accessing data structures in the uncollected space.


    I once mused about such a thing as well. I you think a bit longer about
    this then you'll notice that it won't work: these objects still have to
    be visited because they may have references to other objects not in the
    uncollectable space. You'll probably do not gain much - if anything at all.

    > Now, all of these are corner cases to me because you can get by without
    > them in most situations, unless you are trying to eek out all of the
    > performance you can from your system.


    Use many cores and G1. At least try it out.

    Kind regards

    robert

    --
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Jun 21, 2012
    #15
  16. Eric Sosman

    Lew Guest

    Re: Controlling the Garbage Collector

    markspace wrote:
    > System.gc() will


    request to
    > run the GC,


    which request might not be honored, but if it is, it will

    > move objects into one or more permanent generation spaces, and then dump
    > the large blocks of temporary objects.
    >
    > Many things can alter this, and I see you have some special
    > requirements, but the main bits of the Java GC should give you something
    > very close to what you are looking for.


    Agreed.

    Tell us why the existing options don't reach your goals, OP.

    --
    Lew
     
    Lew, Jun 21, 2012
    #16
  17. Re: Controlling the Garbage Collector

    On 23.06.2012 13:36, Wanja Gayk wrote:

    > Maybe it's just because sometimes people don't like to give up control
    > they once had, just to be sure they could if they needed, even though in
    > 99.99% of all cases they don't.


    In my experience it's different: you need to have control because the
    automatic mechanisms in GC implementations still do not cope well with
    all use cases (at least those we had). Although that does not mean that
    for many use cases defaults are sufficient (which they are I believe).

    Kind regards

    robert

    --
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Jun 23, 2012
    #17
  18. Eric Sosman

    Roedy Green Guest

    Re: Controlling the Garbage Collector

    On Mon, 18 Jun 2012 04:32:15 -0700 (PDT), Robert Klemme
    <> wrote, quoted or indirectly quoted
    someone who said :

    >Be sure to try out G1 collector. In my brief tests so far it worked pretty
    >well with regard to the target timings albeit at the expense of a bit of CPU overhead.


    There is also the GC in Jet. It does not seem to have the lurchiness
    of Oracle GC.

    see http://mindprod.com/jgloss/jet.html
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    Computers are machines that do exactly what you tell them,
    but they still can surprise you with the results.
    ~ Dr. Richard Dawkins 1941-03-26
     
    Roedy Green, Jun 24, 2012
    #18
  19. Re: Controlling the Garbage Collector

    Robert Klemme <> writes:

    > Long lived objects which live shorter than the application (i.e. not
    > classes) are actually the Achilles heel of GC because it is very hard to
    > tune the collector in a way that it does not visit those long living
    > objects too often and yet run often enough to ensure enough free memory is


    It might be good to have a method to gell the GC that a certain
    long-lived object is no longer needed.
    For example, something like
    System.gc(Object old)
    where old is the object that has just been needed the last time.

    --
    Jukka Lahtinen
     
    Jukka Lahtinen, Jun 25, 2012
    #19
  20. Eric Sosman

    Eric Sosman Guest

    Re: Controlling the Garbage Collector

    On 6/25/2012 8:28 AM, Jukka Lahtinen wrote:
    > Robert Klemme <> writes:
    >
    >> Long lived objects which live shorter than the application (i.e. not
    >> classes) are actually the Achilles heel of GC because it is very hard to
    >> tune the collector in a way that it does not visit those long living
    >> objects too often and yet run often enough to ensure enough free memory is

    >
    > It might be good to have a method to gell the GC that a certain
    > long-lived object is no longer needed.
    > For example, something like
    > System.gc(Object old)
    > where old is the object that has just been needed the last time.


    How would you call the method?

    --
    Eric Sosman
    d
     
    Eric Sosman, Jun 25, 2012
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    9
    Views:
    1,131
    Mark Space
    Dec 29, 2007
  2. Steven Woody
    Replies:
    0
    Views:
    478
    Steven Woody
    Jan 9, 2009
  3. markspace
    Replies:
    4
    Views:
    439
    Daniel Pitts
    Jun 16, 2012
  4. Lew
    Replies:
    0
    Views:
    354
  5. Kevin McMurtrie
    Replies:
    3
    Views:
    398
    Robert Klemme
    Jun 19, 2012
Loading...

Share This Page