Does object pooling *ever* make sense?

Discussion in 'Java' started by Chris, Feb 8, 2007.

  1. Chris

    Chris Guest

    I've read recently that object allocation in recent JVMs is so fast that
    it doesn't make sense to create object pools. Just create a new object
    when you need it and let the garbage collector do the work.

    Does this hold true when your objects are very large, though? What if
    your object contains a byte [] of length 100K? Or 1Mb?

    What's the breakeven point beyond which it makes sense to reuse objects?
    Chris, Feb 8, 2007
    #1
    1. Advertising

  2. Chris

    Lew Guest

    Chris wrote:
    > I've read recently that object allocation in recent JVMs is so fast that
    > it doesn't make sense to create object pools. Just create a new object
    > when you need it and let the garbage collector do the work.
    >
    > Does this hold true when your objects are very large, though? What if
    > your object contains a byte [] of length 100K? Or 1Mb?
    >
    > What's the breakeven point beyond which it makes sense to reuse objects?


    As I understand, the memory allocation procedure in the JVM is to add some
    size n to a pointer p and call that region the new object. I don't think there
    is anything much faster than that. It takes the same amount of time to add 10M
    to p as to add 10. (<=1 cycle?)

    The time spent is zeroing the memory, I'd guess, but you'd have to do that
    with an object pool, too.

    I surmise that there is no breakeven point.

    - Lew
    Lew, Feb 8, 2007
    #2
    1. Advertising

  3. Chris

    Daniel Pitts Guest

    On Feb 7, 7:17 pm, Lew <> wrote:
    > Chris wrote:
    > > I've read recently that object allocation in recent JVMs is so fast that
    > > it doesn't make sense to create object pools. Just create a new object
    > > when you need it and let the garbage collector do the work.

    >
    > > Does this hold true when your objects are very large, though? What if
    > > your object contains a byte [] of length 100K? Or 1Mb?

    >
    > > What's the breakeven point beyond which it makes sense to reuse objects?

    >
    > As I understand, the memory allocation procedure in the JVM is to add some
    > size n to a pointer p and call that region the new object. I don't think there
    > is anything much faster than that. It takes the same amount of time to add 10M
    > to p as to add 10. (<=1 cycle?)
    >
    > The time spent is zeroing the memory, I'd guess, but you'd have to do that
    > with an object pool, too.
    >
    > I surmise that there is no breakeven point.
    >
    > - Lew


    I'd suggest running a few tests to find out.

    The best suggestion I can make is to wrap all your calls to "new" in a
    method somewhere, so that if you decide you need to pool, you can.

    Generally, its bad engineering to worry about speed before it becomes
    a problem. The first priority is clarity. Next to functionality of
    course.
    If you find that there is a speed (or memory) issue, THEN you run a
    profiler (don't ever "guess" or speculate on what the problem is,
    you'll most likely be wrong, no matter how good you are)

    The profiler will tell you what you need to optimize.

    Hope this helps.
    Daniel Pitts, Feb 8, 2007
    #3
  4. On Wed, 07 Feb 2007 18:53:15 -0600, Chris wrote:
    > I've read recently that object allocation in recent JVMs is so fast
    > that it doesn't make sense to create object pools. Just create a new
    > object when you need it and let the garbage collector do the work.
    >
    > Does this hold true when your objects are very large, though? What
    > if your object contains a byte [] of length 100K? Or 1Mb?
    >
    > What's the breakeven point beyond which it makes sense to reuse
    > objects?


    A disadvantage to pooling is that it increases the average age of your
    objects, resulting in more live objects at any given time, more
    objects that live beyond the "nursery", and consequently more work for
    the garbage collector.

    You also have to manage the object pool.

    So in order for pooling to be effective, there has to be a saving
    somewhere that outweighs these costs (and possibly others I haven't
    mentioned). I don't think the cost depends as much on the *size* of
    the object as it depends on the amount of additional *initialization*
    that you need to do before you can use it, assuming that you can avoid
    that initialization in the pooled case.

    /gordon

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
    Gordon Beaton, Feb 8, 2007
    #4
  5. Chris <> wrote:
    > Does this hold true when your objects are very large, though? What if
    > your object contains a byte [] of length 100K? Or 1Mb?


    If you can reuse the byte array without resetting it to zeros
    (e.g. if the programs logic is such that every element of
    the byte-array is overwritten, anyway) then it is possible
    that pooling saves some time, but you take the risk that some
    (otherwise minor) bug in your program might turn into nightmare
    when old (non-zero) garbage in the array might slip through.
    Andreas Leitgeb, Feb 8, 2007
    #5
  6. Chris

    Andy Dingley Guest

    On 8 Feb, 00:53, Chris <> wrote:
    > I've read recently that object allocation in recent JVMs is so fast that
    > it doesn't make sense to create object pools.


    Most of my pooled objects are some sort of reference to an external
    resource (e.g. a DB connection) that's inherently expensive. I don't
    care what the cost of the object itself is, they're pooled to
    economise on this external cost and that's not changed by any JVM
    improvement.

    Given the low cost of Java object creation I find it hard to justify
    pooling for objects that are simply entirely Java anyway. What might
    cause me to want to pool them? A stateless singleton is justifiable
    but anything with state attached to it probably means as much effort
    to create the distinct state for each pooled use of the object as it
    does to create a whole new object.
    Andy Dingley, Feb 8, 2007
    #6
  7. Chris

    Lew Guest

    Chris <> wrote:
    >> I've read recently that object allocation in recent JVMs is so fast that
    >> it doesn't make sense to create object pools.


    Andy Dingley wrote:
    > Most of my pooled objects are some sort of reference to an external
    > resource (e.g. a DB connection) that's inherently expensive. I don't
    > care what the cost of the object itself is, they're pooled to
    > economise on this external cost and that's not changed by any JVM
    > improvement.
    >
    > Given the low cost of Java object creation I find it hard to justify
    > pooling for objects that are simply entirely Java anyway. What might
    > cause me to want to pool them? A stateless singleton is justifiable
    > but anything with state attached to it probably means as much effort
    > to create the distinct state for each pooled use of the object as it
    > does to create a whole new object.


    And remember Gordon's points:

    Gordon Beaton wrote:
    >> A disadvantage to pooling is that it increases the average age of your
    >> objects, resulting in more live objects at any given time, more
    >> objects that live beyond the "nursery", and consequently more work for
    >> the garbage collector.
    >>
    >> You also have to manage the object pool.


    That "more work for the garbage collector" relates to the fact that it's
    harder to garbage collect out of the tenured generation than the nursery.
    Except for resource gates, it rarely helps to pool.

    - Lew
    Lew, Feb 8, 2007
    #7
  8. Chris

    Andy Dingley Guest

    On 8 Feb, 13:01, Lew <> wrote:

    > Except for resource gates, it rarely helps to pool.


    One somewhat perverse use I have found for object pooling is to avoid
    memory leakage. In a context of refactoring truly nasty code, a pool
    of re-used objects was one way to avoid coders allocating these things
    willy-nilly and never disposing of them afterwards at all! Finite
    cruft is (marginally) better than inifinite expanding cruft.

    ....And it's less cruel than my other technique, with the hot clueiron.
    Andy Dingley, Feb 8, 2007
    #8
  9. Chris

    Lew Guest

    Andy Dingley wrote:
    > One somewhat perverse use I have found for object pooling is to avoid
    > memory leakage. In a context of refactoring truly nasty code, a pool
    > of re-used objects was one way to avoid coders allocating these things
    > willy-nilly and never disposing of them afterwards at all! Finite
    > cruft is (marginally) better than inifinite expanding cruft.
    >
    > ...And it's less cruel than my other technique, with the hot clueiron.


    Of course, in Java one does not often have to explicitly "dispose of" an
    object. The usual difficulty is making sure to pull all references away from
    an object, but there are a huge number of situations where that isn't necessary.

    Consider:

    public void foo()
    {
    Bax bax = new Bax(); // assume it does not wrap a resource
    // do some stuff with bax
    // but do NOT create other references to the object
    }

    No explicit disposal needed for the new Bax if it is never referenced
    somewhere else, e.g., in a long-lived Collection. Once the variable "bax" goes
    out of scope in that scenario, the programmer can rest easy.

    It is actually a good idiom in Java to "allocat[e] ... things willy-nilly".

    - Lew
    Lew, Feb 9, 2007
    #9
  10. Chris

    Joe Seigh Guest

    Chris wrote:
    > I've read recently that object allocation in recent JVMs is so fast that
    > it doesn't make sense to create object pools. Just create a new object
    > when you need it and let the garbage collector do the work.
    >
    > Does this hold true when your objects are very large, though? What if
    > your object contains a byte [] of length 100K? Or 1Mb?
    >
    > What's the breakeven point beyond which it makes sense to reuse objects?



    Not so much that the gc is so much faster than it's not really all that worse
    and the effects of tying up a lot of memory in object pools is more problematic.
    The copying collector relies on having enough memory so you don't run out
    between normal GC cycles. If you are running out of memory and forcing GC
    to run more often, you could just use an object pool and use a WeakReference to
    point to the object pool. The pool will be reclaimed on every GC cycle more
    or less. The only time a pool will exist is when it's active. I'm assuming
    weak references don't slow down GC all that much.

    --
    Joe Seigh

    When you get lemons, you make lemonade.
    When you get hardware, you make software.
    Joe Seigh, Feb 9, 2007
    #10
  11. Chris

    Lew Guest

    Joe Seigh wrote:
    > Not so much that the gc is so much faster than it's not really all that
    > worse
    > and the effects of tying up a lot of memory in object pools is more
    > problematic.


    The Java mechanisms and GC tend to be much faster than hand-rolled object
    pooling (except for resource guards, but then it's not memory that is the slow
    part any more).

    According to what I've read.

    > The copying collector relies on having enough memory so you don't run out
    > between normal GC cycles.


    What do you mean by "between normal GC cycles"? From what I've read, GC runs
    when memory runs low, not on some kind of timer.

    > If you are running out of memory and forcing GC to run more often,
    > you could just use an object pool and use a WeakReference to
    > point to the object pool. The pool will be reclaimed on every GC cycle
    > more or less. The only time a pool will exist is when it's active. I'm
    > assuming weak references don't slow down GC all that much.


    I have not run across any reference that ties weak references to any
    performance considerations for garbage collection.

    The point is that you do not need to use object pooling. The Java memory
    allocator is more efficient, by all accounts, than any local policy can hope
    to achieve, and using custom schemes is only likely to interfere with the Java
    mechanism.

    The whole point of the Java memory mechanism is to remove responsibility for
    memory management from the programmer. Why would anyone want to mess with that?

    If you want to tune the collector, that's an entirely different matter. You
    are much more likely to gain joy from the java -X and -XX parameters than any
    in-process approach.

    There is a common theme running in this newsgroup of people not wanting Java
    to be Java. This seems to blind them to the strengths of the language, as they
    focus on perceived weaknesses. No language is perfect. Part of the art of
    programming involves playing to the strengths of the tools at hand.

    Java provides a lot of help with memory management. Use what it offers before
    trying to defeat it.

    - Lew
    Lew, Feb 9, 2007
    #11
  12. Chris

    Joe Seigh Guest

    Lew wrote:
    > The point is that you do not need to use object pooling. The Java memory
    > allocator is more efficient, by all accounts, than any local policy can
    > hope to achieve, and using custom schemes is only likely to interfere
    > with the Java mechanism.


    For most Java programmers, that is likely true.

    >
    > The whole point of the Java memory mechanism is to remove responsibility
    > for memory management from the programmer. Why would anyone want to mess
    > with that?
    >


    Well, I mess with a whole lot of things that people think can't be improved.
    Some things can be improved by quite a bit (orders of magnitude).


    --
    Joe Seigh

    When you get lemons, you make lemonade.
    When you get hardware, you make software.
    Joe Seigh, Feb 9, 2007
    #12
  13. Chris

    Andy Dingley Guest

    On 9 Feb, 00:43, Lew <> wrote:

    > Of course, in Java one does not often have to explicitly "dispose of" an object.


    One certainly must if it's not garbage collectable!

    A problem with Java in general, particularly with newbie coders who've
    only learned Java from scratch, is that they pay no attention to gc at
    all and just assume that "the maid will clear it up". This works fine
    for most pure Java objects, but if you start having some object that
    represents another resource and isn't automatically disposable then
    you're back to needing to manage these manually (at least to some
    degree). Leaving it up to Java's gc means that none of them ever get
    disposed, and you've a leak.

    Languages that reduce the need to think about everything are great for
    fast development, but when this extends into a mindset that no longer
    thinks at all, it leads to sloppy development.
    Andy Dingley, Feb 9, 2007
    #13
  14. Chris

    Lew Guest

    Andy Dingley wrote:
    > On 9 Feb, 00:43, Lew <> wrote:
    >
    >> Of course, in Java one does not often have to explicitly "dispose of" an object.

    >
    > One certainly must if it's not garbage collectable!
    >
    > A problem with Java in general, particularly with newbie coders who've
    > only learned Java from scratch, is that they pay no attention to gc at
    > all and just assume that "the maid will clear it up". This works fine
    > for most pure Java objects, but if you start having some object that
    > represents another resource and isn't automatically disposable then
    > you're back to needing to manage these manually (at least to some
    > degree). Leaving it up to Java's gc means that none of them ever get
    > disposed, and you've a leak.


    I completely agree. Note that I disclaimed the case where an object manages an
    external resource, but then in that case it is no longer a memory issue. The
    original question had to do with memory allocation and deallocation overhead,
    and in that context Java's mechanism is entirely satisfactory.

    With regard to not thinking and assumptions about the maid's thoroughness you
    have the right of it. Playing to the strengths of the Java model does not mean
    ignoring it, it means understanding it so well that you can take advantage of it.

    If you understand the memory model then, for example, you can exploit things
    like the WeakReference class.

    WRT external resources, Java gives one the finally block.

    - Lew
    Lew, Feb 9, 2007
    #14
  15. Chris

    Chris Uppal Guest

    Chris wrote:

    > Does this hold true when your objects are very large, though? What if
    > your object contains a byte [] of length 100K? Or 1Mb?


    It is trivially true that object pooling /can/ make sense. The lifetime cost
    of an object is the sum of

    the time taken to allocate its store
    the time taken to zero that store
    the time taken in user-code initialising it
    the time take to reclaim that store

    If you use object pooling then the equation changes to

    the time taken to reset the object's state
    the time take to manage the pool

    So if an object can be "reset" faster than it could be
    created+zeroed+initialised then you have a potential saving. If that saving is
    great enough to outweigh the costs described by other posters, then you are
    making a nett profit.

    But, it always possible (with sufficient ingenuity) to devise examples of
    objects which take arbitrarily longer to initialise than to reset, so it is
    always possible to devise examples of objects which would benefit from pooling.

    Such examples don't often come up in real code, though. It would probably
    require that the object had an extremely complicated initial internal state but
    one to which it could be returned comparatively cheaply. (E.g. some
    complicated pre-computed wordlist/dictionary).

    Whether you can reach that position just by using larger and larger simple
    arrays seems doubtful. Allocating a single object, even a large one, is not an
    expensive operation in itself. It's difficult to imagine a scenario where the
    initialisation (if any) of the array in user code was significantly cheaper
    than resetting it. So that leaves only the time taken by the VM to zero the
    store as a potentially worthwhile saving. That is obviously proportional to
    the size of the array, so that's looking promising so far. But consider the
    overall life of the array -- presumably the code is going to do something with
    it, so it seems almost certain that most positions in the array will be written
    to, and read, at least once by the application code. Both of those operations
    will be at least as expensive as the initial zeroing (per slot), so the time
    saved can be no more than 1/3 of the total runtime and, in realistic
    applications, almost certainly much less -- which moves the potential saving
    into the "it's not worth bothering with" category.

    Of course, that last paragraph assumes that the VM/GC implementation is such
    that it has no great difficulty with "churning" large objects -- that does
    depend on the implementation. It would be interesting to how the numbers work
    out in practise rather than in theory[*].

    -- chris

    [*] And please, don't anyone at this point produce that tired old cliché about
    how "theory and practise are the same in theory [etc]"
    Chris Uppal, Feb 9, 2007
    #15
  16. Chris

    Chris Uppal Guest

    Lew wrote:

    > What do you mean by "between normal GC cycles"? From what I've read, GC
    > runs when memory runs low, not on some kind of timer.


    Depends on the GC implementation. There are algorithms which collect garbage
    on a continuous basis. And it's not an unreasonable technique, for VMs aimed
    at desktop applications and the like, to run a low-priority GC task on a timer
    (that's a VM implementation technique, mind, not a recommendation that desktop
    /applications/ should be coded that way).

    And in fact the Sun GC implementations, which use multiple spaces, don't really
    have a concept of waiting until memory runs low before running GC. They do
    have a, comparatively expensive, "full" GC which is run if memory does run low,
    but the object of the design is to avoid doing that as far as possible.
    Ideally to eliminate it altogether.

    -- chris
    Chris Uppal, Feb 9, 2007
    #16
  17. Chris

    Andy Dingley Guest

    On 9 Feb, 16:18, "Chris Uppal" <-
    THIS.org> wrote:

    > So if an object can be "reset" faster than it could be
    > created+zeroed+initialised then you have a potential saving.


    Of course. Except that zeroed+initialised is trivially fast,
    management isn't and creation is usually pretty speedy too. Your
    conclusion here is _far_ from guaranteed.

    There's an equally over-simplifed proof that garbage collection is
    just one form of pool management, so anything that overlays another
    layer of pool management over any existing layer of pool management
    _must_ be slower than just having a single layer.


    It's all Melvyn Bragg's fault. I've been reading Popper this week and
    now I can't believe _anything_ anyone tells me!
    Andy Dingley, Feb 9, 2007
    #17
  18. Chris

    Chris Uppal Guest

    Andy Dingley wrote:

    [me:]
    > > So if an object can be "reset" faster than it could be
    > > created+zeroed+initialised then you have a potential saving.

    >
    > Of course. Except that zeroed+initialised is trivially fast,
    > management isn't and creation is usually pretty speedy too. Your
    > conclusion here is _far_ from guaranteed.


    I believe that you have misunderstood the whole of my post.


    > It's all Melvyn Bragg's fault. I've been reading Popper this week and
    > now I can't believe _anything_ anyone tells me!


    Reading it seems to have damaged your ability to follow an argument too ;-)
    Maybe your expectations are now set too high...

    -- chris
    Chris Uppal, Feb 9, 2007
    #18
  19. Chris

    Chris Guest

    Chris wrote:
    > I've read recently that object allocation in recent JVMs is so fast that
    > it doesn't make sense to create object pools. Just create a new object
    > when you need it and let the garbage collector do the work.
    >
    > Does this hold true when your objects are very large, though? What if
    > your object contains a byte [] of length 100K? Or 1Mb?
    >
    > What's the breakeven point beyond which it makes sense to reuse objects?


    Thanks, everybody, for the insights, but nobody really shed any light on
    the original question, which was: what's the breakeven point?

    So I wrote a little code to test the question, pasted below. The code
    simply allocates byte [] objects of varying sizes. Here are the results,
    for 100,000 iterations, elapsed time in milliseconds:

    bufsize 10 elapsed = 16
    bufsize 1024 elapsed = 47
    bufsize 10240 elapsed = 313
    bufsize 102400 elapsed = 3078
    bufsize 1048576 elapsed = 316540
    bufsize 10Mb, terminated because it took too long

    JDK 1.6, JVM memory = the default 64mb (increasing JVM memory did not
    alter the results).

    Contrary to some of the earlier advice in this thread, it is *not*
    trivially fast to allocate larger objects. Allocation time increases
    roughly linearly as object sizes increase.

    Given that fetching an object from a pool 100,000 times should generally
    not take more than a few milliseconds (locking & sync included), it
    looks like object pooling is a necessity when speed is important and
    objects get larger than a few dozen Kb in size.

    public static void main(String[] argv) throws Exception {

    // 10, 1K, 10K, 100K, 1Mb, 10Mb
    int [] BUFSIZES = {10, 1024, 10 * 1024, 100 * 1024, 1024 * 1024, 10
    *1024 * 1024};
    int ITERATIONS = 100000;

    for (int bufSizePtr = 0; bufSizePtr < BUFSIZES.length; bufSizePtr++) {
    int bufSize = BUFSIZES[bufSizePtr];
    long start = System.currentTimeMillis();
    for (int i = 0; i < ITERATIONS; i++) {
    byte[] buf = new byte [bufSize];
    buf[0] = 1;
    }
    long elapsed = System.currentTimeMillis() - start;
    System.out.println("bufsize " + bufSize + " elapsed = " + elapsed);
    }
    }
    Chris, Feb 9, 2007
    #19
  20. Chris

    Chris Guest

    >
    > Contrary to some of the earlier advice in this thread, it is *not*
    > trivially fast to allocate larger objects. Allocation time increases
    > roughly linearly as object sizes increase.
    >


    Here's more evidence. The code was rewritten to use System.nanoTime()
    and to double the amount of memory allocated on each cycle. Times are
    still in millisec:

    bufsize 1 elapsed = 61
    bufsize 2 elapsed = 23
    bufsize 4 elapsed = 22
    bufsize 8 elapsed = 23
    bufsize 16 elapsed = 24
    bufsize 32 elapsed = 29
    bufsize 64 elapsed = 41
    bufsize 128 elapsed = 60
    bufsize 256 elapsed = 107
    bufsize 512 elapsed = 192
    bufsize 1024 elapsed = 344
    bufsize 2048 elapsed = 669
    bufsize 4096 elapsed = 1308
    bufsize 8192 elapsed = 2519
    bufsize 16384 elapsed = 4970
    bufsize 32768 elapsed = 9934
    bufsize 65536 elapsed = 19732
    bufsize 131072 elapsed = 39455
    bufsize 262144 elapsed = 73419

    For very small objects, allocation time is constant. As you get to
    larger objects, allocation time doubles as you double the size. Above
    ~1K, allocation time is close to linear.

    Same pattern applies for JDK 1.4, 1.5, and 1.6.
    Chris, Feb 9, 2007
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Phil Sandler

    NULLs from SQL--does this make sense?

    Phil Sandler, Aug 19, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    320
    Phil Sandler
    Aug 19, 2004
  2. Jason
    Replies:
    0
    Views:
    339
    Jason
    Oct 12, 2004
  3. Mike
    Replies:
    5
    Views:
    2,742
    Adam Maass
    Jul 1, 2003
  4. Timo Nentwig
    Replies:
    31
    Views:
    1,183
    Chris Smith
    May 13, 2004
  5. Jason
    Replies:
    0
    Views:
    179
    Jason
    Jul 6, 2004
Loading...

Share This Page