NIO - OutOfMemoryException

Discussion in 'Java' started by Camille, Nov 15, 2005.

  1. Camille

    Camille Guest

    Hi,
    first of all, i'm not sure of the newsgroup to post this message.
    If i'm not in the good place, please tell me.

    I'm using NIO in a client server project, and from a sudden, I get
    OutOfMemoryException.
    After some search on the web, I found that ByteBuffer.allocateDirect
    was often a problem if the heap size was too low.

    BUT, the problem is that I get this exception when writing on NIO, not
    when reading.

    Here's the code :

    FIFOQueue m_responsesQueue = new FIFOQueue();
    ....
    AKSResponse response = null;
    SocketChannel client;
    Charset charset = Charset.forName("UTF-8"); // client encoding

    while (m_responsesQueue.size() > 0)
    {
    response = (AKSResponse) m_responsesQueue.getFirst();
    client = response.getChannel();
    if (client.isOpen())
    {
    if (!client.socket().isClosed())
    {
    if (!client.socket().isOutputShutdown())
    {
    client.write(charset.encode(response.getData()));
    }
    }
    }
    response.free();
    response = null;
    m_responsesQueue.dequeue();
    }

    And I get the following :

    java.lang.OutOfMemoryError
    at sun.misc.Unsafe.allocateMemory(Native Method)
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:99)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
    at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:54)
    at sun.nio.ch.IOUtil.write(IOUtil.java:69)
    at
    sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:300)
    at aks.gameServer.net.AKSWriteDaemon.run
    (AKSWriteDaemon.java:102)

    The line 102 being the client.write(charset.......

    I presume I have to allocate more memory to my VM with the Xmx, Xmn,
    X... settings, but I'm confused with all the possibilites : what about
    XNewSize, XMaxNewSize, Xms, ...

    Thanks in advance for your help,

    Camille
    --
     
    Camille, Nov 15, 2005
    #1
    1. Advertising

  2. Camille wrote:
    > first of all, i'm not sure of the newsgroup to post this message.
    > If i'm not in the good place, please tell me.


    Looks good to me.

    >
    > I'm using NIO in a client server project, and from a sudden, I get
    > OutOfMemoryException.
    > After some search on the web, I found that ByteBuffer.allocateDirect
    > was often a problem if the heap size was too low.


    > java.lang.OutOfMemoryError
    > at sun.misc.Unsafe.allocateMemory(Native Method)
    > at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:99)
    > at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
    > at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:54)
    > at sun.nio.ch.IOUtil.write(IOUtil.java:69)
    > at
    > sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:300)
    > at aks.gameServer.net.AKSWriteDaemon.run
    > (AKSWriteDaemon.java:102)


    The code appears to cache direct buffers, if you are use non-direct
    buffers. Possibly there is a significant delay between SoftReferences
    getting cleared and reference handler cleansing the buffer. If you are
    on Linux or Solaris, you might see the problem disappear when running as
    root (I believe thread priorities actually work then). I don't know the
    details of interaction between SoftReferences, finalisable objects, the
    garbage collector and the reference handler. If it requires another
    appropriate GC cycle to queue the finaliser after the SoftReference is
    cleared, then that may well cause problems.

    If you have many threads, then you can end up allocating a silly number
    of these temporary direct buffers, as the cache is thread-local. Their
    interaction with the garbage collector is such that you don't want to be
    doing that.

    The obvious solution is to use directly allocated buffers yourself. But
    be sure they are long lived, rather than constantly allocated and collected.

    > I presume I have to allocate more memory to my VM with the Xmx, Xmn,
    > X... settings, but I'm confused with all the possibilites : what about
    > XNewSize, XMaxNewSize, Xms, ...


    In general you should set -Xmx to how much heap you want. Set the
    minimum heap, -Xms, to the same in order to avoid unnecessary GC. If you
    don't want to do that, I believe using -server changes SoftReference
    calculations to use free heap space if maximum were allocated in place
    of current free heap space. IIRC, there is a magic -XX option to change
    the maximum space allocable for direct buffers, as a fraction of heap size.

    Tom Hawtin
    --
    Unemployed English Java programmer
    http://jroller.com/page/tackline/
     
    Thomas Hawtin, Nov 16, 2005
    #2
    1. Advertising

  3. Camille

    Guest

    Hi folks,
    Just a little message to say I finnaly found the solution.
    The problem was the SocketChannel.write implementation.
    Indeed, even if you don't use DirectByteBuffer in your code,
    SocketChannel.write does it for you.
    This means that every time you call the write method, a new
    DirectByteBuffer is created. And since there is no real and effective
    GC on this type of objects, on heavy load, your application simply
    crash.

    The solution ? I did create a big DirectBuffer by myself, copy the
    encoded result into it, then use it to do the write stuff.
    Here's the code :

    ByteBuffer outputDirectBuffer =
    ByteBuffer.allocateDirect(OUTPUT_DIRECTBUFFER_SIZE);
    int dataSent, dataToSend;
    while (m_responsesQueue.size() > 0) {
    response = (AKSResponse) m_responsesQueue.getFirst();
    client = response.getChannel();
    if (client.isOpen()) {
    if (!client.socket().isClosed()) {
    if (!client.socket().isOutputShutdown()) {
    outputDirectBuffer.clear();
    outputDirectBuffer.put(charset.encode(response.getData()));
    outputDirectBuffer.flip();
    dataToSend = outputDirectBuffer.limit() -
    outputDirectBuffer.position();
    dataSent = client.write(outputDirectBuffer);
    if (dataSent != dataToSend) {
    //.. not enough bytes were sent. Maybe the network heap full ?
    logger.fatal ("Unable to send " + dataToSend + " bytes : " +
    dataSend + " sent");
    }
    }
    }
    }
    response.free();
    response = null;
    m_responsesQueue.dequeue();
    }


    Using this, I now achieve to let about 6.000 players join games on my
    server, without any memory problem).
    Thanks again for your help,

    Camille
     
    , Dec 5, 2005
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?UmlwdWwgSGFuZGE=?=

    Base Exception:System.OutOfMemoryException error message

    =?Utf-8?B?UmlwdWwgSGFuZGE=?=, Nov 17, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    999
    Natty Gur
    Nov 18, 2003
  2. =?Utf-8?B?amF6?=

    OutOfMemoryException

    =?Utf-8?B?amF6?=, Dec 10, 2003, in forum: ASP .Net
    Replies:
    2
    Views:
    352
    =?Utf-8?B?SmF6?=
    Dec 16, 2003
  3. Pyramis
    Replies:
    0
    Views:
    403
    Pyramis
    Jan 25, 2004
  4. =?Utf-8?B?U3Rhbg==?=

    Exception of type System.OutOfMemoryException is thrown

    =?Utf-8?B?U3Rhbg==?=, Feb 16, 2004, in forum: ASP .Net
    Replies:
    11
    Views:
    27,974
    Ken Cox [Microsoft MVP]
    Dec 28, 2004
  5. iksrazal

    NIO with timeouts != NIO?

    iksrazal, Jun 17, 2004, in forum: Java
    Replies:
    1
    Views:
    6,282
    iksrazal
    Jun 18, 2004
Loading...

Share This Page