Re: How to measure the size of an object?

Discussion in 'Java' started by Oliver Wong, May 29, 2006.

  1. Oliver Wong

    Oliver Wong Guest

    [Crossposted to comp.lang.java.programmer, since more people lurk there]

    "Scirious" <> wrote in message
    news:...
    >>> People, how can I know how big an object is? I mean, I have an object
    >>> the
    >>> collects data from a stream and when it grows to an especific size I
    >>> need
    >>> to create a new object to continue collecting the data and send the
    >>> other
    >>> one to a thread that records it's content to the disk.
    >>>
    >>> How do I do such thing?


    I wrote:
    >>
    >> Are you sure it's the object's size you need, and not, for
    >> example, the
    >> size of the buffer within that object?
    >>
    >> How about whenever you collect a unit of data, you increment a
    >> counter?
    >>


    OP responds:
    >
    > Actually, I'm not! The thing is, I've already seen the JVM misbehave when
    > it uses a lot of memmory. Lets say, for example, If I have 750 MB of RAM
    > and an object needs to store 1.5+ GB of data my system or the JVM is
    > likely to crash. So, for example, I'm writing an e-mail client and I have
    > an object to store the headers I download from the server. I may want to
    > specify that, after downloading 250 MB of headers, I want them to be
    > recorded on disc so the object can give place to a new one and be
    > collected
    > from memory.
    >
    > This way I avoid the possibility of an object getting as big as 1 GB. But
    > to do so I need a way to measure the size it has and, implementing a
    > counter is not the best way sisnce the size of the headers may differ from
    > message to message.


    Not sure the best way to solve this one, which is why I'm crossposting
    it to c.l.j.programmer, so others can contribute.

    I've read about one technique but never tried it: Allocate a big chunk
    of memory at the start of your program, do your memory operation which might
    result in an OutOfMemoryException (OOME), catch it, release the big chunk of
    memory that you got, so you now have some wiggle room to work with,
    serialize the buffer to disk, re-allocate another big chunk of memory (and
    assert that it doesn't fail, since you're supposed to have a lot of free
    memory after the serialization), and repeat.

    - Oliver
     
    Oliver Wong, May 29, 2006
    #1
    1. Advertising

  2. Oliver Wong wrote:
    > [Crossposted to comp.lang.java.programmer, since more people lurk there]
    >
    > "Scirious" <> wrote in message
    > news:...
    >>>> People, how can I know how big an object is? I mean, I have an
    >>>> object the
    >>>> collects data from a stream and when it grows to an especific size I
    >>>> need
    >>>> to create a new object to continue collecting the data and send the
    >>>> other
    >>>> one to a thread that records it's content to the disk.
    >>>>
    >>>> How do I do such thing?

    >
    > I wrote:
    >>>
    >>> Are you sure it's the object's size you need, and not, for
    >>> example, the
    >>> size of the buffer within that object?
    >>>
    >>> How about whenever you collect a unit of data, you increment a
    >>> counter?
    >>>

    >
    > OP responds:
    >>
    >> Actually, I'm not! The thing is, I've already seen the JVM misbehave when
    >> it uses a lot of memmory. Lets say, for example, If I have 750 MB of RAM
    >> and an object needs to store 1.5+ GB of data my system or the JVM is
    >> likely to crash. So, for example, I'm writing an e-mail client and I have
    >> an object to store the headers I download from the server. I may want to
    >> specify that, after downloading 250 MB of headers, I want them to be
    >> recorded on disc so the object can give place to a new one and be
    >> collected
    >> from memory.
    >>
    >> This way I avoid the possibility of an object getting as big as 1 GB. But
    >> to do so I need a way to measure the size it has and, implementing a
    >> counter is not the best way sisnce the size of the headers may differ
    >> from
    >> message to message.

    >
    > Not sure the best way to solve this one, which is why I'm
    > crossposting it to c.l.j.programmer, so others can contribute.
    >
    > I've read about one technique but never tried it: Allocate a big
    > chunk of memory at the start of your program, do your memory operation
    > which might result in an OutOfMemoryException (OOME), catch it, release
    > the big chunk of memory that you got, so you now have some wiggle room
    > to work with, serialize the buffer to disk, re-allocate another big
    > chunk of memory (and assert that it doesn't fail, since you're supposed
    > to have a lot of free memory after the serialization), and repeat.


    In this scenario I'd just count downloaded bytes (I guess this is what
    you meant by "counter" above). As easy as that. With a limit of 250MB
    some bytes overhead for reference to an object's class etc. are negligible.

    And I think you also correctly pointed out that "the size of an object"
    is a too fuzzy term to deal with. Strictly speaking, the size of an
    object is 8 bytes + 4 * size of an object reference + size of any POD
    data members (roughly, I'm not 100% sure about the figures). But then
    again, for a String you would certainly also want to count the size of
    the char array that holds the actual string data etc. Matters become
    even more complicated when two strings share the same array, e.g. you
    use this constructor: public String(String original). Does the char
    array count for each of them? Or is half the length counted for each of
    them? This becomes even more complex with collections and user defined
    classes...

    Kind regards

    robert
     
    Robert Klemme, May 29, 2006
    #2
    1. Advertising

  3. Oliver Wong

    Oliver Wong Guest

    "Robert Klemme" <> wrote in message
    news:...
    >> "Scirious" <> wrote in message


    >>> So, for example, I'm writing an e-mail client and I have
    >>> an object to store the headers I download from the server. I may want to
    >>> specify that, after downloading 250 MB of headers, I want them to be
    >>> recorded on disc so the object can give place to a new one and be
    >>> collected
    >>> from memory.

    [...]
    > In this scenario I'd just count downloaded bytes (I guess this is what you
    > meant by "counter" above). As easy as that. With a limit of 250MB some
    > bytes overhead for reference to an object's class etc. are negligible.


    I think it's a bit trickier than that. Let's say the downloaded bytes
    are 0x48, 0x65, 0x6C, 0x6C, 0x6F. I.e. the string "Hello" if decoded using
    ASCII. If the program then takes these 5 bytes and stores them as a String,
    the JVM is likely to use UTF-16 internally, which will result in 10 bytes
    being used (2 for each character).

    If the downloaded bytes UTF-8 instead of ASCII, and some of the
    characters were above codepoint 256, then the calculation is not simply a
    matter of doubling the downloaded bytes. Some characters will have taken 1
    byte to download, 2 bytes to store; others will have taken 2 bytes to
    download, 2 bytes to store. Still others, 3 bytes to download, 2 bytes to
    store; 3 bytes to download, 3 bytes to store, and so on with various other
    combinations.

    - Oliver
     
    Oliver Wong, May 29, 2006
    #3
  4. Oliver Wong wrote:
    >
    > "Robert Klemme" <> wrote in message
    > news:...
    >>> "Scirious" <> wrote in message

    >
    >>>> So, for example, I'm writing an e-mail client and I have
    >>>> an object to store the headers I download from the server. I may
    >>>> want to
    >>>> specify that, after downloading 250 MB of headers, I want them to be
    >>>> recorded on disc so the object can give place to a new one and be
    >>>> collected
    >>>> from memory.

    > [...]
    >> In this scenario I'd just count downloaded bytes (I guess this is what
    >> you meant by "counter" above). As easy as that. With a limit of
    >> 250MB some bytes overhead for reference to an object's class etc. are
    >> negligible.

    >
    > I think it's a bit trickier than that. Let's say the downloaded bytes
    > are 0x48, 0x65, 0x6C, 0x6C, 0x6F. I.e. the string "Hello" if decoded
    > using ASCII. If the program then takes these 5 bytes and stores them as
    > a String, the JVM is likely to use UTF-16 internally, which will result
    > in 10 bytes being used (2 for each character).
    >
    > If the downloaded bytes UTF-8 instead of ASCII, and some of the
    > characters were above codepoint 256, then the calculation is not simply
    > a matter of doubling the downloaded bytes. Some characters will have
    > taken 1 byte to download, 2 bytes to store; others will have taken 2
    > bytes to download, 2 bytes to store. Still others, 3 bytes to download,
    > 2 bytes to store; 3 bytes to download, 3 bytes to store, and so on with
    > various other combinations.


    Maybe I wasn't clear enough: I meant, ignore real sizes and just use the
    bytecount as limit - alternatively, if the data is converted to chars
    then use the char count as limit. I wouldn't bother to think about
    actual sizes in bytes - the important part for me would be that there is
    a *limit* to the size.

    Kind regards

    robert
     
    Robert Klemme, May 30, 2006
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Christopher Denis
    Replies:
    4
    Views:
    1,089
    Ralf Hildebrandt
    Jan 30, 2005
  2. Joe Fallon
    Replies:
    5
    Views:
    3,517
    Joe Fallon
    Jul 28, 2005
  3. asit dhal

    how to measure size of an array

    asit dhal, Feb 14, 2007, in forum: C Programming
    Replies:
    14
    Views:
    556
  4. Jason Cavett

    Preferred Size, Minimum Size, Size

    Jason Cavett, May 23, 2008, in forum: Java
    Replies:
    5
    Views:
    12,608
    Michael Jung
    May 25, 2008
  5. Max Williams
    Replies:
    4
    Views:
    111
    Max Williams
    Nov 3, 2009
Loading...

Share This Page