any datatype which can handle GB's of Data.

Discussion in 'Java' started by Garg, Apr 23, 2007.

  1. Garg

    Garg Guest

    Hi All,

    Is there any variable in java like Stringbuffer which can handle GB's
    of char data.

    As far as i know every variable has a limit is there any variable
    which can handle unlimited data.

    it sounds little bit stupid but which data type or design pattern can
    solve this problem.

    Thanks
    Taurn Garg
     
    Garg, Apr 23, 2007
    #1
    1. Advertising

  2. Garg

    Daniel Pitts Guest

    On Apr 23, 6:03 am, Garg <> wrote:
    > Hi All,
    >
    > Is there any variable in java like Stringbuffer which can handle GB's
    > of char data.
    >
    > As far as i know every variable has a limit is there any variable
    > which can handle unlimited data.


    No *computer* can handle unlimitted data, so how could any type in
    Java do so?

    In general, you are limitted in any programming language by how much
    memory (virtual or otherwise) you have. Sometimes there are other
    limits, but most people don't have enough memory to hit those other
    limits.

    Its generally better to accept this, and use algorithms that support
    some sort of streaming or subset processing.
    >
    > it sounds little bit stupid but which data type or design pattern can
    > solve this problem.


    Its not a data type or design pattern. In this case, its an algorithm
    or data structure.


    Hope this helps,
    Daniel.
     
    Daniel Pitts, Apr 23, 2007
    #2
    1. Advertising

  3. On 23.04.2007 15:03, Garg wrote:
    > Is there any variable in java like Stringbuffer which can handle GB's
    > of char data.


    In memory? On a 32 bit Windows system a JRE can use approx. 1.5GB heap.

    > As far as i know every variable has a limit is there any variable
    > which can handle unlimited data.
    >
    > it sounds little bit stupid but which data type or design pattern can
    > solve this problem.


    The question is: what do you want to do with that data? Chances are
    that you can only deal with this data on disk or in a database.

    robert
     
    Robert Klemme, Apr 23, 2007
    #3
  4. Daniel Pitts wrote:
    > On Apr 23, 6:03 am, Garg <> wrote:
    >> Hi All,
    >>
    >> Is there any variable in java like Stringbuffer which can handle GB's
    >> of char data.
    >>

    > No *computer* can handle unlimitted data, so how could any type in
    > Java do so?
    >


    Some objects such as java.net.Socket or java.io_OutputStream can handle
    unlimited data...

    But that's probably not what the OP meant
     
    David Zimmerman, Apr 23, 2007
    #4
  5. Garg

    Mark Rafn Guest

    Garg <> wrote:
    >Is there any variable in java like Stringbuffer which can handle GB's
    >of char data.


    StringBuffer is a class, not a variable. I don't mean this just as a nitpick;
    the way you describe a problem has a very large affect on the types of
    answers you'll get.

    If you're imprecise in describing what you're doing, and unclear in your
    description of the Java language, you'll get general answers that assume
    you're a fairly inexperienced developer. Which may be what you're looking
    for :)

    >As far as i know every variable has a limit is there any variable
    >which can handle unlimited data.


    Every computer has a limit. AFAIK, Java's limits for this kind of class are
    based on available memory and the fact that arrays must be indexable by a
    32-bit signed integer. I'll be surprised if someone says there's no JVM on
    any hardware that can handle 1.8 billion characters in a StringBuilder.

    But then you get into WHY you want to do such a thing.

    >it sounds little bit stupid but which data type or design pattern can
    >solve this problem.


    Generally, look for ways to avoid having it all in memory at the same time.
    If possible, a streaming approach is good: read from a file/socket a little at
    a time, process it, and write to a file/socket. Alternately,
    java.io.RandomAccessFile lets you seek() to the section you want to work on.
    In both cases, you'll need to be careful about the distinction between bytes
    and characters.
    --
    Mark Rafn <http://www.dagon.net/>
     
    Mark Rafn, Apr 23, 2007
    #5
  6. Garg

    Guest

    On Apr 23, 9:03 am, Garg <> wrote:
    > Hi All,
    >
    > Is there any variable in java like Stringbuffer which can handle GB's
    > of char data.
    >
    > As far as i know every variable has a limit is there any variable
    > which can handle unlimited data.
    >
    > it sounds little bit stupid but which data type or design pattern can
    > solve this problem.
    >
    > Thanks
    > Taurn Garg



    I can create such a variable for you - for a slight fee! :)
    Seriously, you can use a File (or even memory mapped file) and just
    put a delegate of some sort. E.g.

    public class BigBuf {
    // herein are the public external methods that look just like a
    StringBuffer or what have you
    // herein is the underlieing implementation that actually write/reads
    to/from a file.
    }
     
    , Apr 23, 2007
    #6
  7. Garg

    Garg Guest

    thanks for reply,

    What i really need to do is that that i have to take the data from
    database and then send it to one JSP page where javascript will handle
    that data to show to the user.
    the reading from database and sending to jsp or gui is been done ny
    serlvet.

    so what i am doing is like i am reading the data from database one by
    one record and then creating a table and puting that in the
    StringBuffer. but as data can be of any size. String buffer will not
    be able to handle that. so in this case what should i do.

    thanks
    tarun
     
    Garg, Apr 24, 2007
    #7
  8. Garg

    Lew Guest

    Garg wrote:
    > thanks for reply,
    >
    > What i really need to do is that that i have to take the data from
    > database and then send it to one JSP page where javascript will handle
    > that data to show to the user.
    > the reading from database and sending to jsp or gui is been done ny
    > serlvet.
    >
    > so what i am doing is like i am reading the data from database one by
    > one record and then creating a table and puting that in the
    > StringBuffer. but as data can be of any size. String buffer will not
    > be able to handle that. so in this case what should i do.


    Don't.

    Separate String (or StringBuilder or other type such as java.sql.Date) for
    each column in each row (not record). Don't show all gazillion rows (not
    records) at once - use some paging.

    The datatype to hold everything at once is the "RDBMS" type. Everything else
    should only hold a view into that. Certainly even if your server could suck
    the whole data store into memory at once, there's no way someone's browser can.

    --
    Lew
     
    Lew, Apr 24, 2007
    #8
  9. Garg

    Michael Guest

    On Apr 23, 9:03 am, Garg <> wrote:
    > Hi All,
    >
    > Is there any variable in java like Stringbuffer which can handle GB's
    > of char data.
    >
    > As far as i know every variable has a limit is there any variable
    > which can handle unlimited data.
    >
    > it sounds little bit stupid but which data type or design pattern can
    > solve this problem.
    >
    > Thanks
    > Taurn Garg


    Research the java.nio.ByteBuffer* classes such - specifically the
    MemoryMapped (google is your friend). Basically disk-backed buffers
    that you can perform many convenient slicing on which is more
    efficient than sequentially navigating a file. If you've got the RAM,
    then the OS should properly cache the contents of the file.
     
    Michael, Apr 24, 2007
    #9
  10. Garg

    Daniel Pitts Guest

    On Apr 24, 4:42 am, Garg <> wrote:
    > thanks for reply,
    >
    > What i really need to do is that that i have to take the data from
    > database and then send it to one JSP page where javascript will handle
    > that data to show to the user.
    > the reading from database and sending to jsp or gui is been done ny
    > serlvet.
    >
    > so what i am doing is like i am reading the data from database one by
    > one record and then creating a table and puting that in the
    > StringBuffer. but as data can be of any size. String buffer will not
    > be able to handle that. so in this case what should i do.
    >
    > thanks
    > tarun


    First, you should use StringBuilder over StringBuffer in most cases.
    Second, What makes you think StringBuilder can't handle it? It can
    handle a surprising amount of data. You don't want to load all of
    your database at once though. Try displaying a row at a time.
     
    Daniel Pitts, Apr 25, 2007
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Meredith
    Replies:
    0
    Views:
    417
    Meredith
    Nov 7, 2003
  2. Leo
    Replies:
    0
    Views:
    471
  3. ashu
    Replies:
    7
    Views:
    331
    David Resnick
    Jan 26, 2006
  4. Jeff
    Replies:
    2
    Views:
    1,189
    Steve C. Orr [MCSD, MVP, CSM, ASP Insider]
    Apr 16, 2007
  5. Replies:
    1
    Views:
    147
    Ken Cox [Microsoft MVP]
    Oct 6, 2006
Loading...

Share This Page