Webservices returning large amount of data - need suggestions

Discussion in 'Java' started by DarthLuch@gmail.com, May 3, 2006.

  1. Guest

    I'd like to get suggestions on best practice way to return a large
    amount of xml data using a webservice function.

    Here's what I do today:
    Using java, I write a function that will return an array of some
    object. I write the function to loop through the database, getting each
    record, perform any needed manipulations on it, then add each modified
    record to an array. At the end, I marshall the array to a string (using
    Castor) and return the xml-representation of it. I then use Apache Axis
    to expose the function as a web-service. I know this is not a
    well-though design, but it is easy to program. So now the problem I'm
    having is that I run out of memory... With a large amount of data it
    will eventually crash once it reaches its breaking point of adding so
    many items to the array. I also know that taking the entire object and
    marshalling it to a string to be returned is probably not the best way
    either. Any suggestions and pointers would be appreciated.

    My clients will usually deploy to a Tomcat server, using the default
    settings. I'm not sure if there's even a way to increase the memory
    allotted to it, but I'm not sure that is the best way to proceed anyway.
     
    , May 3, 2006
    #1
    1. Advertising

  2. seank76 Guest

    If I understand the problem correctly, you just need to be able to
    return a huge string via webservice, right?
    The first thing that comes to my mind is to implement iterator type of
    service.
    In other words, you would only return a predefined size of string and
    the client will have to call "hasNext()" method to see if there are
    more, etc.
    This way you always know exactly how much memory is required regardless
    of the length of the string.
    I guess if the performance matters and multiple hits to the web-methods
    are not desired, this might not be the ideal solution for you...
     
    seank76, May 3, 2006
    #2
    1. Advertising

  3. Luch Guest

    Yeah multiple hits to the webmethods is not desired. I also prefer that
    the methods stay state(or session?) less, and each method is it's own
    instance, etc.
    Would like other ideas.
     
    Luch, May 4, 2006
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. James
    Replies:
    29
    Views:
    2,185
    Dale King
    Apr 15, 2006
  2. Thomas Pototschnig

    Need a large amount of memory!

    Thomas Pototschnig, Apr 30, 2005, in forum: Java
    Replies:
    1
    Views:
    363
    Harald
    Apr 30, 2005
  3. dreamvigile

    how to read in large amount of data in C?

    dreamvigile, Jul 18, 2003, in forum: C Programming
    Replies:
    0
    Views:
    345
    dreamvigile
    Jul 18, 2003
  4. Adam
    Replies:
    3
    Views:
    153
    steve-o
    Oct 25, 2004
  5. mehdi_mousavi

    Returning a large amount of records

    mehdi_mousavi, May 6, 2006, in forum: ASP .Net Web Services
    Replies:
    3
    Views:
    134
    Peter Morris [Droopy eyes software]
    May 7, 2006
Loading...

Share This Page