Why is this code not cleaning up int arrays? or How to detect animage is a solid color?

Discussion in 'Java' started by jw, Jul 30, 2008.

  1. jw

    jw Guest

    Hi All,

    I've got one idea I'm currently trying to get this code to better use
    its int arrays, but as written below the system runs out of memory
    after about 6 hours of crunching. The code below is in a loop called
    hundreds of thousands of times, and I'm 97% sure the imgdata int array
    is not getting properly cleaned up. While I'm waiting to see if my
    idea fixed the issue I thought I'd ask if anyone here has a good idea.

    This code splits up a large image into smaller images and saves each
    subimage. The problem code is designed to inspect every 3rd pixel of
    an image and if they are all the same, we assume the image is just a
    solid color so instead of saving the entire image, we check a cache of
    1x1 images for a 1x1 image of the matching color or else we scale it
    to 1x1, put it into the cache and in either case and we set it up to
    save the 1x1 image further below. When I commented out the code that
    states "//begin area of code that is the issue" the system did not run
    out of memory. Anyway, an alternate question is: Is there a better way
    to detect if the image is just a solid color?

    Thanks in advance for your help!

    BufferedImage subImage;
    ByteArrayOutputStream baos;
    int[] imgdata; //variable that is the issue
    for(int i = 0; i < w.tcnt; i++) {
    for (int j = 0; j < w.tcnt; j++) {
    //begin area of code that is the issue
    subImage = w.img.getSubimage(i * tilesize, j * tilesize, tilesize,
    tilesize);
    imgdata =
    ((DataBufferInt)subImage.getRaster().getDataBuffer()).getData();
    if(imgdata.length>0) {
    int firstpixel = imgdata[0];
    int length = imgdata.length;
    while((length=length-3)>0) {
    if(firstpixel!=imgdata[length]) {
    break;
    }
    }
    if(length<4) {
    if(images1x1.containsKey(firstpixel)) {
    subImage = images1x1.get(firstpixel);
    }
    else {
    subImage = subImage.getSubimage(0,0,1,1);
    images1x1.put(firstpixel, subImage);
    }
    }
    }
    //end code that is the issue
    baos = new ByteArrayOutputStream();
    try {
    ImageIO.write(subImage, "PNG", baos);
    //internal call to save the image
    } catch (IOException e) {
    System.err.println("Error converting tile!");
    }
    }
    }
    subImage = null;
    baos = null;
    imgdata = null;
    w.img = null;
    jw, Jul 30, 2008
    #1
    1. Advertising

  2. Re: Why is this code not cleaning up int arrays? or How to detectan image is a solid color?

    jw wrote:
    > Hi All,
    >
    > I've got one idea I'm currently trying to get this code to better use
    > its int arrays, but as written below the system runs out of memory
    > after about 6 hours of crunching. The code below is in a loop called
    > hundreds of thousands of times, and I'm 97% sure the imgdata int array
    > is not getting properly cleaned up. While I'm waiting to see if my
    > idea fixed the issue I thought I'd ask if anyone here has a good idea.
    >
    > This code splits up a large image into smaller images and saves each
    > subimage. The problem code is designed to inspect every 3rd pixel of
    > an image and if they are all the same, we assume the image is just a
    > solid color so instead of saving the entire image, we check a cache of
    > 1x1 images for a 1x1 image of the matching color or else we scale it
    > to 1x1, put it into the cache and in either case and we set it up to
    > save the 1x1 image further below. When I commented out the code that
    > states "//begin area of code that is the issue" the system did not run
    > out of memory. Anyway, an alternate question is: Is there a better way
    > to detect if the image is just a solid color?
    >
    > Thanks in advance for your help!
    >
    > BufferedImage subImage;
    > ByteArrayOutputStream baos;
    > int[] imgdata; //variable that is the issue
    > for(int i = 0; i < w.tcnt; i++) {
    > for (int j = 0; j < w.tcnt; j++) {
    > //begin area of code that is the issue
    > subImage = w.img.getSubimage(i * tilesize, j * tilesize, tilesize,
    > tilesize);
    > imgdata =
    > ((DataBufferInt)subImage.getRaster().getDataBuffer()).getData();
    > if(imgdata.length>0) {
    > int firstpixel = imgdata[0];
    > int length = imgdata.length;
    > while((length=length-3)>0) {
    > if(firstpixel!=imgdata[length]) {
    > break;
    > }
    > }
    > if(length<4) {
    > if(images1x1.containsKey(firstpixel)) {
    > subImage = images1x1.get(firstpixel);
    > }
    > else {
    > subImage = subImage.getSubimage(0,0,1,1);
    > images1x1.put(firstpixel, subImage);


    I guess images1x1 is a HashMap (defined with a suitably large initial
    size). As firstpixel is an int and int has around 4 billion possible
    values this object could conceivably grow quite large.

    Maybe you could check (assert?)
    - that the value of firstpixel is within the range you expect.
    - that images1x1.size() is less than some reasonable limit.


    > }
    > }
    > }
    > //end code that is the issue
    > baos = new ByteArrayOutputStream();
    > try {
    > ImageIO.write(subImage, "PNG", baos);
    > //internal call to save the image
    > } catch (IOException e) {
    > System.err.println("Error converting tile!");
    > }
    > }
    > }
    > subImage = null;
    > baos = null;
    > imgdata = null;
    > w.img = null;



    --
    RGB
    RedGrittyBrick, Jul 30, 2008
    #2
    1. Advertising

  3. jw

    jw Guest

    Thanks for the suggestion RGB, but I know because of the images I'm
    working with there are only about 7-8 possible solid color images.

    Anyway, I first tried moved the declaration of the int array up,
    hoping that would force the system to reuse teh same memory area each
    time through the loop, but that didn't make a difference.

    Then I made two other changes, and I'm not sure which was the actual
    fix. I explicitly called System.gc() every 100th time through the
    loop. Secondly I made a more fundamental change to the algorithm. I
    switched from getting the BufferedImage's int array to repeatedly
    using the BufferedImage's getRGB(x,y) method to get a single pixel of
    info. When I wrote this before, I don't know if that method couldn't
    be used for some reason (different image type for example) or if I
    thought accessing the int array would be faster, or I just didn't know
    about the method. Anyway, I didn't test these separately, so I don't
    know which one made a difference, but I did want to post a follow up
    for anyone interested or for future reference.
    jw, Jul 31, 2008
    #3
  4. jw

    jw Guest

    Forgot to mention I did echo out the size of the Hashmap as suggested
    and it never grew larger than 8 objects, just to be sure.
    jw, Jul 31, 2008
    #4
  5. jw

    Mark Space Guest

    Re: Why is this code not cleaning up int arrays? or How to detectan image is a solid color?

    bugbear wrote:
    > jw wrote:
    >>


    >> thought accessing the int array would be faster, or I just didn't know
    >> about the method. Anyway, I didn't test these separately, so I don't
    >> know which one made a difference.

    >
    > assuming it's not too hard, it would be helpful if you did,
    > to people searching the USENET archive for similar
    > issues.


    Yes, and post an SSCCE of both. This is supposed to be a community
    helping each other, not a tech support line. If you don't post up your
    solutions, how the can the rest of us benefit from your discoveries? It
    puts you in the position of only asking for help and never giving any.

    I'd like to see a SSCCE of both myself, so I can understand better what
    the original problem was....
    Mark Space, Aug 1, 2008
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Schnoffos
    Replies:
    2
    Views:
    1,198
    Martien Verbruggen
    Jun 27, 2003
  2. Hal Styli
    Replies:
    14
    Views:
    1,615
    Old Wolf
    Jan 20, 2004
  3. Mr. SweatyFinger
    Replies:
    2
    Views:
    1,762
    Smokey Grindel
    Dec 2, 2006
  4. Replies:
    9
    Views:
    418
    James Kanze
    Apr 17, 2007
  5. Steve Hicks
    Replies:
    2
    Views:
    1,251
Loading...

Share This Page