Memory Leakage drawing Images from Stream - Minimum example

Discussion in 'Java' started by E. Naubauer, Feb 22, 2006.

  1. E. Naubauer

    E. Naubauer Guest

    Ok folks

    I posted this a number of times ago,but never posted a minimum code
    example for people to look at.

    I was trying to load and draw images from a Mjpeg-Stream, reading from
    the stream in a different thread.

    This is the thread where the images are read by:


    public class Axis2420
    {

    private HttpURLConnection hurlconnection = null;
    private DataInputStream datainstream = null;
    private String serialNumber = null;
    private boolean connected = false;
    private Frame dummy = null;
    private Image image = null;
    private String MJpegStreamUrl = null;

    Runnable decodingThread = null;

    public void SetURL(String url)
    {
    MJpegStreamUrl = url;
    }


    public boolean Connect()
    {

    if(MJpegStreamUrl == null)
    {
    System.out.println("Error: Empty Camera URL String");
    return false;
    }

    try
    {

    URL streamurl = new URL(MJpegStreamUrl);
    hurlconnection = (HttpURLConnection)streamurl.openConnection();
    hurlconnection.setReadTimeout(10000);
    //hurlconnection.setUseCaches(false);


    InputStream instream = hurlconnection.getInputStream();

    connected = true;

    BufferedInputStream bis = new BufferedInputStream(instream);
    datainstream = new DataInputStream(bis);



    } catch(IOException e)
    {
    System.out.println("Fehler");

    try
    {
    hurlconnection.disconnect();
    Thread.sleep(60);
    }
    catch(InterruptedException ie)
    {
    hurlconnection.disconnect();
    Connect();
    }
    catch(Exception ex)
    {return false; }


    System.out.println(e.getMessage());

    return false;
    }

    return true;
    }


    //run method of the reading thread
    public void run()
    {
    //test if still connected
    if(!connected)
    return;


    try
    {
    //decode stream
    if(connected)
    {
    Image i = null;
    //ImageIO.setUseCache(false);
    long before,after;



    while(!Thread.currentThread().isInterrupted())
    {

    i = null;

    String header;
    int k = 0;

    do
    {
    header = datainstream.readLine();
    //System.out.println("Zeile: " +header);
    }
    while(++k < 3);


    int j = (new Integer(header.substring("Content-Length:
    ".length()))).intValue();

    datainstream.readByte();
    datainstream.readByte();
    // datainstream.readLine();
    imageJPG = new byte[j];
    datainstream.readFully(imageJPG, 0, j);
    byte tail[] = new byte[4];
    datainstream.readFully(tail,0,4);


    i =
    Toolkit.getDefaultToolkit().createImage(imageJPG,0,imageJPG.length);

    image = i;
    i = null;

    fireChange(); //calls the stateChanged() method in
    the class 'MainProgram' below

    image = null;
    i = null;

    Thread.currentThread().sleep(10);
    }
    }
    }
    catch(Exception ex)
    {
    ex.printStackTrace();
    }
    finally
    {

    try {
    datainstream.close();
    } catch(Exception ex)
    {
    ex.printStackTrace();
    }
    }
    }

    //for the listening objects so they can fetch the latest image
    public Image getLatestImage()
    {
    return image;
    }

    <....>

    }



    Once a new image has arrived, the fireChange() method calls this method
    on the MainProgram class:




    class MainProgram
    extends JFrame
    implements Runnable,ChangeListener
    {
    public static String WINDOW_TITLE = "BINO GESTENERKENNUNG";
    public static int WINDOW_WIDTH = 640;
    public static int WINDOW_HEIGHT = 480;

    public static final int CAMERA_LEFT = 0;
    public static final int CAMERA_RIGHT = 1;

    //public static final String leftCameraURL =
    "http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=704x576&compression=25&clock=0&date=1&fps=10";


    //public static final String rightCameraURL =
    "http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=704x576&compression=25&clock=0&date=1&fps=10";

    public static final String leftCameraURL =
    "http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=70&date=1&fps=5";


    public static final String rightCameraURL =
    "http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=100&date=1&fps=5";


    Image currentLeftImage;
    Image currentRightImage;

    Graphics leftImageGraphics;
    Graphics rightImageGraphics;
    Graphics leftCameraGraphics;
    Graphics rightCameraGraphics;
    byte[] currentLeftImageJPG;
    byte[] currentRightImageJPG;

    BufferedImage testImage;

    Axis2420 leftCamera = null;
    Axis2420 rightCamera = null;

    CameraCanvas leftCameraCanvas = null;
    CameraCanvas rightCameraCanvas = null;

    Runnable drawThread = null;
    boolean readyToDraw = false;
    boolean newImage = false;

    long frameRateLeft = 0,frameRateRight = 0;
    long frameTimesLeft[],frameTimesRight[];

    //vPointerDetector leftImagePointer;
    //vPointerDetector rightImagePointer;
    vPointerDetector imagePointer;

    boolean updateLeftImage,updateRightImage;




    <.....lots of meaningless stuff.......>



    public void stateChanged(ChangeEvent e)
    {







    currentRightImage = rightCamera.getLatestImage();


    rightCameraCanvas.SetUpdate(true); //otherwise paintComponent()
    of the JCanvas won't draw the image
    rightCameraCanvas.SetImage(currentRightImage);
    rightCameraCanvas.repaint();
    currentRightImage = null;
    currentRightImageJPG = null;
    }
    }



    }



    } //end of MainProgram class



    Basically, all the stateChange method does is getting the image from the
    image-reading thread ('rightCamera' of type Axis2420) after
    rightCamera called fireChange(). The image is passed to an object of the
    class CameraCanvas which derieves from JCanvas. After that, we cast
    repaint() on the CameraCanvas (rightCameraCanvas.repaint() ).



    This is the CameraCanvas that is responsible for drawing:



    public class CameraCanvas extends JPanel
    {

    private Image image = null;
    boolean drawNext = false;
    boolean readyForNext;


    public void paintComponent(Graphics g)
    {
    if(drawNext == true)
    {

    readyForNext =
    g.drawImage(image,0,0,getSize().width,getSize().height,this);
    //image.flush();
    image = null;
    //g.dispose();

    //imageBuffered.flush();
    drawNext = false;
    }

    }

    //indicates that there is a new image
    public void SetUpdate(boolean up)
    {
    drawNext = up;
    }

    //passes a new image to the Canvas
    public void SetImage(Image i)
    {
    image = i;
    }
    }






    The program draws the stream correctly and is smooth like butter.
    However, there is a HUGE memory leak somewhere that raises the memory
    usage by about 40MBytes per second. If I remove the call to

    readyForNext = g.drawImage(image,0,0,getSize().width,getSize().height,this);

    in the CameraCanvas thread, the leak will go down to about 40 kbyte /
    second. If I leave it in and post a System.gc() call afterwards, the
    leak will also go down.


    I think that I'm somewhat polling the AWTEvent thread which lots of
    copies of images waiting to be drawn. Also using
    Toolkit.getDefaultToolkit().createImage(imageJPG,0,imageJPG.length);

    spawns abouth 4 ImageFetcher threads which I think are the reason for
    the small memory leak.


    Now the question: What can I do about it?

    It seems that the garbage collection isn't called after drawing an image
    and casting it manually via System.gc() slows everything down a lot.
    Using ImageIO.read)() instead of the Toolkit seems to remove the small
    leak, but not the big one for drawing. And it tends to be slower since
    it's blocking and keeping the Axis2420 thread from reading from the stream.


    My thesis is due soon, so pleease have a look at it again.!!
    Thanks in advance!!!
    E. Naubauer, Feb 22, 2006
    #1
    1. Advertising

  2. > I posted this a number of times ago,but never posted a minimum code
    > example for people to look at.
    >
    > I was trying to load and draw images from a Mjpeg-Stream, reading from
    > the stream in a different thread.


    I would use only _one_ image.
    set MemoryImageSource animated.
    Read every image to MemoryImageSource's data array and call newpixels();

    --
    Andrey Kuznetsov
    http://uio.imagero.com Unified I/O for Java
    http://reader.imagero.com Java image reader
    http://jgui.imagero.com Java GUI components and utilities
    Andrey Kuznetsov, Feb 22, 2006
    #2
    1. Advertising

  3. E. Naubauer

    E. Naubauer Guest

    But MemoryImageSource only works with RGB pixel data and not JFIF data,
    doesn't it? It would require me to convert it to pixel data first via
    BufferedImage via ImageIO and thats what I'm trying to avoid. By the
    way: I tried using another thread which does the decoding instead of the
    Axis2420 thread with ImageIO and it leaked as well. It seems like
    everything depends on the Axis2420 thread not bringing in new images too
    fast. If for example I run the program with JProfiler (which drops
    performance due to the whole analyzing stuff) the memory-leak doesn't
    appear. It is slow, however.
    E. Naubauer, Feb 22, 2006
    #3
  4. E. Naubauer

    E. Naubauer Guest

    Ok, it seems that I have a semi-stable version running now with what
    Andrey mentioned and frames dropping. Thanks for your help!
    E. Naubauer, Feb 23, 2006
    #4
  5. E. Naubauer

    Roedy Green Guest

    On Wed, 22 Feb 2006 17:56:46 +0100, "E. Naubauer"
    <> wrote, quoted or indirectly quoted someone who
    said :

    >But MemoryImageSource only works with RGB pixel data and not JFIF data,
    >doesn't it?


    is there document that gives say a 5 page overview of what all the
    Java classes for image processing are FOR perhaps with some recipes
    for how you do the usual things?

    It is all a blur. How do you know which classes to use for what sorts
    of problem? How do you know when you glue them together there is not
    some much cleaner way?



    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Feb 23, 2006
    #5
  6. E. Naubauer

    E. Naubauer Guest

    Roedy Green schrieb:
    > On Wed, 22 Feb 2006 17:56:46 +0100, "E. Naubauer"
    > <> wrote, quoted or indirectly quoted someone who
    > said :
    >
    >> But MemoryImageSource only works with RGB pixel data and not JFIF data,
    >> doesn't it?

    >
    > is there document that gives say a 5 page overview of what all the
    > Java classes for image processing are FOR perhaps with some recipes
    > for how you do the usual things?
    >
    > It is all a blur. How do you know which classes to use for what sorts
    > of problem? How do you know when you glue them together there is not
    > some much cleaner way?
    >
    >
    >



    Do you mean something like this?

    http://java.sun.com/developer/technicalArticles/Media/imagestrategies/index.html

    Although you might already know it.



    Ok, let me do a short summary about my experiences of the last weeks
    regarding reading images from an (Axis) network camera:

    Images arrive at a framerate of ~25 frames per second, meaning every
    frame has to be processed within about 40 ms to process them all.
    However, the fashion in which Java deals with those is quite different
    depending on the classes used under the hood, the threading strategy etc. :

    - the best method if it comes only to drawing the images is decoding the
    image with Toolkit.createImage and draw it using a double-buffer
    strategy. The thread, that fetches and decodes the images (and casts
    repaint on the drawing component) should run with the lowest priority,
    or you might cause a memory leak since the images don't get garbage
    collected correctly.

    - ImageIO loads images synchronously and causes less memory problems,
    but keeps the fetching/decoding thread busy which might drop the
    framerate critically.

    - images drawn with drawImage don't seem to be garbage collected
    correctly if repaint() is called too often.
    E. Naubauer, Feb 23, 2006
    #6
  7. E. Naubauer

    Roedy Green Guest

    On Thu, 23 Feb 2006 21:31:36 +0100, "E. Naubauer"
    <> wrote, quoted or indirectly quoted someone who
    said :

    >http://java.sun.com/developer/technicalArticles/Media/imagestrategies/index.html


    that's great. I used that article to construct a table I call "let me
    count the ways" which enumerates all the ways you can create various
    Image related objects.

    See http://mindprod.com/jgloss/image.html#CREATING
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Feb 24, 2006
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gaël

    w3wp crash and memory leakage

    Gaël, Oct 15, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    2,676
    Alvin Bruney
    Oct 16, 2003
  2. John Hilton

    How to identify the memory leakage...

    John Hilton, Dec 16, 2004, in forum: ASP .Net
    Replies:
    0
    Views:
    310
    John Hilton
    Dec 16, 2004
  3. Replies:
    1
    Views:
    633
    Kevin Spencer
    Jan 9, 2006
  4. Sambucus
    Replies:
    19
    Views:
    882
    George Neuner
    May 6, 2004
  5. Devian
    Replies:
    7
    Views:
    377
    Babu Kalakrishnan
    Sep 20, 2004
Loading...

Share This Page