E
E. Naubauer
Ok folks
I posted this a number of times ago,but never posted a minimum code
example for people to look at.
I was trying to load and draw images from a Mjpeg-Stream, reading from
the stream in a different thread.
This is the thread where the images are read by:
public class Axis2420
{
private HttpURLConnection hurlconnection = null;
private DataInputStream datainstream = null;
private String serialNumber = null;
private boolean connected = false;
private Frame dummy = null;
private Image image = null;
private String MJpegStreamUrl = null;
Runnable decodingThread = null;
public void SetURL(String url)
{
MJpegStreamUrl = url;
}
public boolean Connect()
{
if(MJpegStreamUrl == null)
{
System.out.println("Error: Empty Camera URL String");
return false;
}
try
{
URL streamurl = new URL(MJpegStreamUrl);
hurlconnection = (HttpURLConnection)streamurl.openConnection();
hurlconnection.setReadTimeout(10000);
//hurlconnection.setUseCaches(false);
InputStream instream = hurlconnection.getInputStream();
connected = true;
BufferedInputStream bis = new BufferedInputStream(instream);
datainstream = new DataInputStream(bis);
} catch(IOException e)
{
System.out.println("Fehler");
try
{
hurlconnection.disconnect();
Thread.sleep(60);
}
catch(InterruptedException ie)
{
hurlconnection.disconnect();
Connect();
}
catch(Exception ex)
{return false; }
System.out.println(e.getMessage());
return false;
}
return true;
}
//run method of the reading thread
public void run()
{
//test if still connected
if(!connected)
return;
try
{
//decode stream
if(connected)
{
Image i = null;
//ImageIO.setUseCache(false);
long before,after;
while(!Thread.currentThread().isInterrupted())
{
i = null;
String header;
int k = 0;
do
{
header = datainstream.readLine();
//System.out.println("Zeile: " +header);
}
while(++k < 3);
int j = (new Integer(header.substring("Content-Length:
".length()))).intValue();
datainstream.readByte();
datainstream.readByte();
// datainstream.readLine();
imageJPG = new byte[j];
datainstream.readFully(imageJPG, 0, j);
byte tail[] = new byte[4];
datainstream.readFully(tail,0,4);
i =
Toolkit.getDefaultToolkit().createImage(imageJPG,0,imageJPG.length);
image = i;
i = null;
fireChange(); //calls the stateChanged() method in
the class 'MainProgram' below
image = null;
i = null;
Thread.currentThread().sleep(10);
}
}
}
catch(Exception ex)
{
ex.printStackTrace();
}
finally
{
try {
datainstream.close();
} catch(Exception ex)
{
ex.printStackTrace();
}
}
}
//for the listening objects so they can fetch the latest image
public Image getLatestImage()
{
return image;
}
<....>
}
Once a new image has arrived, the fireChange() method calls this method
on the MainProgram class:
class MainProgram
extends JFrame
implements Runnable,ChangeListener
{
public static String WINDOW_TITLE = "BINO GESTENERKENNUNG";
public static int WINDOW_WIDTH = 640;
public static int WINDOW_HEIGHT = 480;
public static final int CAMERA_LEFT = 0;
public static final int CAMERA_RIGHT = 1;
//public static final String leftCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=704x576&compression=25&clock=0&date=1&fps=10";
//public static final String rightCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=704x576&compression=25&clock=0&date=1&fps=10";
public static final String leftCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=70&date=1&fps=5";
public static final String rightCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=100&date=1&fps=5";
Image currentLeftImage;
Image currentRightImage;
Graphics leftImageGraphics;
Graphics rightImageGraphics;
Graphics leftCameraGraphics;
Graphics rightCameraGraphics;
byte[] currentLeftImageJPG;
byte[] currentRightImageJPG;
BufferedImage testImage;
Axis2420 leftCamera = null;
Axis2420 rightCamera = null;
CameraCanvas leftCameraCanvas = null;
CameraCanvas rightCameraCanvas = null;
Runnable drawThread = null;
boolean readyToDraw = false;
boolean newImage = false;
long frameRateLeft = 0,frameRateRight = 0;
long frameTimesLeft[],frameTimesRight[];
//vPointerDetector leftImagePointer;
//vPointerDetector rightImagePointer;
vPointerDetector imagePointer;
boolean updateLeftImage,updateRightImage;
<.....lots of meaningless stuff.......>
public void stateChanged(ChangeEvent e)
{
currentRightImage = rightCamera.getLatestImage();
rightCameraCanvas.SetUpdate(true); //otherwise paintComponent()
of the JCanvas won't draw the image
rightCameraCanvas.SetImage(currentRightImage);
rightCameraCanvas.repaint();
currentRightImage = null;
currentRightImageJPG = null;
}
}
}
} //end of MainProgram class
Basically, all the stateChange method does is getting the image from the
image-reading thread ('rightCamera' of type Axis2420) after
rightCamera called fireChange(). The image is passed to an object of the
class CameraCanvas which derieves from JCanvas. After that, we cast
repaint() on the CameraCanvas (rightCameraCanvas.repaint() ).
This is the CameraCanvas that is responsible for drawing:
public class CameraCanvas extends JPanel
{
private Image image = null;
boolean drawNext = false;
boolean readyForNext;
public void paintComponent(Graphics g)
{
if(drawNext == true)
{
readyForNext =
g.drawImage(image,0,0,getSize().width,getSize().height,this);
//image.flush();
image = null;
//g.dispose();
//imageBuffered.flush();
drawNext = false;
}
}
//indicates that there is a new image
public void SetUpdate(boolean up)
{
drawNext = up;
}
//passes a new image to the Canvas
public void SetImage(Image i)
{
image = i;
}
}
The program draws the stream correctly and is smooth like butter.
However, there is a HUGE memory leak somewhere that raises the memory
usage by about 40MBytes per second. If I remove the call to
readyForNext = g.drawImage(image,0,0,getSize().width,getSize().height,this);
in the CameraCanvas thread, the leak will go down to about 40 kbyte /
second. If I leave it in and post a System.gc() call afterwards, the
leak will also go down.
I think that I'm somewhat polling the AWTEvent thread which lots of
copies of images waiting to be drawn. Also using
Toolkit.getDefaultToolkit().createImage(imageJPG,0,imageJPG.length);
spawns abouth 4 ImageFetcher threads which I think are the reason for
the small memory leak.
Now the question: What can I do about it?
It seems that the garbage collection isn't called after drawing an image
and casting it manually via System.gc() slows everything down a lot.
Using ImageIO.read)() instead of the Toolkit seems to remove the small
leak, but not the big one for drawing. And it tends to be slower since
it's blocking and keeping the Axis2420 thread from reading from the stream.
My thesis is due soon, so pleease have a look at it again.!!
Thanks in advance!!!
I posted this a number of times ago,but never posted a minimum code
example for people to look at.
I was trying to load and draw images from a Mjpeg-Stream, reading from
the stream in a different thread.
This is the thread where the images are read by:
public class Axis2420
{
private HttpURLConnection hurlconnection = null;
private DataInputStream datainstream = null;
private String serialNumber = null;
private boolean connected = false;
private Frame dummy = null;
private Image image = null;
private String MJpegStreamUrl = null;
Runnable decodingThread = null;
public void SetURL(String url)
{
MJpegStreamUrl = url;
}
public boolean Connect()
{
if(MJpegStreamUrl == null)
{
System.out.println("Error: Empty Camera URL String");
return false;
}
try
{
URL streamurl = new URL(MJpegStreamUrl);
hurlconnection = (HttpURLConnection)streamurl.openConnection();
hurlconnection.setReadTimeout(10000);
//hurlconnection.setUseCaches(false);
InputStream instream = hurlconnection.getInputStream();
connected = true;
BufferedInputStream bis = new BufferedInputStream(instream);
datainstream = new DataInputStream(bis);
} catch(IOException e)
{
System.out.println("Fehler");
try
{
hurlconnection.disconnect();
Thread.sleep(60);
}
catch(InterruptedException ie)
{
hurlconnection.disconnect();
Connect();
}
catch(Exception ex)
{return false; }
System.out.println(e.getMessage());
return false;
}
return true;
}
//run method of the reading thread
public void run()
{
//test if still connected
if(!connected)
return;
try
{
//decode stream
if(connected)
{
Image i = null;
//ImageIO.setUseCache(false);
long before,after;
while(!Thread.currentThread().isInterrupted())
{
i = null;
String header;
int k = 0;
do
{
header = datainstream.readLine();
//System.out.println("Zeile: " +header);
}
while(++k < 3);
int j = (new Integer(header.substring("Content-Length:
".length()))).intValue();
datainstream.readByte();
datainstream.readByte();
// datainstream.readLine();
imageJPG = new byte[j];
datainstream.readFully(imageJPG, 0, j);
byte tail[] = new byte[4];
datainstream.readFully(tail,0,4);
i =
Toolkit.getDefaultToolkit().createImage(imageJPG,0,imageJPG.length);
image = i;
i = null;
fireChange(); //calls the stateChanged() method in
the class 'MainProgram' below
image = null;
i = null;
Thread.currentThread().sleep(10);
}
}
}
catch(Exception ex)
{
ex.printStackTrace();
}
finally
{
try {
datainstream.close();
} catch(Exception ex)
{
ex.printStackTrace();
}
}
}
//for the listening objects so they can fetch the latest image
public Image getLatestImage()
{
return image;
}
<....>
}
Once a new image has arrived, the fireChange() method calls this method
on the MainProgram class:
class MainProgram
extends JFrame
implements Runnable,ChangeListener
{
public static String WINDOW_TITLE = "BINO GESTENERKENNUNG";
public static int WINDOW_WIDTH = 640;
public static int WINDOW_HEIGHT = 480;
public static final int CAMERA_LEFT = 0;
public static final int CAMERA_RIGHT = 1;
//public static final String leftCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=704x576&compression=25&clock=0&date=1&fps=10";
//public static final String rightCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=704x576&compression=25&clock=0&date=1&fps=10";
public static final String leftCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=70&date=1&fps=5";
public static final String rightCameraURL =
"http://192.168.1.51/axis-cgi/mjpg/video.cgi?resolution=352x288&compression=100&date=1&fps=5";
Image currentLeftImage;
Image currentRightImage;
Graphics leftImageGraphics;
Graphics rightImageGraphics;
Graphics leftCameraGraphics;
Graphics rightCameraGraphics;
byte[] currentLeftImageJPG;
byte[] currentRightImageJPG;
BufferedImage testImage;
Axis2420 leftCamera = null;
Axis2420 rightCamera = null;
CameraCanvas leftCameraCanvas = null;
CameraCanvas rightCameraCanvas = null;
Runnable drawThread = null;
boolean readyToDraw = false;
boolean newImage = false;
long frameRateLeft = 0,frameRateRight = 0;
long frameTimesLeft[],frameTimesRight[];
//vPointerDetector leftImagePointer;
//vPointerDetector rightImagePointer;
vPointerDetector imagePointer;
boolean updateLeftImage,updateRightImage;
<.....lots of meaningless stuff.......>
public void stateChanged(ChangeEvent e)
{
currentRightImage = rightCamera.getLatestImage();
rightCameraCanvas.SetUpdate(true); //otherwise paintComponent()
of the JCanvas won't draw the image
rightCameraCanvas.SetImage(currentRightImage);
rightCameraCanvas.repaint();
currentRightImage = null;
currentRightImageJPG = null;
}
}
}
} //end of MainProgram class
Basically, all the stateChange method does is getting the image from the
image-reading thread ('rightCamera' of type Axis2420) after
rightCamera called fireChange(). The image is passed to an object of the
class CameraCanvas which derieves from JCanvas. After that, we cast
repaint() on the CameraCanvas (rightCameraCanvas.repaint() ).
This is the CameraCanvas that is responsible for drawing:
public class CameraCanvas extends JPanel
{
private Image image = null;
boolean drawNext = false;
boolean readyForNext;
public void paintComponent(Graphics g)
{
if(drawNext == true)
{
readyForNext =
g.drawImage(image,0,0,getSize().width,getSize().height,this);
//image.flush();
image = null;
//g.dispose();
//imageBuffered.flush();
drawNext = false;
}
}
//indicates that there is a new image
public void SetUpdate(boolean up)
{
drawNext = up;
}
//passes a new image to the Canvas
public void SetImage(Image i)
{
image = i;
}
}
The program draws the stream correctly and is smooth like butter.
However, there is a HUGE memory leak somewhere that raises the memory
usage by about 40MBytes per second. If I remove the call to
readyForNext = g.drawImage(image,0,0,getSize().width,getSize().height,this);
in the CameraCanvas thread, the leak will go down to about 40 kbyte /
second. If I leave it in and post a System.gc() call afterwards, the
leak will also go down.
I think that I'm somewhat polling the AWTEvent thread which lots of
copies of images waiting to be drawn. Also using
Toolkit.getDefaultToolkit().createImage(imageJPG,0,imageJPG.length);
spawns abouth 4 ImageFetcher threads which I think are the reason for
the small memory leak.
Now the question: What can I do about it?
It seems that the garbage collection isn't called after drawing an image
and casting it manually via System.gc() slows everything down a lot.
Using ImageIO.read)() instead of the Toolkit seems to remove the small
leak, but not the big one for drawing. And it tends to be slower since
it's blocking and keeping the Axis2420 thread from reading from the stream.
My thesis is due soon, so pleease have a look at it again.!!
Thanks in advance!!!