streaming problem and thread freeze

[Fly]

Hello,
I have a client program that read an XML from a STREAM every 60 seconds.
From the main Thread I do an infinite loop where I create a new working thread
executing all the operations: read, parse, etc..

This is what I do:

<code>
DownloaderTask tempTask = null;

while (true) {

if (tempTask != null) {
if (tempTask.isAlive()) {
tempTask.cancel();
}
}

tempTask = new DownloaderTask();

tempTask.start();

try {
sleep(frequency * 1000);
} catch (InterruptedException ex) {
log.severe("Main sleep failed");
}

}

</code>

Inside the Run method of the DownloaderTask, I first read from the Stream (a class field) with a BufferedReader, in this way:

<code>
private String readDocumentFromStream(BufferedReader reader) throws IOException {
char[] buffer = new char[4 * 1024];
int charsRead = -1;
String retVal = "";
log.finer("---START reading XML ---");

while ((charsRead = reader.read(buffer, 0, 4 * 1024)) != -1)
{
log.log(Level.FINER, "CharsRead: {0}", charsRead);
retVal += String.copyValueOf(buffer, 0, charsRead);
}
log.finer("---STOP reading XML---");

return retVal;
}
</code>

In the DownloaderTask class I wrote a cancel() method, called by the main thread in the case the working thread was still alive after the sleep().


It's definition:

<code>
synchronized public void cancel() {

this.interrupt();
try {
if (rd != null) {
rd.close();
}
} catch (IOException ex) {
log.severe(ex.getMessage());
}

http_conn.disconnect();

log.log(Level.SEVERE, "Timeout. Thread {0} is being canceled and resource released.", this.getId());

}
</code>

My problems.

Sometimes I see my program active, doing nothing, blocked probably in the read(), because if I check all logs, the last line written is "CharsRead: xxx" and no more.
Here I post one of my saved log (where the read buffer was 1024 bytes):

<LOG>
||-- 2011-05-03 06:44:59 FINE: Updating events data... --||
||-- 2011-05-03 06:44:59 FINE: Events updated! --||
||-- 2011-05-03 06:44:59 INFO: Release date: 2011-05-03 06:45:15 | Received: 187 Inserted: 0 Updated: 0 Deleted: 0 ||| Errors: 2 --||
||-- 2011-05-03 06:44:59 FINER: Normal closing thread 1.723 --||
||-- 2011-05-03 06:44:59 FINER: Thread 1.723 Closing --||
||-- 2011-05-03 06:45:52 FINER: Thread 1.725 Opening --||
||-- 2011-05-03 06:45:52 FINE: Opening server connection to: mysite.com:80 --||
||-- 2011-05-03 06:46:19 FINE: Reading From Stream --||
||-- 2011-05-03 06:46:19 FINER: ---START reading XML --- --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 929 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 872 --||
</LOG>

I noticed another strange thing in this log, that is I can't see the Thread Id: 1724.
Can you help me to find where I am wrong? In the logic or maybe in the syntax of the program?

Thanks in advance,
Flavio

 

[Daniele Futtorovic]

Hello,
I have a client program that read an XML from a STREAM every 60 seconds.
From the main Thread I do an infinite loop where I create a new working thread
executing all the operations: read, parse, etc..

This is what I do:

<code>
DownloaderTask tempTask = null;

while (true) {

if (tempTask != null) {
if (tempTask.isAlive()) {
tempTask.cancel();
}
}

tempTask = new DownloaderTask();

tempTask.start();

try {
sleep(frequency * 1000);
} catch (InterruptedException ex) {
log.severe("Main sleep failed");
}

}

</code>

Inside the Run method of the DownloaderTask, I first read from the Stream (a class field) with a BufferedReader, in this way:

<code>
private String readDocumentFromStream(BufferedReader reader) throws IOException {
char[] buffer = new char[4 * 1024];
int charsRead = -1;
String retVal = "";
log.finer("---START reading XML ---");

while ((charsRead = reader.read(buffer, 0, 4 * 1024)) != -1)
{
log.log(Level.FINER, "CharsRead: {0}", charsRead);
retVal += String.copyValueOf(buffer, 0, charsRead);
}
log.finer("---STOP reading XML---");

return retVal;
}
</code>

In the DownloaderTask class I wrote a cancel() method, called by the main thread in the case the working thread was still alive after the sleep().


It's definition:

<code>
synchronized public void cancel() {

this.interrupt();
try {
if (rd != null) {
rd.close();
}
} catch (IOException ex) {
log.severe(ex.getMessage());
}

http_conn.disconnect();

log.log(Level.SEVERE, "Timeout. Thread {0} is being canceled and resource released.", this.getId());

}
</code>

My problems.

Sometimes I see my program active, doing nothing, blocked probably in the read(), because if I check all logs, the last line written is "CharsRead: xxx" and no more.
Here I post one of my saved log (where the read buffer was 1024 bytes):

<LOG>
||-- 2011-05-03 06:44:59 FINE: Updating events data... --||
||-- 2011-05-03 06:44:59 FINE: Events updated! --||
||-- 2011-05-03 06:44:59 INFO: Release date: 2011-05-03 06:45:15 | Received: 187 Inserted: 0 Updated: 0 Deleted: 0 ||| Errors: 2 --||
||-- 2011-05-03 06:44:59 FINER: Normal closing thread 1.723 --||
||-- 2011-05-03 06:44:59 FINER: Thread 1.723 Closing --||
||-- 2011-05-03 06:45:52 FINER: Thread 1.725 Opening --||
||-- 2011-05-03 06:45:52 FINE: Opening server connection to: mysite.com:80 --||
||-- 2011-05-03 06:46:19 FINE: Reading From Stream --||
||-- 2011-05-03 06:46:19 FINER: ---START reading XML --- --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 929 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 1.024 --||
||-- 2011-05-03 06:46:19 FINER: CharsRead: 872 --||
</LOG>

I noticed another strange thing in this log, that is I can't see the Thread Id: 1724.
Can you help me to find where I am wrong? In the logic or maybe in the syntax of the program?

Thanks in advance,
Flavio

A few things:

Firstly, dunno if this will help the problem, but your main loop ought
to look something like this:

DownloaderTask tempTask = null;
for( Thread t = Thread.currentThread(); ! t.isInterrupted(); ){
if (tempTask != null) {
if (tempTask.isAlive()) {
tempTask.cancel();
}
}

tempTask = new DownloaderTask();
tempTask.start();

try {
TimeUnit.SECONDS.sleep( frequency ); //
}
catch (InterruptedException ex) {
*t.interrupt();*
}
}

Always re-interrupt when you've caught an InterruptedException. The
reason is that catching that exception *clears* the interrupted status.

You possibly might as well try a join() on the worker thread, BTW.

And have a look at java.util.concurrent.Executors.

Secondly, although it is only a minor thing and only my personal
opinion, this:

} catch (IOException ex) {
log.severe(ex.getMessage());
}

is a *very* bad idea. Log the full stack. The message is next to useless
as soon as the code becomes a little complex.

HTH,

 

[Lawrence D'Oliveiro]

Secondly, although it is only a minor thing and only my personal
opinion, this:

} catch (IOException ex) {
log.severe(ex.getMessage());
}

is a *very* bad idea. Log the full stack.

It would be so much easier if you could simply not bother to catch the
exception at all, and let your program crash with a full stack trace.

 

[markspace]

It would be so much easier if you could simply not bother to catch the
exception at all, and let your program crash with a full stack trace.

Partially correct. If it's a small example program, or if it's only
going to be used by developers or other folks who could make use of a
stack trace, then sure, just throwing the error often works.

Many times however that does not work, and then you have to catch the
error, log it, and then possibly rethrow with a different exception that
some higher level code is able to catch. Or you can skip logging it
here, if you re-throw and you know the higher level code will log it.
It takes about 20% art and about 80% prior-planning, but it's not hard
to work out ways to handle errors and logging in any give subsystem.

For end users, the best way imo is to have main catch all errors, then
log and attempt to upload it to a central site. This avoids having to
ask the users to dig through logs, up load them, or interpret them.
It's just easier on everybody, yourself included.

 

[Lew]

It would be so much easier if you could simply not bother to catch the
exception at all, and let your program crash with a full stack trace.

It would be so much easier if programs didn't have to produce correct results
at all, or be friendly to the user at all, because then we all could write ten
times as much code in half the time.

Damn that requirement for correct, useful action!

Where do you come up with this stuff, Larry-boy? Is it a case of, "Let me say
the thing that is so off-the-wall and contrary to any good sense or logic that
it's absolutely guaran-damn-TEED to incite a reaction"? Because that's how it
looks, Larry-boy.

 

[Daniele Futtorovic]

It would be so much easier if you could simply not bother to catch the
exception at all, and let your program crash with a full stack trace.

It would be yet much easier, as you perhaps one day will learn, to
realise that checked exceptions are not a bother, but an almost
invaluable tool.

And in cases you have to do something dirty, you can still simply throw
an Error.

 

[Lew]

It would be yet much easier, as you perhaps one day will learn, to realise
that checked exceptions are not a bother, but an almost invaluable tool.

And in cases you have to do something dirty, you can still simply throw an Error.

You raise a good, if classically controversial, point, the overarching
question of how properly to use exceptions. I, too, am partisan to the
checked exception from two perspectives - that of the maintainer, and that of
the API designer. You don't have to use them, given the rich hierarchy of
Throwables, but they sure can be useful.

Complaints about checked exceptions come pretty much exclusively from
programmers using APIs that require'em. Hmm. I recommend to such folks that
they consider a question, "What purpose does this particular checked exception
serve in the method contract?"

Whether you agree or not, the API designer had an answer to that question.
They have ensured that all clients benefit from their reasoning.

API designers that make different decisions might use runtime exceptions or
some other mechanism.

It's not really a bad thing nor even confusing to a reasonably-trained
programmer to have 'catch' blocks all over the place, no more than the similar
amount of decision blocks one needs for thorough, professional programming.
Code that serves a purpose isn't bloat, and code that creates a reliable
invariant isn't a digression.

If you can't stand the heat, get out of the kitchen.

 

[Lawrence D'Oliveiro]

It would be yet much easier, as you perhaps one day will learn, to
realise that checked exceptions are not a bother, but an almost
invaluable tool.

So how does that help in this case?

 

[Lew]

So how does that help in this case?

It prevents the program from crashing like some first-year student's first
project and making you look like an idiot. Just to name one.

 

[Daniele Futtorovic]

You raise a good, if classically controversial, point, the overarching
question of how properly to use exceptions. I, too, am partisan to the
checked exception from two perspectives - that of the maintainer, and
that of the API designer. You don't have to use them, given the rich
hierarchy of Throwables, but they sure can be useful.

Complaints about checked exceptions come pretty much exclusively from
programmers using APIs that require'em. Hmm. I recommend to such folks
that they consider a question, "What purpose does this particular
checked exception serve in the method contract?"

Whether you agree or not, the API designer had an answer to that
question. They have ensured that all clients benefit from their reasoning.

API designers that make different decisions might use runtime exceptions
or some other mechanism.

Corrollarily, while it is indeed the task of API design that puts the
problem of which exceptions to throw when and how right in one's face, I
would argue that it is nevertheless primarily API /usage/ that teaches
one the virtue of a well-thought-out exception design -- perhaps through
the bother of not-so-well-thought-out ones.

It is healthy to question the usage of Exceptions made by an API every
time one uses a new one. Question it, ponder whether the way things are
done actually help you, or bother you, and how they could be improved.
BUT, give the API writers the benefit of the doubt. Chances are they had
a bad reason for doing what they did. Chances are they had a good
reason. But they almost certainly /had/ a reason. Find that reason, and
find out what it's worth. Thus thy craft will blossom.