when to force an app to consume less resources

[alexandre_paterson]

Hi everyone,

I spent some time optimizing an application that first parses a lot
of text files, which can be in various encodings, then processes
these files (the fact that we're forced to parse lots of text files
isn't
under my control nor my decision, that's the way it is and we have
to do it).

The optimization went great, maybe a little too great, hence the
semi-weird question that is coming after the explanation of what I
did.

I noticed wrapping a BufferedReader around a FileInputStream was
unbearably slow for our needs. So the first optimization step was
to read the text files in byte arrays, then dispatch the actual text
decoding to several threads (wrapping BufferedReader(s) around
ByteArrayInputStream(s)). This is all done using Executors, a
producer/consumer pattern to read the files/etc. We're never I/O
bound (except for the very first file read, as soon as one file is
read,
a worker thread starts decoding it), etc.

Note that I fully understand why it was slow and why it's now one
order of magnitude faster, my question isn't about how to parse
text files faster: I managed to obtain basically 100% cpu usage
(for each core) and to never be I/O bound (and it's the best I can
do, seen the heavy crunching we're doing on these files once
they've been decoded, we'll be CPU bound).

The only "problem" is that my workstation, running Linux
(Debian etch), starts to become kinda unresponsive when
running the optimized version of our parser, while everything
was fine using the old, unoptimized parser (except that the
app was one order of magnitude slower to parse the text
files, of course).

The same, optimized, app on MacOS X acts much nicer :
I can see that both cores (on a Core 2 Duo / Mac Mini) are
being fully used, but I can, say, play videos fine (it was just
a test to see how the app was responding).

I realize it's probably the OS's fault here: if MacOS X keeps
behaving correctly then my Linux system probably should too.

So here comes my question :

Seen that some OSes *shall* start to become laggy when one
application consumes a lot of resources (we've *all* seen that
behavior at least once) and seen that I've got no control over
how the OS runs the app (e.g. I can't ask the user to "nice" the
Linux app to lower its priority), should I artifically limit the
resources used by my app?

For example I noticed that by forcing every worker thread to
sleep for a few milliseconds every 'x' files parsed I could both
obtain a very high troughput and have my workstation stay
responsive.

Stated in another way : once I've optimized my application so
that it's not stupidly unnecessarily bound to some resources, should
I start "freeing some of that resource" to make the OS happy?

In my case it's maxxing CPU usage (and it's the behavior I want), so
should I start "giving the CPU a breath" ?

I could see other cases where you would be hard disk bound once you've
optimized the way your app work, would you then start "giving the hard
disk a breath" ?

Or network usage ?

Has anyone here ever ran into that problem and how do you
deal with that?

Now if I accept the fact that artificially limiting the CPU usage
of our app is a kludgy hack, but still I decide that it's the only
acceptable way to solve our problem (our problem being that
several people's OS becomes too unresponsive when running
our "too optimized" app), what would be the correct way to lower
the CPU usage of our Java app, from our Java program (because
we can't ask the user to re-prioritize our app, nor to force it to run
on only 'x' cores, etc.) ?

Is inserting Thread.sleep(...) the way to go ?

Alex

 

[Andreas Leitgeb]

I noticed wrapping a BufferedReader around a FileInputStream was
unbearably slow for our needs.

[separated reading to byte[] from processing the byte[] ]
This is all done using Executors, a producer/consumer pattern to
read the files/etc.

I wonder, why reading&parsing the file would be so much slower
than first reading it completely to memory and then processing
it from memory.

Perhaps you could have chosen a larger buffer-size by means of
the two-args constructor of the BufferedReader.
I didn't find any specification of the default buffer size
(beyond "many bytes at a time") in the javadocs of BufferedReader,
nor in that of BufferedInputStream.

Have you also tried to do file reading and processing together as
before, but doing that in separate threads for separate files?
(I see no necessitiy to do reading of disk files from only the
main-thread)

Note that I fully understand why it was slow and why it's now one
order of magnitude faster ...

I'm not entirely convinced, yet

The only "problem" is that my workstation, running Linux
(Debian etch), starts to become kinda unresponsive when
running the optimized version of our parser, ...

I realize it's probably the OS's fault here:
(e.g. I can't ask the user to "nice" the Linux app to lower its priority),

You could provide him with a starter-script that does the nice-starting
of java. You could also renice the app once it's running by invoking
renice for the java-process' pid. Didn't see how to obtain that pid
from Java, though, so maybe you'll have to grep the ps output heuristically
in a helper-script before doing the renice.

... to make the OS happy?

nicing processes *is* that way. Maybe, MacOSX just has it's own
heuristics for default nice-levels... like "higher priority for GUI-apps".

I could see other cases where you would be hard disk bound once you've
optimized the way your app work, would you then start "giving the hard
disk a breath" ?

Unless your disks are IDE and still in old "pio"-mode, (rather than
"dma"-mode), that shouldn't be an issue.

(because we can't ask the user to re-prioritize our app

You can do that programmatically from inside,
(Runtime.exec() + find own pid + "/usr/bin/renice")

... nor to force it to run on only 'x' cores, etc.) ?

It wont use more CPU's concurrently as there are threads
running. (keeping in mind also the gc-thread)

 

[Tom Anderson]

The only "problem" is that my workstation, running Linux (Debian etch),
starts to become kinda unresponsive when running the optimized version
of our parser, while everything was fine using the old, unoptimized
parser (except that the app was one order of magnitude slower to parse
the text files, of course).

The same, optimized, app on MacOS X acts much nicer : I can see that
both cores (on a Core 2 Duo / Mac Mini) are being fully used, but I can,
say, play videos fine (it was just a test to see how the app was
responding).

I think this is because Apple's media-playing apps (and possibly some
third-party ones - maybe it happens through the quicktime framework or
something) grab themselves a higher priority, to make sure they can meet
their realtimey guarantees. I guess the window manager does the same.

Seen that some OSes *shall* start to become laggy when one application
consumes a lot of resources (we've *all* seen that behavior at least
once) and seen that I've got no control over how the OS runs the app
(e.g. I can't ask the user to "nice" the Linux app to lower its
priority), should I artifically limit the resources used by my app?

On non-OS X unix, nice/renice really is the way to go here. Put it in a
startup script, run it in a subshell, write a native method that calls
getpid/setpriority, whatever. A launch script would be easiest; if you
can't do that, the native method is fairly easy and nice and clean.

There's probably a comparable API call on windows.

I don't know of a portable way to do it through java, though - that would
be a nice feature.

For example I noticed that by forcing every worker thread to sleep for a
few milliseconds every 'x' files parsed I could both obtain a very high
troughput and have my workstation stay responsive.

Stated in another way : once I've optimized my application so that it's
not stupidly unnecessarily bound to some resources, should I start
"freeing some of that resource" to make the OS happy?

If you're not prepared to use a system-specific mechanism to achieve this,
then yes, i think this is the way to go. Thread.sleep() FTW.

I could see other cases where you would be hard disk bound once you've
optimized the way your app work, would you then start "giving the hard
disk a breath" ?

Or network usage ?

Sleeping works for those too, i guess.

tom

 

[alexandre_paterson]

On Jan 14, 12:47 pm, Andreas Leitgeb <[email protected]>
wrote:
....

Have you also tried to do file reading and processing together as
before, but doing that in separate threads for separate files?
(I see no necessitiy to do reading of disk files from only the
main-thread)

Good point... Actually I haven't tried that for there's now a
unique producer thread doing the I/O and then several consumer
threads doing the in-memory decoding/crunching.

I've got no memory issue and it Just Works [TM] now, so
I probably won't change back.

But point taken, you're right that I could have launched
several worker threads all accessing the hard disk and
using BufferedReader directly.

But that said that wouldn't change anything to my "problem".

It wont use more CPU's concurrently as there are
threads running. (keeping in mind also the gc-thread)

That I know...

What I meant was that I couldn't force the user to enter
a command forcing the app to run on only 'x' cores. If I've
got an application using, say, 12 threads, but the user
forces it to run only on one of his quad-cores, the 4 cores
won't be used (or the OS providing the ability to 'lock'
application to 'x' cores is broken). I was referring, for
example, to both Linux and Windows that provide both that
ability to "restrict" application to a certain number of
cores.


Alex

 

[alexandre_paterson]

Nah rather try setting the thread priority to Low...
This should make your system more responsive.
Thread.currentThread().setPriority(Thread.MIN_PRIORITY);

Done...

Well, for the threads that are maxxing the CPU usage.

Don't know if it really works that said but it feels
snappier

 

[alexandre_paterson]

If you're not prepared to use a system-specific mechanism to achieve this,
then yes, i think this is the way to go. Thread.sleep() FTW.

The problem is: by "nicing" I don't have control on the threads but
on the whole process and I don't want to put the whole JVM at a
disadvantage compared to other apps. I know which threads are
maxxing CPU usage and I'd like only these threads to be "nice"

I used what Christian recommended:

Thread.currentThread().setPriority(Thread.MIN_PRIORITY);

But I may still use a tiny Thread.sleep() FTW !

 

[Tom Anderson]

The problem is: by "nicing" I don't have control on the threads but
on the whole process and I don't want to put the whole JVM at a
disadvantage compared to other apps. I know which threads are
maxxing CPU usage and I'd like only these threads to be "nice"

Nice the whole app, and drop the worker threads' priority a bit below the
other threads? Then at least the time the JVM does get is more likely to
go to the non-worker threads.

I used what Christian recommended:

Thread.currentThread().setPriority(Thread.MIN_PRIORITY);

It would be interesting to measure whether this actually made any
difference. I had the idea that thread priority was about how threads
compete with each other within a JVM, not how the JVM process competes
with other processes. Hence my obsession with renicing. But i could be
wrong. A measurement is the answer.

tom

 

[Andreas Leitgeb]

It would be interesting to measure whether this actually made any
difference.

I'd also be very interested in that result.
Please, alexandre, report back.

 

[alexandre_paterson]

I'd also be very interested in that result.
Please, alexandre, report back.

After reading the Javadocs I think too that setting
Thread.MIN_PRIORITY only affects priority between
threads within the same JVM. The priority of the
main process isn't affected, without much surprise.
That said my Unix administration skills are not strong
enough to detect if this impacted the responsivness
of the system or not.

I still set it and the parser is parsing at the
same speed as before, but besides the EDT there aren't
many threads (typically my application is launched,
user starts a massive "bulk" import, keeps using his
machine, and comes back to my application a few minutes
later once the parsing/crunching has been done).

So as of now I'm living with a few Thread.sleep(...)
in the threads that are otherwise maxxing the CPU
usage as to give the OS a breath. It's not great,
but it's not *that* ugly either and it works.

I realized another solution would be to "nice" the
process (in a non-portable way) to a lower priority
only during the bulk import, then re-nice it to a
normal priority once the bulk import is done, as to
not tax my application regarding the other apps the
user may be running. But this has to run on Windows
and OS X too, so...

Anyway, as usual, thanks for the help and explanations,

Alex

 

[Andreas Leitgeb]

That said my Unix administration skills are not strong
enough to detect if this impacted the responsivness
of the system or not.

I thought watching a video would serve as a test ;-)

So as of now I'm living with a few Thread.sleep(...)
in the threads that are otherwise maxxing the CPU
usage as to give the OS a breath. It's not great,
but it's not *that* ugly either and it works.

Yes, it's just fine (as long as the sum of slept time
doesn't cause the app to take it's pre-optimized time

I realized another solution would be to "nice" the
process (in a non-portable way) to a lower priority
only during the bulk import, then re-nice it to a
normal priority once the bulk import is done

Renicing back to a higher (i.e. even back to 0) priority
would require root-privileges.

As it seems, Thread.sleep() is the ultimate solution,
short of warning the users about possible jerky effects
when watching videos meanwhile.