control CPU usage

K

kakarukeys

Hi,

When I am running a loop for a long time, calculating heavily, the CPU
usage
is at 100%, making the comp not so responsive. Is there a way to
control the
CPU usage at say 80%? putting a time.sleep(0.x) doesn't seem to help
although CPU usage level is reduced, but it's unstable.

Regards,
W.J.F.
 
S

Sean DiZazzo

Hi,

When I am running a loop for a long time, calculating heavily, the CPU
usage
is at 100%, making the comp not so responsive. Is there a way to
control the
CPU usage at say 80%? putting a time.sleep(0.x) doesn't seem to help
although CPU usage level is reduced, but it's unstable.

Regards,
W.J.F.

If you are on linux, you can use the 'nice' command. It will still
take 100%, but will give it up to other processes that need to use the
cpu.

nice -n 19 <script>

In windows, I guess you could change the priority of the process in
the Task Manager. Not sure how to do it programatically.

~Sean
 
D

Dennis Lee Bieber

Hi,

When I am running a loop for a long time, calculating heavily, the CPU
usage
is at 100%, making the comp not so responsive. Is there a way to
control the
CPU usage at say 80%? putting a time.sleep(0.x) doesn't seem to help
although CPU usage level is reduced, but it's unstable.
Reduce the process priority (see your OS documentation on how this
is done) so other things can respond sooner. Should only cause a
slowdown in the number crunching if other things are taking place on the
computer.
 
D

Dave Angel

kakarukeys said:
Hi,

When I am running a loop for a long time, calculating heavily, the CPU
usage
is at 100%, making the comp not so responsive. Is there a way to
control the
CPU usage at say 80%? putting a time.sleep(0.x) doesn't seem to help
although CPU usage level is reduced, but it's unstable.

Regards,
W.J.F.
Controlling a task's scheduling is most definitely OS-dependent., so you
need to say what OS you're running on. And whether it's a multi-core
and or duo processor.

In Windows, there is a generic way to tell the system that you want to
give a boost to whatever task has the user focus (generally the
top-window on the desktop). On some versions, that's the default, on
others, it's not. You change it from Control Panel. I'd have to go
look to tell you what applet, but I don't even know if you're on Windows.

In addition, a program can adjust its own priority, much the way the
Unix 'nice' command works. You'd use the Win32 library for that.

And as you already tried, you can add sleep() operations to your
application.

But if you're looking at the task list in the Windows Task Manager, you
aren't necessarily going to see what you apparently want. There's no
way to programmatically tell the system to use a certain percentage for
a given task. If there's nothing else to do, then a low priority task
is still going to get nearly 100% of the CPU. Good thing. But even if
there are other things to do, the scheduling is a complex interaction
between what kinds of work the various processes have been doing lately,
how much memory load they have, and what priority they're assigned.

If you just want other processes to be "responsive" when they've got the
focus, you may want to make that global setting. But you may need to
better define "responsive" and "unstable."

DaveA
 
D

Dave Angel

Jiang said:
Dear All,

Thank you for the information. I think I've some idea what the problem is
about after seeing the replies.

More information about my system and my script

PIII 1Ghz, 512MB RAM, Windows XP SP3

The script monitors global input using PyHook,
and calculates on the information collected from the events to output some
numbers. Based on the numbers, the script then performs some automation
using SendKeys module.

here is the memory usage:
firefox.exe, 69MB, 109MB
svchost.exe, 26MB, 17MB
pythonw.exe, 22MB, 17MB
searchindexer.exe, 16MB, 19MB

My first guess is that the script calculated for too long time after
receiving an event before propagating it to the default handler, resulting
the system to be non-responsive. I will try to implement the calculation
part in another thread.
Then the separate will have 100% CPU usage, hope the task scheduling of
Windows works in my favour.
(You top-posted this message, putting the whole stream out of order. So
I deleted the history.)

All my assumptions about your environment are now invalid. You don't
have a CPU-bound application, you have a Windows application with event
loop. Further, you're using SendKeys to generate a keystroke to the
other process. So there are many things that could be affecting your
latency, and all my previous guesses are useless.

Adding threads to your application will probably slow the system down
much more. You need to find out what your present problem is before
complicating it.

You haven't really described the problem. You say the system is
unresponsive, but you made it that way by creating a global hook; a
notoriously inefficient mechanism. That global hook inserts code into
every process in the system, and you've got a pretty low-end environment
to begin with. So what's the real problem, and how severe is it? And
how will you measure improvement? The Task manager numbers are probably
irrelevant.

My first question is whether the pyHook event is calling the SendKeys
function directly (after your "lengthy" calculation) or whether there
are other events firing off in between. If it's all being done in the
one event, then measure its time, and gather some statistics (min time,
max time, average...). The task manager has far too simplistic
visibility to be useful for this purpose.

What else is this application doing when it's waiting for a pyHook
call? Whose event loop implementation are you using? And the program
you're trying to control -- is there perhaps another way in?

DaveA
 
K

kakarukeys

(You top-posted this message, putting the whole stream out of order. So
I deleted the history.)

All my assumptions about your environment are now invalid. You don't
have a CPU-bound application, you have a Windows application with event
loop. Further, you're using SendKeys to generate a keystroke to the
other process. So there are many things that could be affecting your
latency, and all my previous guesses are useless.

Adding threads to your application will probably slow the system down
much more. You need to find out what your present problem is before
complicating it.

You haven't really described the problem. You say the system is
unresponsive, but you made it that way by creating a global hook; a
notoriously inefficient mechanism. That global hook inserts code into
every process in the system, and you've got a pretty low-end environment
to begin with. So what's the real problem, and how severe is it? And
how will you measure improvement? The Task manager numbers are probably
irrelevant.

My first question is whether the pyHook event is calling the SendKeys
function directly (after your "lengthy" calculation) or whether there
are other events firing off in between. If it's all being done in the
one event, then measure its time, and gather some statistics (min time,
max time, average...). The task manager has far too simplistic
visibility to be useful for this purpose.

What else is this application doing when it's waiting for a pyHook
call? Whose event loop implementation are you using? And the program
you're trying to control -- is there perhaps another way in?

DaveA

Hi,

Sorry I wasn't sure how to use Google groups to post a msg to the
newsgroup, I used Gmail to write my previous reply. What you and the
other guy have provided me isn't useless. Now I understand the non-
responsiveness may not be caused by high CPU usage, as the OS, be it
Windows or Linux, has a way to prioritize the tasks. This is a vital
clue to me.

By "not responsive", I mean, for some time, the mouse pointer is not
moving smoothly, to such extent that I can't do anything with the
mouse. It's like playing a multi-player game on a connection with a
lot of lag. It's not caused by global hook, because it happens under
certain condition, i.e. when fpa.ProcessEvent(word) is computing.

I included my main script for your reference. Comments:
(1) The automation method tc.Auto() is slow, but it doesn't cause any
problem, because the user would wait for the automation to finish,
before he continues to do something.

(2) all other methods invoked are fast, except fpa.ProcessEvent(word)
(this information is obtained from profiling). It is this method that
causes 100% CPU usage. I'm planning to move this method to a separate
thread, so that OnEvent(event) can finish executing, while the
separate thread goes on to finish its calculation. Is this a good
idea?

import pyHook
import TypingAnalyzer
import GUI

def OnEvent(event):
if hasattr(event, "Key") and event.Ascii == 9 and event.Key == "Tab"
and event.Injected == 0 and event.Alt == 0:
tc.Auto()
return False
else:
recognized = rk.ProcessEvent(event)
if recognized:
tc.MatchChar(recognized)
paragraph = rc.ProcessEvent(recognized)
if paragraph:
for word in paragraph:
fpa.ProcessEvent(word)

return True

hm = pyHook.HookManager()
hm.MouseAllButtonsDown = OnEvent
hm.KeyDown = OnEvent
hm.HookMouse()
hm.HookKeyboard()

rk = TypingAnalyzer.ReadKey()
rc = TypingAnalyzer.ReadChar()
fpa = TypingAnalyzer.Analysis()
tc = TypingAnalyzer.Automation(fpa)

if __name__ == '__main__':
app = GUI.AWAApp()
app.MainLoop()

Thank you for your attention.
 
D

Dave Angel

kakarukeys said:
Hi,

Sorry I wasn't sure how to use Google groups to post a msg to the
newsgroup, I used Gmail to write my previous reply. What you and the
other guy have provided me isn't useless. Now I understand the non-
responsiveness may not be caused by high CPU usage, as the OS, be it
Windows or Linux, has a way to prioritize the tasks. This is a vital
clue to me.

By "not responsive", I mean, for some time, the mouse pointer is not
moving smoothly, to such extent that I can't do anything with the
mouse. It's like playing a multi-player game on a connection with a
lot of lag. It's not caused by global hook, because it happens under
certain condition, i.e. when fpa.ProcessEvent(word) is computing.

I included my main script for your reference. Comments:
(1) The automation method tc.Auto() is slow, but it doesn't cause any
problem, because the user would wait for the automation to finish,
before he continues to do something.

(2) all other methods invoked are fast, except fpa.ProcessEvent(word)
(this information is obtained from profiling). It is this method that
causes 100% CPU usage. I'm planning to move this method to a separate
thread, so that OnEvent(event) can finish executing, while the
separate thread goes on to finish its calculation. Is this a good
idea?

import pyHook
import TypingAnalyzer
import GUI

def OnEvent(event):
if hasattr(event, "Key") and event.Ascii == 9 and event.Key == "Tab"
and event.Injected == 0 and event.Alt == 0:
tc.Auto()
return False
else:
recognized = rk.ProcessEvent(event)
if recognized:
tc.MatchChar(recognized)
paragraph = rc.ProcessEvent(recognized)
if paragraph:
for word in paragraph:
fpa.ProcessEvent(word)

return True

hm = pyHook.HookManager()
hm.MouseAllButtonsDown = OnEvent
hm.KeyDown = OnEvent
hm.HookMouse()
hm.HookKeyboard()

rk = TypingAnalyzer.ReadKey()
rc = TypingAnalyzer.ReadChar()
fpa = TypingAnalyzer.Analysis()
tc = TypingAnalyzer.Automation(fpa)

if __name__ == '__main__':
app = GUI.AWAApp()
app.MainLoop()

Thank you for your attention.
I can't readily comment on your code, since it's entirely written to
three imports that I have never seen. I have to assume that
TypingAnalyzer contains logic to automate your external program. I have
no idea where GUI comes from, but I have to hope it knows how to
efficiently use the CPU, like most gui event loops. If you don't do
the pyHook, is the system quite responsive?

I actually doubt if moving some of the logic to another thread is going
to make any difference. In Python, threading helps to overlap CPU bound
stuff and I/O bound stuff. But theoretically, as soon as you return to
the event loop, your main thread is going to block, so it'll schedule
the other thread. The real question is whether there's communication
going between the two processes in those various rk, rc, etc pieces.
It's easy for such protocols to become very inefficient.

Since I can't directly help, at least I could suggest looking at XP's
ControlPanel->System->Advanced. That gets you to a dialog entitled
Performance Options.

On the Advanced tab of that dialog, the first section is "processor
scheduling". Mine is set to "Programs" which means give extra priority
to the GUI program that has the focus. Presumably in your environment
that'll be the typing code, not your python app. Setting it to
"Background services" would give more priority to the CPU bound of your
python app.

But once these two apps are aware of each other, they could be doing all
sorts of things to confuse the scheduler.

DaveA
 
K

kakarukeys

I can't readily comment on your code, since it's entirely written to
three imports that I have never seen.  I have to assume that
TypingAnalyzer contains logic to automate your external program.  I have
no idea where GUI comes from, but I have to hope it knows how to
efficiently use the CPU, like most gui event loops.  If  you don't do
the pyHook, is the system quite responsive?

I actually doubt if moving some of the logic to another thread is going
to make any difference.  In Python, threading helps to overlap CPU bound
stuff and I/O bound stuff.  But theoretically, as soon as you return to
the event loop, your main thread is going to block, so it'll schedule
the other thread.  The real question is whether there's communication
going between the two processes in those various rk, rc, etc pieces.  
It's easy for such protocols to become very inefficient.

Since I can't directly help, at least I could suggest looking at XP's  
ControlPanel->System->Advanced.  That gets you to a dialog entitled
Performance Options.

On the Advanced tab of that dialog, the first section is "processor
scheduling".   Mine is set to "Programs" which means give extra priority
to the GUI program that has the focus.  Presumably in your environment
that'll be the typing code, not your python app.  Setting it to
"Background services" would give more priority to the CPU bound of your
python app.

But once these two apps are aware of each other, they could be doing all
sorts of things to confuse the scheduler.

DaveA

Hi DaveA,
If you don't do the pyHook, is the system quite responsive

I'm not sure how to answer this question. The program needs pyHook, so
I can't leave it out. Is there other alternative that could monitor
global input? Running other programs without pyHook creates no
problem. Running the pyHook example demo creates no problem either.
But theoretically, as soon as you return to the event loop, your main thread is going to block, so it'll schedule the other thread.

Could you explain more on this part? Your statement is mind boggling
to me.

Cheers,
W.J.F.
 
D

Dave Angel

kakarukeys said:
Hi DaveA,



I'm not sure how to answer this question. The program needs pyHook, so
I can't leave it out. Is there other alternative that could monitor
global input? Running other programs without pyHook creates no
problem. Running the pyHook example demo creates no problem either.



Could you explain more on this part? Your statement is mind boggling
to me.

Cheers,
W.J.F.
An event loop should be blocking, waiting for the OS to produce another
event. If so, then other threads would get control as needed.

But this afternoon I came up with another plausible explanation for the
jerky behavior on your system. You're doing a system hook, and I
suspect there's some form of release needed before the system is a able
to process any more events of the same type. As I said before, I don't
have pyHook, so I haven't read its documentation. But I'm guessing the
hook gets released when you return to the event loop. So if you have
more processing to do, rather than moving it to another thread, move it
to an extra event.

Read the docs for GUI. It will undoubtedly have some way to create new
event types. So register a new event type, attach a handler to it, and
post it from your current event handler using something like: "postafter"

That new event handler is where you do the time-consuming stuff that
you're having trouble with.

DaveA
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,902
Latest member
Elena68X5

Latest Threads

Top