hyperthreading locks up sleeping threads

O

OlafMeding

Below are 2 files. The first is a Python program that isolates the
problem within less than 1 hour (often just a few minutes). The second
is a C++ program that shows that the Win32 Sleep() function works as
expected (ran from Friday afternoon until Monday morning).

Note, the Python programs hangs (stops responding) with hyper-threading
turned on (a BIOS setting), but works as expected with hyper-threading
turned off.

This problem happens on Windows only (not on Linux for days).

Variations of the Python program also lock up:

Tried importing win32api instead of time and using the
win32api.GetTickCount() and win32api.Sleep() methods.

Tried using lock = threading.Event() and lock.wait() instead of
time.sleep().

Tried import Queue using q = Queue.Queue() and q.get(True, self.t).

Note, the Windows task manager shows 2 CPUs on the Performance tab with
hyper-threading is turned on.

Both Python 2.3.5 and 2.4.3 (downloaded from python.org) have this
problem.

The operating system is MS Windows XP Professional.

winmsd.exe shows:
2CPUs: x86 Family 15 Model 4 Stepping 1 GenuineIntel ~3000 MHz
Version: 5.1.2600 Service Pack 2 Build 2600

Could someone with a hyper-threading (or dual core or multi processor)
CPU please
confirm this bug?

Many Thanks

Olaf


Here is the expected output of both programs (the progam has locked up
if the numbers stop printing):
python testsleep.py
thread 1 started, sleep time 0.010
thread 2 started, sleep time 0.003
1 1 1 2 1 1 1 2 1 1 1 2 1 1 1 1 2 1 1 1


# testsleep.py
import threading
import time

class Task(threading.Thread):
def __init__(self, n, t):
threading.Thread.__init__(self)
self.n = n # thread id
self.t = t # sleep time
def run(self):
print 'thread %d started, sleep time %.3f' % (self.n, self.t)
count = 0
printCount = int(10 / self.t)
while True:
start = time.clock()
time.sleep(self.t)
stop = time.clock()
if stop - start > 1.0:
print 'thread', self.n, stop - start

count += 1
if count > printCount:
count = 0
print self.n, # print sign of live

def test():
thread1 = Task(1, 0.01) # thread 1, sleep 10 ms
thread2 = Task(2, 0.003) # thread 2, sleep 3 ms
thread1.start()
thread2.start()

test()

----------------------------------------------------------------------------------

// testsleep.cpp
// Compiled with Visual C++ version 6 as a Win32 console application.

#include <windows.h>
#include <stdio.h>
#include <time.h>

typedef struct {
int id;
int ms;
} param_s;


DWORD WINAPI threadFunction(LPVOID param)
{
param_s* p = (param_s*)param;
long elapsedTime;
long time1, time2;
long printCount = long(10000 / p->ms); // loop iterations in 10
seconds
long count = 0;

printf("thread %d started, sleep time: %d ms" "\n", p->id, p->ms);

while(true) {
time1 = GetTickCount();
Sleep(p->ms);
time2 = GetTickCount();

elapsedTime = time2 - time1;
if(elapsedTime > 1000)
printf("thread %d slept for %d ms" "\n", p->id,
elapsedTime);

count++;
if(count > printCount) {
count = 0;
printf("%d ", p->id); // print sign of live
}
}

return 0;
}


int main(int argc, char* argv[])
{
long time1, time2;
param_s p1, p2;

p1.id = 1;
p1.ms = 10;

p2.id = 2;
p2.ms = 3;

time1 = GetTickCount();
while(true) {
time2 = GetTickCount();
if (time1 != time2) {
printf("clock resolution: %d ms" "\n", time2 - time1);
break;
}
}

CreateThread(NULL, 0, threadFunction, (void*)&p1, 0, NULL);
CreateThread(NULL, 0, threadFunction, (void*)&p2, 0, NULL);

getchar(); // wait until the user presses the enter key.

return 0;
}
 
T

Tim Peters

[[email protected]]
Below are 2 files. The first is a Python program that isolates the
problem within less than 1 hour (often just a few minutes).

It does not on my box. I ran that program, from a DOS shell, using
the released Windows Python 2.4.3. After an hour, it was still
printing. I left it running, and started a second instance of the
test program from another DOS box. That was 4 hours ago, and both
instances are still printing.
...
Note, the Python programs hangs (stops responding) with hyper-threading
turned on (a BIOS setting), but works as expected with hyper-threading
turned off.

Hyper-threading was turned on here.
...
Note, the Windows task manager shows 2 CPUs on the Performance tab with
hyper-threading is turned on.

Same here, although do note that whether the Performance tab shows one
or two CPUs when HT is enabled depends on how the user sets Task
Manager's View -> CPU History option.
Both Python 2.3.5 and 2.4.3 (downloaded from python.org) have this
problem.

I used the python.org 2.4.3.
The operating system is MS Windows XP Professional.

Same here.
winmsd.exe shows:
2CPUs: x86 Family 15 Model 4 Stepping 1 GenuineIntel ~3000 MHz
Version: 5.1.2600 Service Pack 2 Build 2600

Different processor here:

2CPUs: x86 Family 15 Model 3 Stepping 4 GenuineIntel ~3400 Mhz
Version 5.1.2600 Service Pack 2 Build 2600
Could someone with a hyper-threading (or dual core or multi processor)
CPU please confirm this bug?

Not me, and I'm tired of trying :)
 
O

OlafMeding

Tim

Many thanks for trying and reporting the details of your environment.
All our hyper-threading PC are identical. However, we identified one
that is different and we are installing Windows XP on it now ...

My hope is that other people will try this, too.

Olaf
 
S

Serge Orlov

Tried importing win32api instead of time and using the
win32api.GetTickCount() and win32api.Sleep() methods.

What about win32api.SleepEx? What about

WaitForMultipleObjects
WaitForMultipleObjectsEx
WaitForSingleObject
WaitForSingleObjectEx

when the object is not expected to produce events and the function
timeouts?
 
R

Robin Becker

Below are 2 files. The first is a Python program that isolates the
problem within less than 1 hour (often just a few minutes). The second
is a C++ program that shows that the Win32 Sleep() function works as
expected (ran from Friday afternoon until Monday morning).

Note, the Python programs hangs (stops responding) with hyper-threading
turned on (a BIOS setting), but works as expected with hyper-threading
turned off.
.......
I ran this on winxp ht+python-2.4.3 for about 16 hours and no lockup.
 
R

Roel Schroeven

(e-mail address removed) schreef:
Below are 2 files. The first is a Python program that isolates the
problem within less than 1 hour (often just a few minutes). The second
is a C++ program that shows that the Win32 Sleep() function works as
expected (ran from Friday afternoon until Monday morning).

Note, the Python programs hangs (stops responding) with hyper-threading
turned on (a BIOS setting), but works as expected with hyper-threading
turned off.

This problem happens on Windows only (not on Linux for days).
The operating system is MS Windows XP Professional.
Could someone with a hyper-threading (or dual core or multi processor)
CPU please
confirm this bug?

Doesn't lock up on my system after 6 hours. Windows XP Pro, Python
2.4.2, hyperthreading Pentium 4.
 
O

OlafMeding

Dave send me the below as an email. Olaf

Hi Olaf,

I'm running your test for you - it's been going for about an hour now
and is continuing to generate output[1].

c:\>py
Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
winmsd says:
OS Name Microsoft Windows XP Professional
Version 5.1.2600 Service Pack 1 Build 2600
Processor x86 Family 15 Model 2 Stepping 9 GenuineIntel ~2992 Mhz
Processor x86 Family 15 Model 2 Stepping 9 GenuineIntel ~2992 Mhz

(it's a hyperthreaded P4 - not 2 true CPUs - so it should be similar to
your setup)

-Dave

[1] Here's the output (line breaks added by my email program):

c:\Temp>testsleep.py
thread 1 started, sleep time 0.010
thread 2 started, sleep time 0.003
1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2
1 2 1 2 1
1 2 1 2
1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1
2 1 2 1 1
2 1 2 1
2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1
1 2 1 2 1
2 1 2 1
2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1
2 1 2 1 2
1 2 1 1
2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1
2 1 2 1 2
1 1 2 1
2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2
1 2 1 1 2
1 2 1 2
1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2
1 1 2 1 2
1 2 1 2
1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2
1 2 1 2 1
2 1 2 1
1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2
1 2 1 2 1
2 1 1 2
1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1
2 1 2 1 1
2 1 2 1
2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1
2 1 1 2 1
2 1 2 1
2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1
2 1 2 1 2
1 2 1 2
1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1
2 1 2 1 2
1 2 1 1
2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2 1 2 1 2 1 2 1 1 2 1 2
1 2
 
O

OlafMeding

Robin and Roel

Thanks for trying and reporting the results. Both of you and Tim and
Dave have run the .py program now (all on hyper-threaded CPUs) and none
of you were able to reproduce the problem.

So that indicates that there is something special about our PC. We are
planing to re-install Windows XP (and nothing else) and try again.

Also, we have one other PC (with a different configuration/CPU) and are
planing to do the same to that PC.

Olaf
 
A

Avizoa

No lockup for me after 28 hours.

Of course, I don't have HT. It's a dual Opteron system with WinXP.
 
O

OlafMeding

This is an update on what we found so far:

We located one other PC that was not identical to the PC with the
problem. So we installed Windows XP on it and ran the Python test
program. It ran fine all night w/o locking up.

Here is what winmsd reports for this PC:
winmsd:
OS Name: Microsoft Windows XP Professional
Version: 5.1.2600 Service Pack 2 Build 2600
Processor: x86 Family 15 Model 4 Stepping 3 GenuineIntel ~3600Mhz
Processor: x86 Family 15 Model 4 Stepping 3 GenuineIntel ~3600Mhz

Okay so this together with what others reported on in this newsgroup
leads to the conclusion that the problem is with the specific software
or hardware configuration of our PC.

So to make sure that no other software is involved we took the PC with
the problem, reset the BIOS to its defaults (which also enables
hyper-threading) and installed Windows XP Professional (SP2) on it (but
no further driver software, etc). We accepted all defaults during the
installation.

Once Windows XP was running we installed Python downloaded from (and
accepted all defaults during the installation):
http://www.python.org/ftp/python/2.4.3/python-2.4.3.msi

And we then ran our Python test program and it hung after less than 15
minutes!

We are now contacting the company that puts that PC together for
further advice.

Olaf
 
G

Grant Edwards

So to make sure that no other software is involved we took the
PC with the problem, reset the BIOS to its defaults (which
also enables hyper-threading) and installed Windows XP
Professional (SP2) on it (but no further driver software,
etc). We accepted all defaults during the installation.

Once Windows XP was running we installed Python downloaded
from (and accepted all defaults during the installation):
http://www.python.org/ftp/python/2.4.3/python-2.4.3.msi

And we then ran our Python test program and it hung after less
than 15 minutes!

We are now contacting the company that puts that PC together
for further advice.

You might want to run some memory tests. I don't see why
hyperthreading would make a difference, but running the test is
dead simple: download the ISO image, burn a CD, boot from the
CD, come back in a day or two:

http://www.memtest86.com/
 
D

David Reed

Grant


We have multiple identical boxes and they all have the same problem.

Olaf


They might all have flaky memory - I would follow the other poster's
advice and run memtest86 on them.

Dave
 
G

Grant Edwards

Grant


We have multiple identical boxes and they all have the same problem.

Maybe whoever built them got a bad batch of RAM. Or maybe the
just used RAM with the wrong specs. It doesn't really cost
anything to run a memory test for a few hours, and if it
passes, then you can cross that off the list.
 
O

OlafMeding

Update

The problem turned out to be the BIOS of the PC we were using. The
Python test program has been running fine for 5 days now (after we
upgraded the system BIOS) and is still running fine.

Sorry, I do not have any information as to what was fixed in the BIOS.
Also, I do not know exactly who made the motherboard or the BIOS. I
will post another update if this information becomes available.

This was sure a strange bug!

Olaf
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,768
Messages
2,569,574
Members
45,050
Latest member
AngelS122

Latest Threads

Top