Some help needed with small multi-threaded program!

[Oltmans]

I've a small multi-threaded program that is written by keeping
Producer-Consumer pattern in mind. In the Producer part of the program
we create new users by doing a POST to a certain web URL whereas in
the Consumer part of the program we try to login newly created users.
Problem is that program sort of runs fine, however, it never
terminates itself and, more importantly, it doesn't print last-three
lines of the program(which is important for my purposes). I've looked
at the docs, but I couldn't figure out a solution so if someone can
spot the problem and suggest a solution that will be highly
appreciated. Many thanks in advance.

-----
#!/usr/bin/env python

import Queue
import re
import random
import time
import threading
import urllib2
import simplejson
from urllib2 import HTTPError
import urllib

base_url = "http://example.com/"
queue = Queue.Queue()
total_users = 0
#keeps the record of time it takes to create a user
create_user_times = {}
#keeps the record of time it takes to authenticate a user
login_user_times = {}
lock = threading.Lock()
errors = []

class User():
def
__init__(self,firstName,lastName,email,password,userName,active):
self.firstName = firstName
self.lastName = lastName
self.password = password
self.email = email
self.userName = userName
#producer
class CreateUsers(threading.Thread):
def __init__(self,queue,limit):
threading.Thread.__init__(self)
self.queue = queue
self.limit = limit

def create_user(self):

url = base_url + "users/"
email = "stokeywonder+"+str(random.randrange(0,100000))
+"@example.com"
obj = {'firstName' : 'Coyotee',
'lastName' : 'Stevey', 'email' : email,
'password':'7887',
'userName':email
}
req = urllib2.Request(url)
req.add_header('User-Agent', 'Mozilla/4.0 (compatible; MSIE
5.5; Windows NT)')
req.add_header("Content-Type","application/json")
req.add_data(simplejson.dumps(obj))

try:

t = time.time()
f = urllib2.urlopen(req)
page = f.read()
if str(f.code) != '200':
pass
else:
create_user_times[obj['userName']] = str("%.
2f"%float(time.time() - t))
user = User(**obj)
self.queue.put(user)

except:
errors.append(obj['userName'])

def run(self):
global total_users
global lock

while total_users < self.limit:
lock.acquire()
try:
total_users += 1
finally:
lock.release()

self.create_user()
if total_users == self.limit:
try:
self.queue.task_done()
except ValueError:
pass

#Consumer
class TestService(threading.Thread):
def __init__(self,queue):
threading.Thread.__init__(self)
self.queue = queue

def login(self,user):

user_name = user.userName
pwd = user.password
url = base_url+'u?uName='+user_name+'&pwd='+pwd
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
req = urllib2.Request(url)
req.add_header('User-Agent', user_agent)
try:
t = time.time()

f = urllib2.urlopen(req)

if str(f.code) != '200':
pass
else:
login_user_times[user_name] = str("%.
2f"%float(time.time() - t))
except:
errors.append(user_name)

def run (self):
while True:
user = self.queue.get()
if user:
self.login(user)


if __name__ == "__main__":
#limit = number of users
limit = 5
th = []
for i in range(2):
t = CreateUsers(queue,limit)
t1 = TestService(queue)
t.start()
t1.start()
th.append(t1)
th.append(t)

for t in th:
t.join()
print create_user_times
print '---------------'
print login_user_times

 

[Bryan]

I've a small multi-threaded program that is written by keeping
Producer-Consumer pattern in mind. [...]
Problem is that program sort of runs fine, however, it never
terminates itself and, more importantly, it doesn't print last-three
lines of the program(which is important for my purposes). I've looked
at the docs, but I couldn't figure out a solution so if someone can
spot the problem and suggest a solution that will be highly
appreciated. Many thanks in advance.

Well, "the problem" may be a bit presumptuous, but here's *a* problem:

class TestService(threading.Thread): [...]
    def run (self):
        while True:
            user = self.queue.get()
            if user:
                self.login(user)

That thread will not exit, and your main thread tries to join it.

In other news. There's no reason to call queue.task_done() if you
never call queue.join(). Your exception handling is kind of whacked;
that errors[] list doesn't collect enough info and if your program
hangs it's no use.

Your Producer-Consumer pattern is a misfit. Registering a user then
trying to log her in is a nice sequential unit of work that one thread
could handle. The reason for threading in this problem is so that when
one user's work is waiting for the network, you can make progress on
other users.

Hope that helps.

-Bryan Olson