Best way to implement a timed queue?

[Thomas Ploch]

Hello folks,

I am having troubles with implementing a timed queue. I am using the
'Queue' module to manage several queues. But I want a timed access, i.e.
only 2 fetches per second max. I am horribly stuck on even how I
actually could write it. Has somebody done that before? And when yes,
how is the best way to implement it?

Thanks,
Thomas

 

[Bjoern Schliessmann]

I am having troubles with implementing a timed queue. I am using
the 'Queue' module to manage several queues. But I want a timed
access, i.e. only 2 fetches per second max. I am horribly stuck on
even how I actually could write it. Has somebody done that before?
And when yes, how is the best way to implement it?

If you use an event loop system you could derive a class from your
queue class whose "pop" method only returns an element if some
timer has run out. After the maximum number of fetches you'd have
to reset the timer.

Regards,


Björn

 

[=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=]

I am having troubles with implementing a timed queue. I am using the
'Queue' module to manage several queues. But I want a timed access, i.e.
only 2 fetches per second max. I am horribly stuck on even how I
actually could write it. Has somebody done that before? And when yes,
how is the best way to implement it?

You could put a wrapper around the queue which synchronizes the get
operations, and then delays access until 1s after the last-but-one
access.

The following code is untested:

import threading, time

class ThrottledQueue(threading.Queue):
def __init__(self):
threading.Queue.__init__(self)
self.old = self.older = 0
self.get_lock = threading.Lock()

def get(self):
with self.get_lock: # synchronize get
# check whether the next get should be in the future
now = time.time()
next = self.older + 1
if now < next: time.sleep(next-now)
# really fetch one item; this may block
result = threading.Queue.get(self)
self.older = self.old
# set the last get time to the time when the get completed,
# not when it started
self.old = time.time()
return result

HTH,
Martin