is there enough information?

C

castironpi

Specify


def thloop( th ):
while th.cont:
with th.step[2]:
th.ret= th.cmd+ 1

def acq( th ):
with th.step[1]:
th.cmd= 100
with th.step[3]:
ret1= th.ret
th.step.reset()
assert ret1== 101

Is it enough?
 
P

Paul McGuire

Is it enough?

(Reminds me of the movie Marathon Man, in which Dustin Hoffman is
repeatedly asked by Laurence Olivier, "Is it safe?" Hoffman had no
idea what Olivier was talking about. So Olivier tortured Hoffman some
more to get the answer.)

Enough for what?

Is "th" short for "thread"? Maybe a little too short - try "thd" at
least, and spelling out "thread" isn't out of the question.

What happened to step[0]? Did you know that lists start at 0? Why
are you using a list of synchronization objects?

You would be better off using attributes of th with meaningful names,
instead of elements of a generically named list attribute, "th.step".
Names like "th.assign_command_synch", "th.return_value_synch", etc.
And since there is no synch object that is locked in both methods,
then the methods will run completely devoid of any synchronization.
 
C

castironpi

Is it enough?

(Reminds me of the movie Marathon Man, in which Dustin Hoffman is
repeatedly asked by Laurence Olivier, "Is it safe?"  Hoffman had no
idea what Olivier was talking about.  So Olivier tortured Hoffman some
more to get the answer.)

Enough for what?

Is "th" short for "thread"?  Maybe a little too short - try "thd" at
least, and spelling out "thread" isn't out of the question.

What happened to step[0]?  Did you know that lists start at 0?  Why
are you using a list of synchronization objects?

You would be better off using attributes of th with meaningful names,
instead of elements of a generically named list attribute, "th.step".
Names like "th.assign_command_synch", "th.return_value_synch", etc.
And since there is no synch object that is locked in both methods,
then the methods will run completely devoid of any synchronization.

I recently ran into a case (* would that be helpful to describe here?)
where thread1 had to do something, thread2 had to do something after
that, and thread1 had to wait for that, then do something else, and
thread2 again had to wait before starting the first thing again.

Clarify:

def thdloop( thd ):
while thd.cont:
thd.sig1event.wait()
ret= thd.cmd()
thd.sig2event.set()
thd.seg3event.wait()

and

def consumer( thd ):


EXCEPT that,
 
C

castironpi

Clarify:
def thdloop( thd ):
   while thd.cont:
      thd.sig1event.wait()
      ret= thd.cmd() thd.result= ret
      thd.sig2event.set()
      thd.seg3event.wait()

and

def consumer( thd ):
thd.cmd= function
thd.sig1event.set()
thd.sig2event.wait()
ret= thd.result
thd.sig3event.set()

Except, I missed the shot on the sigevent.clear() placement, and spent
the better part of the day debugging it.

Two options occurred to me, which the first showed up in the earlier
extremely skeletal and cryptic post: Create a class which will ensure
turn-taking of events, using a get method with and integer index, by
waiting for the prior index to complete before starting the next.

You could name the steps in the class and take a specific order of
them too:

class CmdStep( Step ):
order= SetCommand, CallCommand, ReadResult

so cmdstep= CmdStep() / with cmdstep.SetCommand: or with
cmdstep[SetCommand]: blocks until the specified time.

The other option, was to join both these threads, execute the command,
then split them again: if that's simpler and just as general, then
Step is cool, but rather useless.
 
C

castironpi

 Create a class which will ensure
turn-taking of events, using a get method with and integer index, by
waiting for the prior index to complete before starting the next.

from thread import start_new_thread as launch
from threading import Lock
import time
from functools import partial

class WithObj:
def __init__( self, enter, exit ):
self.__enter__, self.__exit__= enter, exit

class Step:
def __init__( self ):
self._index= 0
self._locks= {}
self._opened= False
self._oplock= Lock()
def __getitem__( self, index ):
with self._oplock:
lock= self._locks.get( index, None )
if None is lock:
lock= self._locks[ index ]= Lock()
if index!= self._index:
assert lock.acquire( False )
return WithObj(
partial( self.ienter, index ),
partial( self.iexit, index ) )
def ienter( self, index ):
with self._oplock:
if self._opened:
return self
lock= self._locks.get( index )
assert lock.acquire()
with self._oplock:
return self
def iexit( self, index, *a ):
with self._oplock:
self._index+= 1
lock= self._locks.get( self._index )
if None is not lock:
lock.acquire( False )
lock.release()
def complete( self ):
with self._oplock:
self._index= 0
lock= self._locks.get( 0 )
if None is not lock:
lock.acquire( False )
lock.release()
def open( self ):
with self._oplock:
self._opened= True
for lock in self._locks.values():
lock.acquire( False )
lock.release()

class CustThread:
count= 0
def __init__( self ):
CustThread.count+= 1
self.id= CustThread.count
self.step= Step()
self.cont= True
self.cmd= None
self.ret= None
def __repr__( self ):
return '<CustThread %i>'% self.id

def thloop( thd ):
while thd.cont:
with thd.step[1]:
if not thd.cont: break
print( 'step 1', end= ' ' )
thd.ret= thd.cmd+ 1
with thd.step[3]:
print( 'step 3' )
thd.ret= None
thd.step.complete()

def op100( thd ):
with thd.step[0]:
print( 'step 0', end= ' ' )
thd.cmd= 100
with thd.step[2]:
print( 'step 2', end= ' ' )
ret1= thd.ret
assert ret1== 101

def main( func ):
if __name__== '__main__':
func()

@main
def fmain():
class Case:
def __init__( self ):
self.th1= CustThread()

while 1:
print( '===============================' )
class Case1:
case= Case()
launch( thloop, ( case.th1, ) )
for _ in range( 10 ):
case.th1.cmd= None
case.th1.ret= None
op100( case.th1 )
case.th1.cont= False
case.th1.step.open()
print( 'case complete' )

while 1: time.sleep( 1 )

'''
Kind of trivial, interesting technique, and general. Enter with
thd.step[n]: to indicate step n in a process. with thd.step[m] for m!
= n will block until it exits, and step[n+1] gets the go, or step[0]
with step.complete(). step.open() grants every step, such as in the
case of completion; set termination conditions prior to its call, and
check them in any sensitive loops.

One way to replace magic sequence numbers with meaning is the "Step1
Step2 Step3" style used in namedtuple, but they're only shown bare
here.

Sequence numbers are looked up in a hash; special applications can use
an array, and account for the possibility that with step[10]: may be
called before even step[0]. 0- and 1-based options are customizable.
Sequence length may be known at creation time; if so, the Step
instance can be initialized with it; contraindicating cases may
include with step[n] ... with step[n+2], that don't mean the right
thing in a separate instance.

__getitem__ returns a WithObj, calling __enter__ and __exit__ on which
routes to the Step instance, paired with the specified index. You
could probably cache those.

The example is a 1-SetOp, 2-DoOp, 1-GetReturn triplet.
acquire( False ) / release() pairs release a thread, whether it is
acquired already or not. _oplock synchronizes _index and lock lookup
operations across an instance.

Kind of cool.
'''
 
P

Preston Landers

Two options occurred to me, which the first showed up in the earlier
extremely skeletal and cryptic post:

Perhaps you would be more likely to get the kind of help you seem to
want
if you refrained from posting "cryptic and skeletal" messages. The
fact that many
other people have pointed this out to you as of late would tend to
suggest
you are trolling, i.e. intentionally trying to foster miscommunication
and threads
that do nothing to advance anyones understanding.

And regarding your other recent post about trying to find a "solution"
to the "problem"
of immutable types... Due to the above reasons you are unlikely to
influence the
design of the core language with half-baked stream of consciousness
ramblings. These
belong in your LiveJournal, not in c.l.python.

If you have a problem you need help with, please read this entire
document about 3 times
before posting anything else:

http://catb.org/~esr/faqs/smart-questions.html

Specifically this:

http://catb.org/~esr/faqs/smart-questions.html#beprecise

and this:

http://catb.org/~esr/faqs/smart-questions.html#goal
 
C

castironpi

Perhaps you would be more likely to get the kind of help you seem to
want
if you refrained from posting "cryptic and skeletal" messages. The
fact that many
other people have pointed this out to you as of late would tend to
suggest
you are trolling, i.e. intentionally trying to foster miscommunication
and threads
that do nothing to advance anyones understanding.

And regarding your other recent post about trying to find a "solution"
to the "problem"
of immutable types...  Due to the above reasons you are unlikely to
influence the
design of the core language with half-baked stream of consciousness
ramblings. These
belong in your LiveJournal, not in c.l.python.

If you have a problem you need help with, please read this entire
document about 3 times
before posting anything else:

http://catb.org/~esr/faqs/smart-questions.html

Specifically this:

http://catb.org/~esr/faqs/smart-questions.html#beprecise

and this:

http://catb.org/~esr/faqs/smart-questions.html#goal

Ugh, very well. You call for an explanation.

Back home, the original post would be interesting, so I wrote it.
Whatever reactions other people have to them is information that is
unavailable to me. I don't know you. I'm rather irked by a
proportion of posts, but for my part, it's hard to get me to point a
finger.

I am not a troll. I want a sustainable, healthy, productive,
educational, informative relationship with frequenters of c.l.p, the
Python community at large, and anyone who has anything non-negative to
contribute. If you are wanting to see how I react to hostility, just
ask. I'll fake it for you, but only for a second at a time.

Now, what help is it that you believe I seem to want? All I asked for
was, ideas.
 
P

Preston Landers

All I asked for was, ideas.

Nope, you didn't. You said exactly this: "Specify {...} Is it
enough?" and included a snipped of code that was not standalone,
provided no context or explanatory information, and gave us no clue
what you might be trying to accomplish.

To me this does not qualify as asking for ideas. It's more like
posting nonsense with a thin veneer that looks like it's intended to
suggest there is some hidden meaning lurking within and that we are
expected to play 20 questions with you to tease out whatever it is you
might be getting at, presumably some brilliant gem of an insight that
will cause everyone to question our basic intellectual assumptions in
some profound philosophical epiphany.

This has all the hallmarks of being a fun game for you but a waste of
time for us.
 
M

Marc 'BlackJack' Rintsch

Back home, the original post would be interesting, so I wrote it.

So you think of this group as your personal notepad. That explains a lot. :-/

Ciao,
Marc 'BlackJack' Rintsch
 
J

Jeff Schwab

Ugh, very well. You call for an explanation.

Back home, the original post would be interesting, so I wrote it.
Whatever reactions other people have to them is information that is
unavailable to me. I don't know you. I'm rather irked by a
proportion of posts, but for my part, it's hard to get me to point a
finger.

I am not a troll. I want a sustainable, healthy, productive,
educational, informative relationship with frequenters of c.l.p, the
Python community at large, and anyone who has anything non-negative to
contribute. If you are wanting to see how I react to hostility, just
ask. I'll fake it for you, but only for a second at a time.

Wow. I sure hope I don't come across like castiron does here.
Now, what help is it that you believe I seem to want? All I asked for
was, ideas.

It's a little difficult for me to interpret your code, partly because I
am nbt very familiar with Python's support for concurrency. But what
are you trying to a achieve?

You mentioned: "I recently ran into a case (* would that be helpful to
describe here?) where thread1 had to do something, thread2 had to do
something after that, and thread1 had to wait for that, then do
something else, and thread2 again had to wait before starting the first
thing again."

This is ordinarily called a Producer-Consumer model. It is often
implemented using semaphores. Googling "python semaphore" turns up this
documentation:

http://www.python.org/doc/lib/semaphore-objects.html

That page, in turn, links to an example of the proper use of semaphores
in Python. Does that help?
 
C

castironpi

Wow.  I sure hope I don't come across like castiron does here.


It's a little difficult for me to interpret your code, partly because I
am nbt very familiar with Python's support for concurrency.  But what
are you trying to a achieve?

You mentioned:  "I recently ran into a case (* would that be helpful to
describe here?) where thread1 had to do something, thread2 had to do
something after that, and thread1 had to wait for that, then do
something else, and thread2 again had to wait before starting the first
thing again."

This is ordinarily called a Producer-Consumer model.  It is often
implemented using semaphores.  Googling "python semaphore" turns up this
documentation:

http://www.python.org/doc/lib/semaphore-objects.html

That page, in turn, links to an example of the proper use of semaphores
in Python.  Does that help?- Hide quoted text -

- Show quoted text -

Hi Jeff. I've enjoyed your recent posts.

I'm not quite sure a semaphore is exactly the synchronization object
I'm looking for, but I'm a little new to concurrency myself.

In the interface I design, only one with-call can get the result at
once. It was my understanding that semaphores, and many other synch.
objs. returned control at random.

In fact, in the background, I'm working on something a little more
substantial than this, but it's not done, so the only review of it I
can perform is of its interface.

If someone has a "yes, but in half the lines, at twice the speed,"
then tear my posts to shreds.
 
J

Jeff Schwab

Hi Jeff. I've enjoyed your recent posts.

I'm not quite sure a semaphore is exactly the synchronization object
I'm looking for, but I'm a little new to concurrency myself.

In the interface I design, only one with-call can get the result at
once. It was my understanding that semaphores, and many other synch.
objs. returned control at random.

I take this to mean that your interface offers a function returns
immediately, rather than waiting for the work to complete. Is that correct?

In fact, in the background, I'm working on something a little more
substantial than this, but it's not done, so the only review of it I
can perform is of its interface.

The interface is (in my opinion) usually the best place to start the
code, anyway.

If someone has a "yes, but in half the lines, at twice the speed,"
then tear my posts to shreds.

It is not quite clear what your code is intended to do. That doesn't
mean there's necessarily anything wrong with it, but it's hard for most
Usenetters to take the time to read such long sequences of code. Would
it be possible for you to post a complete program, that we can actually
run? Wherever your code is not yet ready, just put a line or two of
"stub" code, and add a comment to explain what should be happening.
 
C

castironpi

I take this to mean that your interface offers a function returns
immediately, rather than waiting for the work to complete.  Is that correct?


The interface is (in my opinion) usually the best place to start the
code, anyway.


It is not quite clear what your code is intended to do.  That doesn't
mean there's necessarily anything wrong with it, but it's hard for most
Usenetters to take the time to read such long sequences of code.  Would
it be possible for you to post a complete program, that we can actually
run?  Wherever your code is not yet ready, just put a line or two of
"stub" code, and add a comment to explain what should be happening.- Hide quoted text -

- Show quoted text -

Sure. And honestly, I have no idea what the best way to go about this
is, except keep trying.

th1 th2
set cmd
run cmd
get result
acknowledge
continue continue

th2 won't -run cmd- until th1 completes -set cmd-. th1 won't -get
result- until th2 completes -run cmd-. and once -acknowledge-
completes, both can go about their merry ways. In the example last
night, th2 continued to loop to handle requests in a collection of
threads, but th1 had pressing business elsewhere.

Dated 05:07 PST, the code should be runnable. But the only thing is,
I developed it in Python 3.0a2. In particular, line 71 def
thloop( thd ), and line 82 def op100( thd ), should demonstrate that
interface.

Newsgroups: comp.lang.python
From: (e-mail address removed)
Date: Tue, 26 Feb 2008 05:07:48 -0800 (PST)
Local: Tues, Feb 26 2008 7:07 am
Subject: Re: is there enough information?

The interface is an awesome place to start.
 
J

Jeff Schwab

Sure. And honestly, I have no idea what the best way to go about this
is, except keep trying.

th1 th2
set cmd
run cmd
get result
acknowledge
continue continue

th2 won't -run cmd- until th1 completes -set cmd-. th1 won't -get
result- until th2 completes -run cmd-. and once -acknowledge-
completes, both can go about their merry ways.

That is exactly the kind of case where semaphores are usually used.
Thread1 can "fill" the semaphore when the command is ready, then Thread2
can "empty" the semaphore once it has finished running the command.

In the example last
night, th2 continued to loop to handle requests in a collection of
threads, but th1 had pressing business elsewhere.

No problem. It should be waiting on a semaphore, though, not just
"hot-looping." (I am amused by the idea of a thread having pressing
business elsewhere.)

Dated 05:07 PST, the code should be runnable. But the only thing is,
I developed it in Python 3.0a2. In particular, line 71 def
thloop( thd ), and line 82 def op100( thd ), should demonstrate that
interface.

Sorry, I am still having some trouble following it. :-( Maybe somebody
else here is also trying 3.0 already.
 
C

castironpi

That is exactly the kind of case where semaphores are usually used.
Thread1 can "fill" the semaphore when the command is ready, then Thread2
can "empty" the semaphore once it has finished running the command.


No problem.  It should be waiting on a semaphore, though, not just
"hot-looping."  (I am amused by the idea of a thread having pressing
business elsewhere.)


Sorry, I am still having some trouble following it.  :-(  Maybe somebody
else here is also trying 3.0 already.- Hide quoted text -

- Show quoted text -

The relevant snippet is:

def thloop( thd ):
while thd.cont:
with thd.step[1]:
if not thd.cont: break
print( 'step 1', end= ' ' )
thd.ret= thd.cmd+ 1
with thd.step[3]:
print( 'step 3' )
thd.ret= None
thd.step.complete()

def op100( thd ):
with thd.step[0]:
print( 'step 0', end= ' ' )
thd.cmd= 100
with thd.step[2]:
print( 'step 2', end= ' ' )
ret1= thd.ret
assert ret1== 101

I don't see how semaphores could prevent thd.step[2] and thd.step[1]
from taking turns in the wrong order. Polling is bad.
 
J

Jeff Schwab

That is exactly the kind of case where semaphores are usually used.
Thread1 can "fill" the semaphore when the command is ready, then Thread2
can "empty" the semaphore once it has finished running the command.

No problem. It should be waiting on a semaphore, though, not just
"hot-looping." (I am amused by the idea of a thread having pressing
business elsewhere.)

Sorry, I am still having some trouble following it. :-( Maybe somebody
else here is also trying 3.0 already.- Hide quoted text -

- Show quoted text -

The relevant snippet is:

def thloop( thd ):
while thd.cont:
with thd.step[1]:
if not thd.cont: break
print( 'step 1', end= ' ' )
thd.ret= thd.cmd+ 1
with thd.step[3]:
print( 'step 3' )
thd.ret= None
thd.step.complete()

def op100( thd ):
with thd.step[0]:
print( 'step 0', end= ' ' )
thd.cmd= 100
with thd.step[2]:
print( 'step 2', end= ' ' )
ret1= thd.ret
assert ret1== 101

I don't see how semaphores could prevent thd.step[2] and thd.step[1]
from taking turns in the wrong order. Polling is bad.

I'll try to take a closer look at this a little later. It will be a
good chance to learn Python's semaphore interface. However, I am
getting the feeling you may have to solve this one on your own. :(
 
C

castironpi

The relevant snippet is:
def thloop( thd ):
    while thd.cont:
        with thd.step[1]:
            if not thd.cont: break
            print( 'step 1', end= ' ' )
            thd.ret= thd.cmd+ 1
        with thd.step[3]:
            print( 'step 3' )
            thd.ret= None
        thd.step.complete()
def op100( thd ):
    with thd.step[0]:
        print( 'step 0', end= ' ' )
        thd.cmd= 100
    with thd.step[2]:
        print( 'step 2', end= ' ' )
        ret1= thd.ret
    assert ret1== 101
I don't see how semaphores could prevent thd.step[2] and thd.step[1]
from taking turns in the wrong order.  Polling is bad.

I'll try to take a closer look at this a little later.  It will be a
good chance to learn Python's semaphore interface.  However, I am
getting the feeling you may have to solve this one on your own. :(- Hide quoted text -

- Show quoted text -

I want to awaken -that- -specific- -thread-.

Semaphore acquire: "The implementation may pick one at random."
Lock acquire: "which one of the waiting threads proceeds is not
defined"
Event set: "All threads waiting for it to become true are awakened"
Condition notify: "This method wakes up one of the threads waiting"
Condition notifyAll: "Wake up all threads waiting on this condition"

I will contend it's impossible without a specialization. Certainly an
array of semaphores could fit the bill; is that what you were
suggesting?
 
D

Dennis Lee Bieber

Unfortunately, your posts come across more as "give, give, give"
with no "take"
I'm not quite sure a semaphore is exactly the synchronization object
I'm looking for, but I'm a little new to concurrency myself.
Then wouldn't it behoove you to study a few papers on the general
subject of concurrency, followed by mapping the concepts to the Python
implementations?
In the interface I design, only one with-call can get the result at
once. It was my understanding that semaphores, and many other synch.
objs. returned control at random.
Perhaps: http://www.greenteapress.com/semaphores/
will be of use. Almost 300 pages of concurrency concerns, with a chapter
specific to Python.
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/
 
C

castironpi

        Unfortunately, your posts come across more as "give, give, give"
with no "take"> I'm not quite sure a semaphore is exactly the synchronization object

        Then wouldn't it behoove you to study a few papers on the general
subject of concurrency, followed by mapping the concepts to the Python
implementations?


        Perhaps:  http://www.greenteapress.com/semaphores/
will be of use. Almost 300 pages of concurrency concerns, with a chapter
specific to Python.
--
        Wulfraed        Dennis Lee Bieber               KD6MOG
        (e-mail address removed)                (e-mail address removed)
                HTTP://wlfraed.home.netcom.com/
        (Bestiaria Support Staff:               (e-mail address removed))
                HTTP://www.bestiaria.com/

I want to waken a specific thread that's waiting on a S.O., synchro
object, based on an index.

From the docs:
Semaphore acquire: "The implementation may pick one at random."

Thus semaphores will not do, outside of an array of them. Did I miss
something?

A: I want a specific thread.
B: Semaphores pick one at random.
||| I don't want semaphores.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
474,431
Messages
2,571,679
Members
48,796
Latest member
Greg L.

Latest Threads

Top