thread execution order

[Rembrandt Q Einstein]

Hi, I have an app that writes text messages into a pipe and reads it out.
Other programs write into this pipe too. Sometimes the pipe is full and
my app freezes if it wants to write in that moment. It is not an option
for me to open the pipe with O_NONBLOCK, so I thought I could
use threads. Always when something shall be written into the pipe a new
thread is created. These threads may hang on os.write, but they'll write out
the data ASAP.

I wouldn't do this with threads. I'd use select() or similar to see if
the pipe is writable and if it isn't, buffer internally.

 

[Axel Mittendorf]

Hi, I have an app that writes text messages into a pipe and reads it out.
Other programs write into this pipe too. Sometimes the pipe is full and
my app freezes if it wants to write in that moment. It is not an option
for me to open the pipe with O_NONBLOCK, so I thought I could
use threads. Always when something shall be written into the pipe a new
thread is created. These threads may hang on os.write, but they'll write out
the data ASAP.
See the example below (I substituted os.write with lock.acquire(1)).
But if the pipe was full, three threads were created (all hang on os.write)
and
now the pipe is emptied, which one will write its data at first? The threads
ordered by the time of thier creation (first no 1 then no 2,3,4 ... n) or
just one of them
(not ordered, maybe no4 then no1 and then no3)?
I ask this since the order of the messages is important.

# little example
import threading

class TO(object):
def __init__(self):
self.lock = threading.Lock()
self.lock.acquire(0)
def dowrite(self,s):
self.lock.acquire(1) # this should be a blocking os.write
print "dowrite:",s
self.lock.release()
def write(self,s):
print "write:",s
t = threading.Thread(target= self.dowrite,args=(s,))
t.start()

to = TO()
to.write("one")
to.write("two")
to.write("three")
to.lock.release()

TIA, Axel Mittendorf

 

[Axel Mittendorf]

I wouldn't do this with threads. I'd use select() or similar to see if
the pipe is writable and if it isn't, buffer internally.

I thought about something like this too, but since I'm not allowed to
use some kinda timer to write out the pending data I can't it that way.
Therefor I wondered to use threads.

THX, Axel

 

[Dennis Lee Bieber]

I thought about something like this too, but since I'm not allowed to
use some kinda timer to write out the pending data I can't it that way.
Therefor I wondered to use threads.

Possibly using ONE writing thread, and a Python Queue -- since
you say the message order is significant.

{pseudo-code}

loop
msg = q.get() #block if no message
pipe.write(msg) #can also block if pipe full


Probably need to create the Queue with no limit on messages,
otherwise you're back to blocking in the main routine.

--

 

[Rembrandt Q Einstein]

I thought about something like this too, but since I'm not allowed to
use some kinda timer to write out the pending data I can't it that way.
Therefor I wondered to use threads.

You don't need a timer. Just every time your program tries to write, it
first checks the pipe. If it can write, it does. If it can't, it
buffers until the next time it tries.

 

[Axel Mittendorf]

You don't need a timer. Just every time your program tries to write, it
first checks the pipe. If it can write, it does. If it can't, it
buffers until the next time it tries.

What if there is no "next time". The buffered message might be buffered
until
my not yet planned child can program perl (no way, tv is enough)
or Linux is sold in bottles ;-)

THX, Axel

 

[Rembrandt Q Einstein]

What if there is no "next time". The buffered message might be buffered
until
my not yet planned child can program perl (no way, tv is enough)
or Linux is sold in bottles ;-)

THX, Axel

Does the pipe eventually close? Write the rest of the data then.

Otherwise, I like the idea of a single thread that reads a common buffer
and writes and blocks however much it wants.

 

[Ville Vainio]

Axel> use threads. Always when something shall be written into the
Axel> pipe a new thread is created. These threads may hang on
Axel> os.write, but they'll write out the data ASAP.

Use single thread that feeds the pipe from messages it gets from
a Queue.Queue instance.

 

[Elbert Lev]

I thought about something like this too, but since I'm not allowed to
use some kinda timer to write out the pending data I can't it that way.
Therefor I wondered to use threads.

THX, Axel

Alex!

In simular situations I use one writer thread and Queue per
pipe/socket etc. Worker threads put messages into the queue and
continue. Writer thread gets messages from the queue and writes them
into the pipe. If pipe is full writer waits, but workers are
operational. Writer thread is alive as long as the pipe exists. Queue
is used as a buffer.

Good luck.
Lev.

 

[G. S. Hayes]

I thought about something like this too, but since I'm not allowed to
use some kinda timer to write out the pending data I can't it that way.
Therefor I wondered to use threads.

You shouldn't need a timer to do it with select(). That's how I'd do it too.