Z
Zac Burns
Greetings,
It seems that marshal.load will lock the problem if the file object
(in this case a pipe) is not ready to be read from - even if it's done
in a thread.
The use case here is in writing a scripting interface for perforce
using the -G option
(http://www.perforce.com/perforce/doc.072/manuals/cmdref/o.gopts.html#1040647).
We are using the sub process module to communicate through pipes to
avoid using the hard-drive. Furthermore, because the machine is using
windows we need threads for each pipe (stdin, stdout, stderr)
initialized from the beginning to keep from buffer lock.
If marshal.load would accept a cStringIO object, I would have a
workaround - marshal.load(cStringIO.cStringIO(pipe.read())) because
cStringIO does not have the same problem and I'm not as worried about
the extra memory as I am the hard-drive use.
I can't read the C code yet, but if I had to make a guess I would say
python is grabbing the GIL and then attempting to read from the pipe,
instead of the other way around.
Could someone help me to understand what is going on and suggest a
workaround? I hope I've been clear about the nature of the problem.
Thanks,
It seems that marshal.load will lock the problem if the file object
(in this case a pipe) is not ready to be read from - even if it's done
in a thread.
The use case here is in writing a scripting interface for perforce
using the -G option
(http://www.perforce.com/perforce/doc.072/manuals/cmdref/o.gopts.html#1040647).
We are using the sub process module to communicate through pipes to
avoid using the hard-drive. Furthermore, because the machine is using
windows we need threads for each pipe (stdin, stdout, stderr)
initialized from the beginning to keep from buffer lock.
If marshal.load would accept a cStringIO object, I would have a
workaround - marshal.load(cStringIO.cStringIO(pipe.read())) because
cStringIO does not have the same problem and I'm not as worried about
the extra memory as I am the hard-drive use.
I can't read the C code yet, but if I had to make a guess I would say
python is grabbing the GIL and then attempting to read from the pipe,
instead of the other way around.
Could someone help me to understand what is going on and suggest a
workaround? I hope I've been clear about the nature of the problem.
Thanks,