Writing to same file from two threads

[Jens Thoms Toerring]

Hi,

I noticed in someone elses program that it writes single
lines to the same file from (what I call for loss of a better
name) the "main thread" of the program and from a thread sub-
sequentally started. This got me worried if it might result
in garbled output (i.e. having some output from A inside a
line written by B or vice versae) because the "main thread" or
the other thread could be interrupted during a call of write().
Is this a valid concern (and thus locking the file object is
required before writing to it) or am I guaranteed that this
can't happen? In the latter case I would be grateful for an
explanation what mechanism is responsible for this never to
happen.
Thanks and best regards, Jens

PS: I already have determined experimentally that a context
switch definitely can happen between two calls of write()
(and I expected nothing else), what I'm worried about are
context switches somewhere within the very innards of what
write() does.

 

[Paul Rubin]

in garbled output (i.e. having some output from A inside a
line written by B or vice versae) because the "main thread" or

Yes they do get garbled like that. Preferred Python style is put a
single thread in charge of all the i/o to that file, and communicate
with it by message passing through Queue objects. That is safer than
directly using locks.

 

[Jens Thoms Toerring]

Yes they do get garbled like that. Preferred Python style is put a
single thread in charge of all the i/o to that file, and communicate
with it by message passing through Queue objects. That is safer than
directly using locks.

Thank you for confirmig my suspicion;-) But you have induced
another question: why is using a Queue safer than locking (not
that I doubt that it might be more elegant etc.). Is it "safer"
because it's less likely that one gets it wrong (e.g. by for-
grtting to acquire the lock) or is there something inherently
unsafe about locks?

Thank you and best regards, Jens

 

[Antoine Pitrou]

Thank you for confirmig my suspicion But you have induced
another question: why is using a Queue safer than locking (not
that I doubt that it might be more elegant etc.). Is it "safer"
because it's less likely that one gets it wrong (e.g. by for-
grtting to acquire the lock) or is there something inherently
unsafe about locks?

For the record, binary files are thread-safe in Python 3, but text files
are not.
Locks are safe if you use them well. As you point out, if you forget
to acquire your lock, or if you devise a situation where there is a
deadlock between competing locks, you can have difficult to diagnose
issues. Queues have their internal locking all done for you.

Regards

Antoine.

 

[Jens Thoms Toerring]

For the record, binary files are thread-safe in Python 3, but text files
are not.
Locks are safe if you use them well. As you point out, if you forget
to acquire your lock, or if you devise a situation where there is a
deadlock between competing locks, you can have difficult to diagnose
issues. Queues have their internal locking all done for you.

Thank you for your kind answers!
Best regards, Jens

 

[Steven D'Aprano]

For the record, binary files are thread-safe in Python 3, but text files
are not.

Where is this documented please?

 

[Antoine Pitrou]

Where is this documented please?

In the documentation, of course
http://docs.python.org/3.3/library/io.html#multi-threading

Regards

Antoine.