Writing to same file from two threads

Discussion in 'Python' started by Jens Thoms Toerring, Feb 26, 2013.

  1. Hi,

    I noticed in someone elses program that it writes single
    lines to the same file from (what I call for loss of a better
    name) the "main thread" of the program and from a thread sub-
    sequentally started. This got me worried if it might result
    in garbled output (i.e. having some output from A inside a
    line written by B or vice versae) because the "main thread" or
    the other thread could be interrupted during a call of write().
    Is this a valid concern (and thus locking the file object is
    required before writing to it) or am I guaranteed that this
    can't happen? In the latter case I would be grateful for an
    explanation what mechanism is responsible for this never to
    happen.
    Thanks and best regards, Jens

    PS: I already have determined experimentally that a context
    switch definitely can happen between two calls of write()
    (and I expected nothing else), what I'm worried about are
    context switches somewhere within the very innards of what
    write() does.
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Feb 26, 2013
    #1
    1. Advertising

  2. Jens Thoms Toerring

    Paul Rubin Guest

    (Jens Thoms Toerring) writes:
    > in garbled output (i.e. having some output from A inside a
    > line written by B or vice versae) because the "main thread" or


    Yes they do get garbled like that. Preferred Python style is put a
    single thread in charge of all the i/o to that file, and communicate
    with it by message passing through Queue objects. That is safer than
    directly using locks.
     
    Paul Rubin, Feb 26, 2013
    #2
    1. Advertising

  3. Paul Rubin <> wrote:
    > (Jens Thoms Toerring) writes:
    > > in garbled output (i.e. having some output from A inside a
    > > line written by B or vice versae) because the "main thread" or


    > Yes they do get garbled like that. Preferred Python style is put a
    > single thread in charge of all the i/o to that file, and communicate
    > with it by message passing through Queue objects. That is safer than
    > directly using locks.


    Thank you for confirmig my suspicion;-) But you have induced
    another question: why is using a Queue safer than locking (not
    that I doubt that it might be more elegant etc.). Is it "safer"
    because it's less likely that one gets it wrong (e.g. by for-
    grtting to acquire the lock) or is there something inherently
    unsafe about locks?

    Thank you and best regards, Jens
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Feb 26, 2013
    #3
  4. Jens Thoms Toerring <jt <at> toerring.de> writes:
    >
    > Paul Rubin <no.email <at> nospam.invalid> wrote:
    > > jt <at> toerring.de (Jens Thoms Toerring) writes:
    > > > in garbled output (i.e. having some output from A inside a
    > > > line written by B or vice versae) because the "main thread" or

    >
    > > Yes they do get garbled like that. Preferred Python style is put a
    > > single thread in charge of all the i/o to that file, and communicate
    > > with it by message passing through Queue objects. That is safer than
    > > directly using locks.

    >
    > Thank you for confirmig my suspicion But you have induced
    > another question: why is using a Queue safer than locking (not
    > that I doubt that it might be more elegant etc.). Is it "safer"
    > because it's less likely that one gets it wrong (e.g. by for-
    > grtting to acquire the lock) or is there something inherently
    > unsafe about locks?


    For the record, binary files are thread-safe in Python 3, but text files
    are not.
    Locks are safe if you use them well. As you point out, if you forget
    to acquire your lock, or if you devise a situation where there is a
    deadlock between competing locks, you can have difficult to diagnose
    issues. Queues have their internal locking all done for you.

    Regards

    Antoine.
     
    Antoine Pitrou, Feb 27, 2013
    #4
  5. Antoine Pitrou <> wrote:
    > Jens Thoms Toerring <jt <at> toerring.de> writes:
    > >
    > > Paul Rubin <no.email <at> nospam.invalid> wrote:
    > > > jt <at> toerring.de (Jens Thoms Toerring) writes:
    > > > > in garbled output (i.e. having some output from A inside a
    > > > > line written by B or vice versae) because the "main thread" or

    > >
    > > > Yes they do get garbled like that. Preferred Python style is put a
    > > > single thread in charge of all the i/o to that file, and communicate
    > > > with it by message passing through Queue objects. That is safer than
    > > > directly using locks.

    > >
    > > Thank you for confirmig my suspicion But you have induced
    > > another question: why is using a Queue safer than locking (not
    > > that I doubt that it might be more elegant etc.). Is it "safer"
    > > because it's less likely that one gets it wrong (e.g. by for-
    > > grtting to acquire the lock) or is there something inherently
    > > unsafe about locks?


    > For the record, binary files are thread-safe in Python 3, but text files
    > are not.
    > Locks are safe if you use them well. As you point out, if you forget
    > to acquire your lock, or if you devise a situation where there is a
    > deadlock between competing locks, you can have difficult to diagnose
    > issues. Queues have their internal locking all done for you.


    Thank you for your kind answers!
    Best regards, Jens
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Feb 27, 2013
    #5
  6. On Wed, 27 Feb 2013 13:26:18 +0000, Antoine Pitrou wrote:

    > For the record, binary files are thread-safe in Python 3, but text files
    > are not.


    Where is this documented please?



    --
    Steven
     
    Steven D'Aprano, Feb 28, 2013
    #6
  7. Steven D'Aprano <steve+comp.lang.python <at> pearwood.info> writes:
    >
    > On Wed, 27 Feb 2013 13:26:18 +0000, Antoine Pitrou wrote:
    >
    > > For the record, binary files are thread-safe in Python 3, but text files
    > > are not.

    >
    > Where is this documented please?


    In the documentation, of course ;)
    http://docs.python.org/3.3/library/io.html#multi-threading

    Regards

    Antoine.
     
    Antoine Pitrou, Mar 1, 2013
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. HNguyen
    Replies:
    4
    Views:
    2,421
    HNguyen
    Dec 21, 2004
  2. Shawn
    Replies:
    2
    Views:
    383
    =?Utf-8?B?RWx0b24gVw==?=
    Feb 19, 2006
  3. darrel
    Replies:
    2
    Views:
    478
    darrel
    Apr 5, 2006
  4. Sona
    Replies:
    1
    Views:
    431
    Gordon Beaton
    Sep 28, 2003
  5. id10t error
    Replies:
    3
    Views:
    739
    Alexey Smirnov
    Aug 31, 2009
Loading...

Share This Page