mailbox.mbox not locking mbox properly

Discussion in 'Python' started by tinnews@isbd.co.uk, Aug 9, 2010.

  1. Guest

    I'm using the python mailbox class in a script that processes incoming
    mail and delivers it to various mbox format mailboxes. It appears
    that, although I am calling the lock method on the destination before
    writing to the mbox and calling unlock afterwards the locking isn't
    working correctly.

    I am reading the mail with mutt, if I deliver the mail direct using
    Postfix (i.e. bypassing my python script) mutt works perfectly and
    shows new mail arriving in the mailbox as expected with an 'N' flag
    beside it. If I deliver the mail with my python script I get an error
    from mutt saying "Mailbox was externally modified. Flags may be
    wrong.", or if I'm reading the mbox via NFS I get all sorts of strange
    errors (though again it works perfectly if Postfix writes to the mbox).

    So it seems that python's mailbox class locking isn't playing nicely
    with mutt's mailbox locking whereas postfix's locking does work
    correctly.

    Has anyone seen this problem before, and/or do I need to anything more
    than the following for the locking to work correctly:-

    #
    #
    # set up the mb for adding the new message, will create if it doesn't exist
    #
    dest = mailbox.mbox(mbName, factory=None)

    dest.lock()
    dest.add(m) # add the new message
    dest.flush()
    dest.unlock()

    Any help/comments most welcome

    --
    Chris Green
     
    , Aug 9, 2010
    #1
    1. Advertising

  2. Guest

    Tim Roberts <> wrote:
    > wrote:
    > >
    > >I'm using the python mailbox class in a script that processes incoming
    > >mail and delivers it to various mbox format mailboxes. It appears
    > >that, although I am calling the lock method on the destination before
    > >writing to the mbox and calling unlock afterwards the locking isn't
    > >working correctly.
    > >...
    > >So it seems that python's mailbox class locking isn't playing nicely
    > >with mutt's mailbox locking whereas postfix's locking does work
    > >correctly.

    >
    > Correct. The "dest.flush()" method creates a temporary file, copies the
    > entire modified mailbox into it, removed the original file, and renames the
    > temp file into place.
    >

    Yes, I just took a look at the mailbox.py code and it does exactly
    that which of course screws up just about any normal MUA looking at
    the mbox. Grrrrrr!


    > The Postfix MDA, like most MDAs, just opens the existing file and appends
    > the new data to it.
    >
    > >Has anyone seen this problem before, and/or do I need to anything more
    > >than the following for the locking to work correctly:-

    >
    > It's not the locking. It's the flush mechanism. The mbox class doesn't
    > know that the ONLY thing you did was an append. You might have modified
    > other messages in the middle. If you want to do an append, you'll need to
    > write your own subclass of mbox.


    OK, thanks. In reality I can probably just use straightforward file
    reading and writing as the *only* thing I will ever be doing is to
    append a message to a mailbox file.

    I think there should be a big warning in the mailbox documentation to
    this effect as doing it the way that Python's mailbox class does it
    will break all sorts of things. There should maybe be a specific
    'append' method.

    --
    Chris Green
     
    , Aug 10, 2010
    #2
    1. Advertising

  3. Chris Rebert Guest

    On Tue, Aug 10, 2010 at 2:01 AM, <> wrote:
    > Tim Roberts <> wrote:
    >> wrote:
    >> >
    >> >I'm using the python mailbox class in a script that processes incoming
    >> >mail and delivers it to various mbox format mailboxes.  It appears
    >> >that, although I am calling the lock method on the destination before
    >> >writing to the mbox and calling unlock afterwards the locking isn't
    >> >working correctly.
    >> >...
    >> >So it seems that python's mailbox class locking isn't playing nicely
    >> >with mutt's mailbox locking whereas postfix's locking does work
    >> >correctly.

    >>
    >> Correct.  The "dest.flush()" method creates a temporary file, copies the
    >> entire modified mailbox into it, removed the original file, and renames the
    >> temp file into place.
    >>

    > Yes, I just took a look at the mailbox.py code and it does exactly
    > that which of course screws up just about any normal MUA looking at
    > the mbox.  Grrrrrr!
    >
    >
    >> The Postfix MDA, like most MDAs, just opens the existing file and appends
    >> the new data to it.
    >>
    >> >Has anyone seen this problem before, and/or do I need to anything more
    >> >than the following for the locking to work correctly:-

    >>
    >> It's not the locking.  It's the flush mechanism.  The mbox class doesn't
    >> know that the ONLY thing you did was an append.  You might have modified
    >> other messages in the middle.  If you want to do an append, you'll need to
    >> write your own subclass of mbox.

    >
    > OK, thanks.  In reality I can probably just use straightforward file
    > reading and writing as the *only* thing I will ever be doing is to
    > append a message to a mailbox file.
    >
    > I think there should be a big warning in the mailbox documentation to
    > this effect as doing it the way that Python's mailbox class does it
    > will break all sorts of things.  There should maybe be a specific
    > 'append' method.


    File a documentation and/or library bug:
    http://bugs.python.org/

    Cheers,
    Chris
     
    Chris Rebert, Aug 10, 2010
    #3
  4. John Nagle Guest

    On 8/10/2010 2:25 AM, Chris Rebert wrote:
    > On Tue, Aug 10, 2010 at 2:01 AM,<> wrote:
    >> Tim Roberts<> wrote:
    >>> wrote:
    >>>>
    >>>> I'm using the python mailbox class in a script that processes incoming
    >>>> mail and delivers it to various mbox format mailboxes. It appears
    >>>> that, although I am calling the lock method on the destination before
    >>>> writing to the mbox and calling unlock afterwards the locking isn't
    >>>> working correctly.
    >>>> ...
    >>>> So it seems that python's mailbox class locking isn't playing nicely
    >>>> with mutt's mailbox locking whereas postfix's locking does work
    >>>> correctly.
    >>>
    >>> Correct. The "dest.flush()" method creates a temporary file, copies the
    >>> entire modified mailbox into it, removed the original file, and renames the
    >>> temp file into place.
    >>>

    >> Yes, I just took a look at the mailbox.py code and it does exactly
    >> that which of course screws up just about any normal MUA looking at
    >> the mbox. Grrrrrr!
    >>
    >>
    >>> The Postfix MDA, like most MDAs, just opens the existing file and appends
    >>> the new data to it.
    >>>
    >>>> Has anyone seen this problem before, and/or do I need to anything more
    >>>> than the following for the locking to work correctly:-
    >>>
    >>> It's not the locking. It's the flush mechanism. The mbox class doesn't
    >>> know that the ONLY thing you did was an append. You might have modified
    >>> other messages in the middle. If you want to do an append, you'll need to
    >>> write your own subclass of mbox.

    >>
    >> OK, thanks. In reality I can probably just use straightforward file
    >> reading and writing as the *only* thing I will ever be doing is to
    >> append a message to a mailbox file.
    >>
    >> I think there should be a big warning in the mailbox documentation to
    >> this effect as doing it the way that Python's mailbox class does it
    >> will break all sorts of things. There should maybe be a specific
    >> 'append' method.

    >
    > File a documentation and/or library bug:
    > http://bugs.python.org/
    >
    > Cheers,
    > Chris
     
    John Nagle, Aug 27, 2010
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    231
  2. Replies:
    18
    Views:
    570
    Matthew Woodcraft
    Apr 28, 2009
  3. Replies:
    2
    Views:
    363
  4. Skye
    Replies:
    1
    Views:
    386
    Dennis Lee Bieber
    Sep 24, 2010
  5. Tim Chase
    Replies:
    0
    Views:
    225
    Tim Chase
    Aug 30, 2012
Loading...

Share This Page