mailbox.mbox.add() sets access time as well as modification time

Discussion in 'Python' started by tinnews@isbd.co.uk, Apr 23, 2009.

  1. Guest

    It seems to me that mailbox.mbox.add() sets the access time of a mbox
    file as well as the modification time. This is not good for MUAs that
    detect new mail by looking to see if the access time is before the
    modification time.

    Have I done something wrong somewhere or is mailbox.mbox.add() really
    as broken as it would appear?

    The snippet of code that adds messages to a mbox is:-

    # previous code works out what destDir is (yes, I know it's not actually a directory)
    dest = mailbox.mbox(destDir, factory=None)
    dest.lock()
    dest.add(m)
    dest.flush()
    dest.unlock()


    --
    Chris Green
     
    , Apr 23, 2009
    #1
    1. Advertising

  2. MRAB Guest

    wrote:
    > It seems to me that mailbox.mbox.add() sets the access time of a mbox
    > file as well as the modification time. This is not good for MUAs that
    > detect new mail by looking to see if the access time is before the
    > modification time.
    >
    > Have I done something wrong somewhere or is mailbox.mbox.add() really
    > as broken as it would appear?
    >

    [snip]
    The access time is the time it was last accessed, ie read or modified.

    The modification time is the time it was last modified.

    The access time can never be before the modification time because it
    must be accessed in order to be modified!
     
    MRAB, Apr 24, 2009
    #2
    1. Advertising

  3. MRAB Guest

    Grant Edwards wrote:
    > On 2009-04-23, MRAB <> wrote:
    >> wrote:
    >>> It seems to me that mailbox.mbox.add() sets the access time of a mbox
    >>> file as well as the modification time. This is not good for MUAs that
    >>> detect new mail by looking to see if the access time is before the
    >>> modification time.
    >>>
    >>> Have I done something wrong somewhere or is mailbox.mbox.add() really
    >>> as broken as it would appear?
    >>>

    >> [snip]
    >> The access time is the time it was last accessed, ie read or modified.

    >
    > Usually.
    >
    >> The modification time is the time it was last modified.

    >
    > Usually.
    >
    >> The access time can never be before the modification time because it
    >> must be accessed in order to be modified!

    >
    > Nonsense. You can set atime and mtime to anything you want.
    >

    Well, yes, you can always change atime and mtime and make it look like
    something happened at a different time to when it actually happened...

    > SOP for writing to to an mbox formatted mailbox is to preserve
    > the atime (changing only the mtime) so that other programs know
    > that that there is "new" mail in the mbox. I know mutt works
    > that way, and I believe that the "you've got new mail" features
    > in some shells work that way. AFAIK, atime<mtime has been the
    > "standard" way to determine when an mbox contains new mail for
    > at least 20 years.
    >

    So atime is used to indicate when it was last read, not last accessed?
    Hmm...

    > Anybody writing to an mbox mailbox has to follow the rules if
    > they expect to interoperate with other mail applications. If
    > mailbox.mbox.add() doesn't preserve the atime when writing to
    > an mbox, then mailbox.mbox.add is broken.
    >
    > http://www.qmail.org/qmail-manual-html/man5/mbox.html
    >
     
    MRAB, Apr 24, 2009
    #3
  4. In message <>, Grant Edwards
    wrote:

    > AFAIK, atime<mtime has been the "standard" way to determine when an mbox
    > contains new mail for at least 20 years.


    Doesn't apply to maildir though, does it?

    Updating atime adds a lot of filesystem overhead; that's why the relatime
    option was invented <http://lwn.net/Articles/244829/>. But better still is
    not to bother.
     
    Lawrence D'Oliveiro, Apr 24, 2009
    #4
  5. Guest

    MRAB <> wrote:
    > wrote:
    > > It seems to me that mailbox.mbox.add() sets the access time of a mbox
    > > file as well as the modification time. This is not good for MUAs that
    > > detect new mail by looking to see if the access time is before the
    > > modification time.
    > >
    > > Have I done something wrong somewhere or is mailbox.mbox.add() really
    > > as broken as it would appear?
    > >

    > [snip]
    > The access time is the time it was last accessed, ie read or modified.
    >
    > The modification time is the time it was last modified.
    >
    > The access time can never be before the modification time because it
    > must be accessed in order to be modified!


    Well in that case very MUA that I have ever come across is broken!
    Access time *can* be before modification time, that's how most MUAs
    work with mbox files.

    --
    Chris Green
     
    , Apr 25, 2009
    #5
  6. Guest

    Grant Edwards <invalid@invalid> wrote:
    > On 2009-04-24, Lawrence D'Oliveiro <_zealand> wrote:
    > > In message <>, Grant Edwards
    > > wrote:
    > >
    > >> AFAIK, atime<mtime has been the "standard" way to determine
    > >> when an mbox contains new mail for at least 20 years.

    > >
    > > Doesn't apply to maildir though, does it?

    >
    > Nope. With maildir, there's a completely separate directory
    > where one puts new messages.
    >
    > > Updating atime adds a lot of filesystem overhead; that's why
    > > the relatime option was invented <http://lwn.net/Articles/244829/>.

    >
    > The relatime should cut down a great deal on system overhead
    > and yet still preserves the atime/mtime semantics used by mbox
    > clients.
    >

    Yes, my filesystems are mounted with relatime set. It's not much use
    if mailbox.mbox.add() doesn't play by the rules though.

    mbox has several advantages over maildir (for me anyway):-

    It allows easy removal of empty mailboxes (in my case by the MUA)

    It's much easier to search

    It's easier to move things around

    It doesn't have silly ways of delimiting directories as do some
    maildir implementations.

    --
    Chris Green
     
    , Apr 25, 2009
    #6
  7. Guest

    Grant Edwards <invalid@invalid> wrote:
    > On 2009-04-24, Grant Edwards <invalid@invalid> wrote:
    >
    > > Anybody writing to an mbox mailbox has to follow the rules if
    > > they expect to interoperate with other mail applications. If
    > > mailbox.mbox.add() doesn't preserve the atime when writing to
    > > an mbox, then mailbox.mbox.add is broken.

    >
    > I should qualify that: since the documentation for add()
    > doesn't specify whether or not it's supposed to add a "new"
    > message or an "old" message, one could argue that either
    > behavior is correct.
    >
    > However, since the maildir add() method adds a "new" message,
    > one would reasonably expect that the mbox add() method do the
    > same. Or, I suppose one might expect the converse: since mbox
    > add() creates an "old" message, then maildir add() should do
    > the same.
    >
    > I have my filesystems mouted with the "noatime" option and use
    > maildir...
    >

    Whatever, if mailbox.mbox.add() doesn't preserve the atime when adding
    messages to an mbox then it's close to useless.

    I suppose I could do the following:-

    lock the mbox
    get the atime
    add the new message with mailbox.mbox.add()
    restore the atime
    unlock the mbox

    All I need to do now is find out how to get and set atime with python.

    --
    Chris Green
     
    , Apr 25, 2009
    #7
  8. MRAB Guest

    wrote:
    > Grant Edwards <invalid@invalid> wrote:
    >> On 2009-04-24, Grant Edwards <invalid@invalid> wrote:
    >>
    >>> Anybody writing to an mbox mailbox has to follow the rules if
    >>> they expect to interoperate with other mail applications. If
    >>> mailbox.mbox.add() doesn't preserve the atime when writing to
    >>> an mbox, then mailbox.mbox.add is broken.

    >> I should qualify that: since the documentation for add()
    >> doesn't specify whether or not it's supposed to add a "new"
    >> message or an "old" message, one could argue that either
    >> behavior is correct.
    >>
    >> However, since the maildir add() method adds a "new" message,
    >> one would reasonably expect that the mbox add() method do the
    >> same. Or, I suppose one might expect the converse: since mbox
    >> add() creates an "old" message, then maildir add() should do
    >> the same.
    >>
    >> I have my filesystems mouted with the "noatime" option and use
    >> maildir...
    >>

    > Whatever, if mailbox.mbox.add() doesn't preserve the atime when adding
    > messages to an mbox then it's close to useless.
    >
    > I suppose I could do the following:-
    >
    > lock the mbox
    > get the atime
    > add the new message with mailbox.mbox.add()
    > restore the atime
    > unlock the mbox
    >
    > All I need to do now is find out how to get and set atime with python.
    >

    os.path.getmtime(path)
    os.path.getatime(path)
    os.utime(path, (atime, mtime))
     
    MRAB, Apr 25, 2009
    #8
  9. In message <49f33d8d$0$516$>,
    wrote:

    > mbox has several advantages over maildir (for me anyway):-
    >
    > It allows easy removal of empty mailboxes (in my case by the MUA)


    Really? I just created a "junk" mail folder via my IMAP server using
    Thunderbird, and was able to delete it the same way. Or did you mean
    something else?

    > It's much easier to search


    Thunderbird seems to search maildir files just fine.

    > It's easier to move things around


    Actually, no. Maildir makes it easier--no need to worry about locking, so it
    works reliably even on network volumes.

    > It doesn't have silly ways of delimiting directories as do some
    > maildir implementations.


    What do you mean?
     
    Lawrence D'Oliveiro, Apr 26, 2009
    #9
  10. In message <>, Grant Edwards
    wrote:

    > ... if one didn't care about backwards-compatiblity with old e-mail
    > apps, then one would use a less broken mailbox format like
    > maildir.


    It's only in the proprietary-software world that we need to worry about
    backward compatibility with old, obsolete software that the vendors cannot
    or will not fix. In the Free Software world, we fix the software to bring it
    up to date.
     
    Lawrence D'Oliveiro, Apr 26, 2009
    #10
  11. Guest

    Lawrence D'Oliveiro <_zealand> wrote:
    > In message <49f33d8d$0$516$>,
    > wrote:
    >
    > > mbox has several advantages over maildir (for me anyway):-
    > >
    > > It allows easy removal of empty mailboxes (in my case by the MUA)

    >
    > Really? I just created a "junk" mail folder via my IMAP server using
    > Thunderbird, and was able to delete it the same way. Or did you mean
    > something else?
    >

    If you're using IMAP then whether the IMAP server uses maildir or mbox
    is (or at least should be) completely invisible to you. I'm referring
    to the situation where my mbox files are in my home directory.

    > > It's much easier to search

    >
    > Thunderbird seems to search maildir files just fine.
    >

    Horrible GUI! I'm running across an ssh connection, that's how I read
    my mail remotely, thunderbird is not much use in that situation.

    > > It's easier to move things around

    >
    > Actually, no. Maildir makes it easier--no need to worry about locking, so it
    > works reliably even on network volumes.
    >

    Locking is nearly irrelevant in a single user situation which is what
    I'm talking about.


    > > It doesn't have silly ways of delimiting directories as do some
    > > maildir implementations.

    >
    > What do you mean?
    >

    Lots of maildir implementations (Courier and its derivatives in
    particular) don't use real directories, they use . delimiters in
    filenames, e.g.:-

    mbox/holidays/2009/France in a real directory structure

    becomes:-

    Maildir/holidays.2009.France a very long filename in the Maildir directory

    I beleive that sometimes it's:-

    Maildir/.holidays.2009.France

    Either way the format with dots in it is a real pain to manage if you
    want to rename/move things.

    --
    Chris Green
     
    , Apr 26, 2009
    #11
  12. Guest

    Grant Edwards <invalid@invalid> wrote:
    > > I suppose I could do the following:-
    > >
    > > lock the mbox
    > > get the atime
    > > add the new message with mailbox.mbox.add()
    > > restore the atime
    > > unlock the mbox

    >
    > You could fix mbox.add(). ;)
    >

    Yes, but I'm not sure that I'm that competant!


    > > All I need to do now is find out how to get and set atime with python.

    >
    > You use os.utime().
    >
    > From http://docs.python.org/library/os.html#module-os:
    >
    > os.utime(path, times)
    >
    > Set the access and modified times of the file specified by
    > path. If times is None, then the file's access and
    > modified times are set to the current time. (The effect is
    > similar to running the Unix program touch on the path.)
    > Otherwise, times must be a 2-tuple of numbers, of the form
    > (atime, mtime) which is used to set the access and modified
    > times, respectively. Whether a directory can be given for
    > path depends on whether the operating system implements
    > directories as files (for example, Windows does not). Note
    > that the exact times you set here may not be returned by a
    > subsequent stat() call, depending on the resolution with
    > which your operating system records access and modification
    > times; see stat().
    >
    > Changed in version 2.0: Added support for None for times.
    >
    > Availability: Unix, Windows.
    >

    Thanks!

    --
    Chris Green
     
    , Apr 26, 2009
    #12
  13. Aahz Guest

    In article <gt0f2d$t52$>,
    Lawrence D'Oliveiro <_zealand> wrote:
    >
    >It's only in the proprietary-software world that we need to worry about
    >backward compatibility with old, obsolete software that the vendors
    >cannot or will not fix. In the Free Software world, we fix the software
    >to bring it up to date.


    Are you volunteering to maintain trn3.6?
    --
    Aahz () <*> http://www.pythoncraft.com/

    "If you think it's expensive to hire a professional to do the job, wait
    until you hire an amateur." --Red Adair
     
    Aahz, Apr 27, 2009
    #13
  14. In message <gt3a7k$8ds$>, Aahz wrote:

    > In article <gt0f2d$t52$>,
    > Lawrence D'Oliveiro <_zealand> wrote:
    >>
    >>It's only in the proprietary-software world that we need to worry about
    >>backward compatibility with old, obsolete software that the vendors
    >>cannot or will not fix. In the Free Software world, we fix the software
    >>to bring it up to date.

    >
    > Are you volunteering to maintain trn3.6?


    Either there are enough people using it to care about it, in which case
    somebody in the community will fix it, it or there are not, in which case
    it's not worth bothering with.
     
    Lawrence D'Oliveiro, Apr 28, 2009
    #14
  15. In message <>, Grant
    Edwards wrote:

    > On 2009-04-26, Lawrence D'Oliveiro <_zealand>
    > wrote:
    >
    >> In message <>, Grant Edwards
    >> wrote:
    >>
    >>> ... if one didn't care about backwards-compatiblity with old e-mail
    >>> apps, then one would use a less broken mailbox format like
    >>> maildir.

    >>
    >> It's only in the proprietary-software world that we need to worry about
    >> backward compatibility with old, obsolete software that the vendors
    >> cannot or will not fix. In the Free Software world, we fix the software
    >> to bring it up to date.

    >
    > Who's "we"? Are you volunteering to fix all of the MUAs and
    > MTAs out there that have mbox code in them that do follow the
    > rules to make them compatible with _one_ broken library module?


    All the MUAs and MTAs I'm aware of that are worth bothering about have the
    option to support maildir format these days.
     
    Lawrence D'Oliveiro, Apr 28, 2009
    #15
  16. Re: mailbox.mbox.add() sets access time as well as modificationtime

    On Tue, 28 Apr 2009 11:19:51 +1200, Lawrence D'Oliveiro wrote:

    > In message <gt3a7k$8ds$>, Aahz wrote:
    >
    >> In article <gt0f2d$t52$>, Lawrence D'Oliveiro
    >> <_zealand> wrote:
    >>>
    >>>It's only in the proprietary-software world that we need to worry about
    >>>backward compatibility with old, obsolete software that the vendors
    >>>cannot or will not fix. In the Free Software world, we fix the software
    >>>to bring it up to date.

    >>
    >> Are you volunteering to maintain trn3.6?

    >
    > Either there are enough people using it to care about it, in which case
    > somebody in the community will fix it, it or there are not, in which
    > case it's not worth bothering with.



    That's EXACTLY the same as the proprietary software world. Either there
    are enough people using it to care about it, in which case the vendor
    will fix it, it or there are not, in which case it's not worth bothering
    with.

    Or, to put it another way... if you are one of those people who *need* to
    worry about backward compatibility with old, obsolete software, for
    whatever reason you have, then neither the proprietary or free software
    worlds are particularly good to you.

    The advantage to open source software in this case is that if you can
    maintain a backwards compatible fork, then you are permitted to do so. If
    you can't do it yourself (and that includes paying somebody to do it for
    you), then you're just as much out of luck as somebody who wants
    Microsoft to continue supporting Windows 3.1.



    --
    Steven
     
    Steven D'Aprano, Apr 28, 2009
    #16
  17. In message <>, Steven
    D'Aprano wrote:

    > On Tue, 28 Apr 2009 11:19:51 +1200, Lawrence D'Oliveiro wrote:
    >
    >> Either there are enough people using it to care about it, in which case
    >> somebody in the community will fix it, it or there are not, in which
    >> case it's not worth bothering with.

    >
    > That's EXACTLY the same as the proprietary software world.


    No it's not. A proprietary product can have lots of users who care about it,
    yet the vendor can't be bothered keeping it up to date any more. For
    instance, it was estimated that there were 40 million people still using
    Windows 98 when Microsoft announced an end to support.

    You don't get that with open-source products.
     
    Lawrence D'Oliveiro, Apr 28, 2009
    #17
  18. Guest

    Lawrence D'Oliveiro <_zealand> wrote:
    > In message <>, Grant
    > Edwards wrote:
    >
    > > On 2009-04-26, Lawrence D'Oliveiro <_zealand>
    > > wrote:
    > >
    > >> In message <>, Grant Edwards
    > >> wrote:
    > >>
    > >>> ... if one didn't care about backwards-compatiblity with old e-mail
    > >>> apps, then one would use a less broken mailbox format like
    > >>> maildir.
    > >>
    > >> It's only in the proprietary-software world that we need to worry about
    > >> backward compatibility with old, obsolete software that the vendors
    > >> cannot or will not fix. In the Free Software world, we fix the software
    > >> to bring it up to date.

    > >
    > > Who's "we"? Are you volunteering to fix all of the MUAs and
    > > MTAs out there that have mbox code in them that do follow the
    > > rules to make them compatible with _one_ broken library module?

    >
    > All the MUAs and MTAs I'm aware of that are worth bothering about have the
    > option to support maildir format these days.
    >

    Yes, but as I explained earlier in this thread there are reasons why
    one might want to stay with mbox. I use mutt which can quite happily
    cope with either mbox or maildir (and some other formats), it can even
    work with a mix of mailbox types.

    I've used mutt for several years now and have tried to move to maildir
    more than once and have always returned to mbox because the
    disadvantages of maildir outweigh the benefits (for me). Currently I
    run mutt on a remote server where I have to use maildir because their
    file systems are mounted noatime. I am moving to reading mail on my
    own Linux box just because I want to get back to mbox, this python
    issue is about the only thing I have to overcome before I have what I
    want.

    --
    Chris Green
     
    , Apr 28, 2009
    #18
  19. Chris Green wrote:
    > Currently I run mutt on a remote server where I have to use maildir
    > because their file systems are mounted noatime. I am moving to reading
    > mail on my own Linux box just because I want to get back to mbox, [...]


    For what it's worth, setting 'check_mbox_size = yes' is usually good
    enough in practice to make Mutt happy with mbox on a noatime filesystem.

    -M-
     
    Matthew Woodcraft, Apr 28, 2009
    #19
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    234
  2. Replies:
    2
    Views:
    365
  3. Replies:
    3
    Views:
    587
    John Nagle
    Aug 27, 2010
  4. Skye
    Replies:
    1
    Views:
    394
    Dennis Lee Bieber
    Sep 24, 2010
  5. Tim Chase
    Replies:
    0
    Views:
    227
    Tim Chase
    Aug 30, 2012
Loading...

Share This Page