mailbox.mbox.add() sets access time as well as modification time

[tinnews]

It seems to me that mailbox.mbox.add() sets the access time of a mbox
file as well as the modification time. This is not good for MUAs that
detect new mail by looking to see if the access time is before the
modification time.

Have I done something wrong somewhere or is mailbox.mbox.add() really
as broken as it would appear?

The snippet of code that adds messages to a mbox is:-

# previous code works out what destDir is (yes, I know it's not actually a directory)
dest = mailbox.mbox(destDir, factory=None)
dest.lock()
dest.add(m)
dest.flush()
dest.unlock()

 

[MRAB]

It seems to me that mailbox.mbox.add() sets the access time of a mbox
file as well as the modification time. This is not good for MUAs that
detect new mail by looking to see if the access time is before the
modification time.

Have I done something wrong somewhere or is mailbox.mbox.add() really
as broken as it would appear?

[snip]
The access time is the time it was last accessed, ie read or modified.

The modification time is the time it was last modified.

The access time can never be before the modification time because it
must be accessed in order to be modified!

 

[MRAB]

It seems to me that mailbox.mbox.add() sets the access time of a mbox
file as well as the modification time. This is not good for MUAs that
detect new mail by looking to see if the access time is before the
modification time.

Have I done something wrong somewhere or is mailbox.mbox.add() really
as broken as it would appear?

[snip]
The access time is the time it was last accessed, ie read or modified.
Usually.

The modification time is the time it was last modified.
Usually.

The access time can never be before the modification time because it
must be accessed in order to be modified!

Nonsense. You can set atime and mtime to anything you want.

Well, yes, you can always change atime and mtime and make it look like
something happened at a different time to when it actually happened...

SOP for writing to to an mbox formatted mailbox is to preserve
the atime (changing only the mtime) so that other programs know
that that there is "new" mail in the mbox. I know mutt works
that way, and I believe that the "you've got new mail" features
in some shells work that way. AFAIK, atime<mtime has been the
"standard" way to determine when an mbox contains new mail for
at least 20 years.

So atime is used to indicate when it was last read, not last accessed?
Hmm...

 

[Lawrence D'Oliveiro]

AFAIK, atime<mtime has been the "standard" way to determine when an mbox
contains new mail for at least 20 years.

Doesn't apply to maildir though, does it?

Updating atime adds a lot of filesystem overhead; that's why the relatime
option was invented <http://lwn.net/Articles/244829/>. But better still is
not to bother.

 

[tinnews]

It seems to me that mailbox.mbox.add() sets the access time of a mbox
file as well as the modification time. This is not good for MUAs that
detect new mail by looking to see if the access time is before the
modification time.

Have I done something wrong somewhere or is mailbox.mbox.add() really
as broken as it would appear?

[snip]
The access time is the time it was last accessed, ie read or modified.

The modification time is the time it was last modified.

The access time can never be before the modification time because it
must be accessed in order to be modified!

Well in that case very MUA that I have ever come across is broken!
Access time *can* be before modification time, that's how most MUAs
work with mbox files.

 

[tinnews]

Nope. With maildir, there's a completely separate directory
where one puts new messages.


The relatime should cut down a great deal on system overhead
and yet still preserves the atime/mtime semantics used by mbox
clients.

Yes, my filesystems are mounted with relatime set. It's not much use
if mailbox.mbox.add() doesn't play by the rules though.

mbox has several advantages over maildir (for me anyway):-

It allows easy removal of empty mailboxes (in my case by the MUA)

It's much easier to search

It's easier to move things around

It doesn't have silly ways of delimiting directories as do some
maildir implementations.

 

[tinnews]

I should qualify that: since the documentation for add()
doesn't specify whether or not it's supposed to add a "new"
message or an "old" message, one could argue that either
behavior is correct.

However, since the maildir add() method adds a "new" message,
one would reasonably expect that the mbox add() method do the
same. Or, I suppose one might expect the converse: since mbox
add() creates an "old" message, then maildir add() should do
the same.

I have my filesystems mouted with the "noatime" option and use
maildir...

Whatever, if mailbox.mbox.add() doesn't preserve the atime when adding
messages to an mbox then it's close to useless.

I suppose I could do the following:-

lock the mbox
get the atime
add the new message with mailbox.mbox.add()
restore the atime
unlock the mbox

All I need to do now is find out how to get and set atime with python.

 

[MRAB]

Whatever, if mailbox.mbox.add() doesn't preserve the atime when adding
messages to an mbox then it's close to useless.

I suppose I could do the following:-

lock the mbox
get the atime
add the new message with mailbox.mbox.add()
restore the atime
unlock the mbox

All I need to do now is find out how to get and set atime with python.

os.path.getmtime(path)
os.path.getatime(path)
os.utime(path, (atime, mtime))

 

[Lawrence D'Oliveiro]

mbox has several advantages over maildir (for me anyway):-

It allows easy removal of empty mailboxes (in my case by the MUA)

Really? I just created a "junk" mail folder via my IMAP server using
Thunderbird, and was able to delete it the same way. Or did you mean
something else?

It's much easier to search

Thunderbird seems to search maildir files just fine.

It's easier to move things around

Actually, no. Maildir makes it easier--no need to worry about locking, so it
works reliably even on network volumes.

It doesn't have silly ways of delimiting directories as do some
maildir implementations.

What do you mean?

 

[Lawrence D'Oliveiro]

... if one didn't care about backwards-compatiblity with old e-mail
apps, then one would use a less broken mailbox format like
maildir.

It's only in the proprietary-software world that we need to worry about
backward compatibility with old, obsolete software that the vendors cannot
or will not fix. In the Free Software world, we fix the software to bring it
up to date.

 

[tinnews]

Really? I just created a "junk" mail folder via my IMAP server using
Thunderbird, and was able to delete it the same way. Or did you mean
something else?

If you're using IMAP then whether the IMAP server uses maildir or mbox
is (or at least should be) completely invisible to you. I'm referring
to the situation where my mbox files are in my home directory.

Thunderbird seems to search maildir files just fine.

Horrible GUI! I'm running across an ssh connection, that's how I read
my mail remotely, thunderbird is not much use in that situation.

Actually, no. Maildir makes it easier--no need to worry about locking, so it
works reliably even on network volumes.

Locking is nearly irrelevant in a single user situation which is what
I'm talking about.

What do you mean?

Lots of maildir implementations (Courier and its derivatives in
particular) don't use real directories, they use . delimiters in
filenames, e.g.:-

mbox/holidays/2009/France in a real directory structure

becomes:-

Maildir/holidays.2009.France a very long filename in the Maildir directory

I beleive that sometimes it's:-

Maildir/.holidays.2009.France

Either way the format with dots in it is a real pain to manage if you
want to rename/move things.

 

[tinnews]

You could fix mbox.add().

Yes, but I'm not sure that I'm that competant!

You use os.utime().

From http://docs.python.org/library/os.html#module-os:

os.utime(path, times)

Set the access and modified times of the file specified by
path. If times is None, then the file's access and
modified times are set to the current time. (The effect is
similar to running the Unix program touch on the path.)
Otherwise, times must be a 2-tuple of numbers, of the form
(atime, mtime) which is used to set the access and modified
times, respectively. Whether a directory can be given for
path depends on whether the operating system implements
directories as files (for example, Windows does not). Note
that the exact times you set here may not be returned by a
subsequent stat() call, depending on the resolution with
which your operating system records access and modification
times; see stat().

Changed in version 2.0: Added support for None for times.

Availability: Unix, Windows.

Thanks!

 

[Aahz]

It's only in the proprietary-software world that we need to worry about
backward compatibility with old, obsolete software that the vendors
cannot or will not fix. In the Free Software world, we fix the software
to bring it up to date.

Are you volunteering to maintain trn3.6?

 

[Lawrence D'Oliveiro]

Are you volunteering to maintain trn3.6?

Either there are enough people using it to care about it, in which case
somebody in the community will fix it, it or there are not, in which case
it's not worth bothering with.

 

[Lawrence D'Oliveiro]

Who's "we"? Are you volunteering to fix all of the MUAs and
MTAs out there that have mbox code in them that do follow the
rules to make them compatible with _one_ broken library module?

All the MUAs and MTAs I'm aware of that are worth bothering about have the
option to support maildir format these days.

 

[Steven D'Aprano]

Either there are enough people using it to care about it, in which case
somebody in the community will fix it, it or there are not, in which
case it's not worth bothering with.

That's EXACTLY the same as the proprietary software world. Either there
are enough people using it to care about it, in which case the vendor
will fix it, it or there are not, in which case it's not worth bothering
with.

Or, to put it another way... if you are one of those people who *need* to
worry about backward compatibility with old, obsolete software, for
whatever reason you have, then neither the proprietary or free software
worlds are particularly good to you.

The advantage to open source software in this case is that if you can
maintain a backwards compatible fork, then you are permitted to do so. If
you can't do it yourself (and that includes paying somebody to do it for
you), then you're just as much out of luck as somebody who wants
Microsoft to continue supporting Windows 3.1.

 

[Lawrence D'Oliveiro]

That's EXACTLY the same as the proprietary software world.

No it's not. A proprietary product can have lots of users who care about it,
yet the vendor can't be bothered keeping it up to date any more. For
instance, it was estimated that there were 40 million people still using
Windows 98 when Microsoft announced an end to support.

You don't get that with open-source products.

 

[tinnews]

All the MUAs and MTAs I'm aware of that are worth bothering about have the
option to support maildir format these days.

Yes, but as I explained earlier in this thread there are reasons why
one might want to stay with mbox. I use mutt which can quite happily
cope with either mbox or maildir (and some other formats), it can even
work with a mix of mailbox types.

I've used mutt for several years now and have tried to move to maildir
more than once and have always returned to mbox because the
disadvantages of maildir outweigh the benefits (for me). Currently I
run mutt on a remote server where I have to use maildir because their
file systems are mounted noatime. I am moving to reading mail on my
own Linux box just because I want to get back to mbox, this python
issue is about the only thing I have to overcome before I have what I
want.

 

[Matthew Woodcraft]

Currently I run mutt on a remote server where I have to use maildir
because their file systems are mounted noatime. I am moving to reading
mail on my own Linux box just because I want to get back to mbox, [...]

For what it's worth, setting 'check_mbox_size = yes' is usually good
enough in practice to make Mutt happy with mbox on a noatime filesystem.

-M-