EiFUB, which means "Everything is Fucked Up Badly".

Sorry about the language...

Simultaneous access to mailboxes

Imagine the following synopsis: your imap4d daemon is working with a mailbox. Part of its data is cached into memory, any modifications to the mailbox are not flushed to disk until user explicitely requires it.

Now, the same user launches another mail client that opens the same mailbox, deletes some messages, and closes it.

The user makes some modifications to the mailbox in imap4d, requests EXPUNGE and closes the mailbox. Imap4d performs flush using the data cached in memory and, voilà - the mailbox is trashed.

You'd say the obvious solution is to lock the mailbox while it is being operating upon. But no, that won't work for imap4d, because appending to the mailbox is quite normal, and it should be allowed. Locking the mailbox will cause mail delivery to be blocked during the client's session (and as a sysadmin, I may say that the number of users that keep their imap session open for 24 hours a day is surprisingly big!)

Of course, the mailbox must be locked while it is being written to, and we do this, but that's another story.

In GNU mailutils we use a function called mailbox_is_updated to determine whether the mailbox was modified recently. The function returns 0 if it was modified and 1 otherwise. For plain UNIX mailboxes a simple approach is used: the function checks if the size of the mailbox file has changed.

It is assumed that if it the new size is greater than previously recorded one, than some new message were appended to the mailbox. If the size is less than the recorded one, then, well... see the quote at the start of this article :)

The method is week since:

• The mailbox can be modified without changing its size (well, yes, it's rather rare, but nevertheless possible);
• Size growth may be produced by editing some message(s) in the middle of the mailbox.

Both cases will result in mailbox corruption.

And finally, the case of mailbox size shrinkage was still not handled in imap4d. Still, because the issue has quite long history: it dates back to April of 2001, when I first joined the Mailutils team.

Here is a short historical essay on the subject:

21 Apr 2001

I noticed that pop3d broke on simultaneous accesses to the mailbox. The story with pop3d is quite different, since it does use mailbox locking (well, that's to say now. Back then it still did not...), anyways:

... what seems to be a real problem is simultaneous access to
the maildrop. All locking/unlocking works just fine, but I have
come across the following situation: two sessions access the same
mailbox simultaneously, both do LIST, then one of them does
DELE on some message, then QUIT. The remaining session does a RETR
and dies miserably, since its internal indices are already out of sync
with the real maildrop contents. The comment on mbox_is_updated()
function addresses this but the function itself is never called.
So I added a call to mailbox_is_updated() to the pop3_mainloop()
function just before the processing of a keyword. Also I have added
ERR_MBOX_SYNC define to help handle the situation when
mbox_is_updated() bails out.
After these changes, the second session just gets the message

      -ERR Mailbox updated by other party or corrupt

and the connection is closed. This behaviour seems to be compatible
with RFC, but maybe it would be better to deny simultaneous
accesses altogether?

23 Apr 2001

Alain agreed to the proposed solution.

30 May 2001

Nice comment on a correctness of the name mailbox_is_updated. It raised a small discussion (see my next posting and Dawe's answer to it), but did not lead to renaming the function :)

As a side note, the story of linguistic inventions in bug-mailutils list is quite amusing... I certainly should catalogue them :^)

10 Nov 2001

In course of an unrelated discussion, Alain reminds that

We are trying to be smart by guessing that if the newsize is larger,
it is because a new mail was ___APPENDED___ to the mailbox so the offsets
maintained by the mailbox_t will still be sane and mailbox_expunge ()
will do the right thing (i.e. seeing that they was new mail when doing
the ftruncate())

13 Mar 2002

An unrelated issue, but still worth mentioning: on some machines disabling mmap speeds up pop3d. Rationale: mailbox_is_updated indirectly calls msync which makes the daemon spend quite some time in uninterruptible sleep.

16 Feb 2005

After a four-year delay I have added to imap4d support similar to that of pop3d.

The approach used by imap4d is a bit different from what we use in pop3d. Essentially, upon selecting a mailbox imap4d does the following:

• Install the MU_EVT_MAILBOX_CORRUPT handler. The handler raises mailbox_corrupt flag when the event is generated.
• Before processing the new command, call imap4d_sync. This function calls mailbox_is_updated and issues RFC2060 untagged responses if the mailbox was modified. Imap4d always did this. The essential change is the following.
• If the mailbox was modified and mailbox_corrupt is set, close the mailbox and open in again. This discards all cached data and reads the new mailbox contents.

The drawback of this approach is that all modifications to the mailbox the user might have made during the imap4 session are discarded. In my opinion it is a reasonable price for preserving the mailbox contents. However, this issue can be addressed too. More on this later.

While solving mailbox shrink problem, the method does not solve the described above weaknesses of mailbox_is_updated.

Correct Diagnosis is 50% of Treatment

To actually solve the issue mailbox_is_updated must be able to tell that the mailbox was modified, and how exactly it was modified, independently of its size. Unfortunately this is easier said than done. Currently mailutils is able to handle various mailbox formats and the way to do the proper diagnosis will vary with the format.

Let's first see what can be done for plain UNIX mailboxes. Obviously, mailbox_is_updated must first check mtime of the mailbox file. If it did not change, the function can return 1 immediately. Otherwise, it should assume some changes were done to the mailbox. Now, the question is whether these changes affect the cached data. The code in mailbox/mbox/mbox.c keeps offsets of UNIX 'From ' lines in the mailbox file. It will suffice to compare these with real offsets to decide whether the mailbox was modified (in the terms of libmailutils corrupted) or a new message was added to it. If any of the registered offsets differ, we scream bloody murder and raise MU_EVT_MAILBOX_CORRUPT event¹⁾ . In any case, we return 0 indicating that the mailbox has been changed.

A similar approach could be used for other mailbox formats.

As a side note, we could also prevent undesirable modifications of the plain UNIX mailbox if we apply kernel exclusive lock on the file from offset 0 up to its current size...