Apple Mail's Amnesia

For as long as many can remember.

Get It Try It

Apple's Mail.app has a very dangerous propensity for not removing messages the user assumes are long gone. There is a lot of buggy code in Mail.app and it's therefore not surprising things like this happen. But that's little consolation for the user whose files get compromised, whose confidential data gets stolen.

It's one thing to forget to delete (shred) sensitive data but it's quite another to go through these motions blissfully unaware that Apple's offering for the #2 killer app isn't holding its end up.

If a user deletes a message then it's supposed to be gone. It's supposed to first go to the trash. And from the trash a simple command (shortcut ⇧⌘⌫ on Snow Leopard) is supposed to see everything's gone. And at that point the user can shred disk free space and feel reasonably safe.



But that doesn't hold with Apple's Mail.app for Snow Leopard. To be perfectly frank it didn't hold for Apple's Mail.app for the previous release Leopard either. Some things never get fixed.

Do like this to test.

1. Remove everything in your Drafts folder to the trash.
2. Use the keyboard shortcut ⇧⌘⌫ to empty the trash.

Now open Terminal.app and run the following command

$ find ~/Library/Mail -name *\.emlx -type f | grep Drafts\.mbox

What you'll get back is a list of all the 'orphaned' messages Mail.app claims are no longer there.

/Users/macuser/Library/Mail/.../Drafts.mbox/Messages/7.emlx
/Users/macuser/Library/Mail/.../Drafts.mbox/Messages/2.emlx
/Users/macuser/Library/Mail/.../Drafts.mbox/Messages/26.emlx

Your mileage may vary on this one - it all depends on what you've done and how long you've done it. But tests show this behaviour is reliable - Mail.app slips up on this one right as rain all the time.

What's Going On?

The old NeXT Mail.app used a far out idea: it told you what messages were available by (gasp) checking on disk. Back in those simpler days there was one file per account (as still used on servers today). That Mail.app was smart enough to parse the files, show you what was in them, and even show you what directory you were in at the top of the window in the title bar.

That was then. Today Mail.app uses Envelope Index and other files to keep track of what's on disk. Obviously this method of using second hand information is easily susceptible to flaws.

Compound this with the irrepressible urge to hide the drafts folder if at all possible and one sees the train wreck coming from far off.

Most of the lost messages come from Mail.app's autosave function.

Mime-Version: 1.0 (Apple Message framework v1077)
Content-Type: text/html;
    charset=us-ascii
X-Apple-Auto-Saved: 1
X-Apple-Windows-Friendly: 1
Content-Transfer-Encoding: quoted-printable
X-Uniform-Type-Identifier: com.apple.mail-draft

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>date-sent</key>
    <real>1254134720</real>
    <key>flags</key>
    <integer>33415745</integer>
    <key>subject</key>
    <string></string>
</dict>
</plist>

Several things in the above excerpt from an 'orphan' are indicative.

• 'X-Apple-Auto-Saved: 1'. Mail.app seems to treat autosaves differently from explicit saves.
• '<key>date-sent</key>'. Mail.app is aware the autosave was sent and should now be removed.

Except it doesn't. Now kick in part two of this farce: as Mail.app no longer goes to disk to determine what messages are available, the orphaned autosaved messages don't turn up anywhere. Unless you go to a command line and look for yourself.

According to Mail.app these messages don't exist.

radr://?

This bug has not been reported again for the simple reason (and others) Apple never respond with fixes. It was first reported years ago; if they haven't fixed it by now then they simply don't care. They're too busy creating new Automator actions.

But you might need to care - you might have sensitive data on your computer. And be under strict orders to shred mail you're no longer working with. And in such case you'll have to go to the command line regularly to remove the orphans, at which time you can shred your disk free space as usual.

See Also
Coldspots: Snow Leopard's Windows Executables