Mail.app Orphans Redux

No fix in over five years.

Get It Try It

Cleaning Mail.app's orphaned 'EMLX' message files off a hard drive can be a treat. Normally people don't see how Mail.app drops orphans all over the place. But that's no excuse for sloppy programming.

People don't normally notice this because people don't normally root around in their ~/Library/Mail folder hive. And because people normally use their mail client as both a turnstile and a permanent library. But Mail.app leaves orphans all over the place. And has been doing so consistently since OS X version 10.4 released April 2005.

What's an orphan? An orphan is a file that's no longer referenced anywhere. It's a file that Mail.app was supposed to delete but got sloppy and didn't. Mail.app orphans are most often found in 'Drafts' folders but they don't have to be - they're sometimes found in 'Sent Messages' as well.

Orphans are a waste of disk space, can be an annoyance, but most crucially can also pose a privacy/security risk. As they're not referenced anywhere, disk cleaning/shredding tools can assume they're legitimate files and should be left alone.

The Mail.app orphans bug rears its head because some pinhead in Cupertino decided to change the way Mail.app works.

Back in the old days, NeXT's Mail.app dutifully reported on the exact contents of each mail folder on disk - it read this information from the hard drive itself. (And you could see the folder path in the title bar.) But that was too easy for some people. So Mail.app of today (of the 'Apple' legacy) runs a database ('Envelope Index') that's supposed to correlate what you see in Mail.app with what's actually on disk. This is an unnecessary step quite common in beginner programs but the Mail.app team might have got away with it - if only their algorithm and code weren't bugged.

[Note this bug can work in reverse too: Mail.app can display listings of messages it doesn't have - it can insist you have messages when the program's accidentally deleted them. Trying to access them will get you nowhere fast. The first solution is to introduce a clean database and to simultaneously remove all messages from the program. Not a lot of fun. The second solution is to use another mail client that's engineered better. Not a lot of fun either.]

This issue has been written about several times previously; it's been submitted to Apple bug reporter at least once; nothing's been done about it in five years.

Oh What A Perfect Day

Today's exercise looked at three different volumes running 10.4, 10.5, and 10.6. This is what was found.

Mail.app - OS X 10.4

7 items, 289497 bytes, 584 blocks, 0 bytes in extended attributes.

Drafts.mbox/Messages/158.emlx
Drafts.mbox/Messages/23.emlx
Drafts.mbox/Messages/159.emlx
Drafts.mbox/Messages/16.emlx
Drafts.mbox/Messages/2.emlx
Drafts.mbox/Messages/43.emlx
Sent Messages.mbox/Messages/15.emlx

Date: Sat, 1 Aug 2009 00:52:16 (GMT)
Date: Sun, 9 Aug 2009 23:46:33 (GMT)
Date: Thu, 03 Sep 2009 17:55:22 (GMT)
Date: Thu, 03 Sep 2009 17:55:22 (GMT)
Date: Fri, 11 Sep 2009 20:25:28 (GMT)
Date: Fri, 11 Sep 2009 23:13:33 (GMT)
Date: Sun, 14 Sep 2009 04:03:20 (GMT)

Mail.app - OS X 10.5

10 items, 936563 bytes, 1872 blocks, 0 bytes in extended attributes.

Drafts.mbox/Messages/127.emlx
Drafts.mbox/Messages/17.emlx
Drafts.mbox/Messages/28.emlx
Drafts.mbox/Messages/5.emlx
Drafts.mbox/Messages/115.emlx
Drafts.mbox/Messages/141.emlx
Drafts.mbox/Messages/2.emlx
Drafts.mbox/Messages/22.emlx
Drafts.mbox/Messages/35.emlx
Drafts.mbox/Messages/39.emlx

Date: Mon, 07 Sep 2009 21:01:30 (GMT)
Date: Mon, 20 Jul 2009 22:14:46 (GMT)
Date: Sat, 22 Aug 2009 19:32:01 (GMT)
Date: Wed, 5 Aug 2009 03:03:08 (GMT)
Date: Mon, 3 Aug 2009 00:31:58 (GMT)
Date: Tue, 8 Sep 2009 03:17:30 (GMT)
Date: Mon, 20 Jul 2009 00:37:51 (GMT)
Date: Wed, 19 Aug 2009 11:05:03 (GMT)
Date: Mon, 17 Aug 2009 04:52:21 (GMT)
Date: Sat, 18 Jul 2009 15:40:01 (GMT)

Mail.app - OS X 10.6

2 items, 8939 bytes, 24 blocks, 0 bytes in extended attributes.

Drafts.mbox/Messages/2.emlx
Drafts.mbox/Messages/58.emlx

Date: Sat, 26 Jun 2010 06:00:04 (GMT)
Date: Fri, 2 Jul 2010 00:18:13 (GMT)

Nineteen (19) Mail.app orphan files were found for a total of 1,234,999 bytes in use or 2,480 blocks for 1,269,760 bytes of actual disk use. All wasted, all inaccessible through Mail.app, all because the Mail,app team have had a sloppy bug for the past five years and still haven't got around to fixing it.

Steve Jobs says he's put OS X on a back burner with a minimal maintenance team. One hopes that team has enough time to take care of elementary, really dumb bugs that waste users' disk space and jeopardise their privacy and security. And maybe Steve Jobs could turn that back burner up a notch.