About | ACP | Buy | Forum | Industry Watch | Learning Curve | Search | Twitter | Xnews
Home » Learning Curve » Developers Workshop

Disastrous Design Decisions

Everybody makes them. The wiser know to back up.

Buy It

Try It

BILL GRAHAM CIVIC AUDITORIUM (Rixstep) — It was like Neil Armstrong setting a first foot on the moon. But it was Apple once again leaving the surface of planet Earth. Pundits will continue to write for days to come (some longer) but Apple under Tim Cook's leadership once again proved that Apple founder Steve Jobs is very much alive. And the competitors again have a lot of catching up to do.

But one product, formerly a flagship and once the only ship, didn't get the limelight. Apple announced the release of OS X 10.11 at the end of this month, but that didn't warrant premium stage time.

OS X to Apple is not quite what Flash has been to Adobe: everyone everywhere wants Flash dead, but Apple and many others need OS X to keep building those cool gadgets running Apple's cool UIs.

It took Apple five years to get a polished OPENSTEP out the door in 2002 as 10.2 'Jag-wire', then only a few years later, Scott Forstall was roaming through the landscape looking for promising developers for the iPhone project, this per his talk at Stanford afterwards. iPhone, then iPad, then Apple Watch, and now a new Apple TV. But what, if anything, did they leave behind?

The Fellowship of the Loop

Macworld published a 'how-to' the other day.


The details were taken from a post at Apple's discussion forums about Mavericks over a year and a half earlier.


The problem to solve is that when creating files like this:

You can end up - on both Mavericks and Yosemite - getting alerts like this:

'The document XXX.xxx could not be saved/exported as XYX.yxy.'

No explanation is given, either by Linc Davis at the Apple discussion forum or by Macworld, as to why this works, or what's in play, or how it happens. The solution seems nonsensical, and it brings distant stirring - and haunting - things back again.

There's a long history to this, dating quite a way back. Its genesis actually started when a dweeb inside or close to Cupertino decided it was too much of a hassle keeping track of one's own file permissions. Most of the time when a file is not writeable in one's own home area, that's because you the user made it so. (After all, it's only you, the owner, that can change permissions?) But to be reminded of this by Cocoa's NSDocumentController was, however, too much for the poor sod. And so begins the slip and slide down the road to perdition.

You see, it is actually possible to overwrite files that are not writeable - especially if you take Apple's (NeXT's) method for more secure writes. The original method, used in early versions of OS X, was to let the document controller write first to a neutral path somewhere else. This can be seen in the Cocoa APIs provided to the client: the path is passed to the client and the client doesn't actually deal with the path at all, but merely passes it on. The document controller's already decided the location of this file, and the client needs only to provide the data.

The reasoning seems to have been that writing to a separate location can prevent mishaps through blackouts and brownouts. The original file isn't touched until a secure write to that separate location is established. After that, the system can keep writing to the actual destination until the copy can complete and be verified.

Good enough.

But the operative word above is 'copy': the file at that separate location is copied to the actual destination. It is not, for example, moved. That would be just dumb. The file's inode is left intact because the file already exists. The write is repeated until it works. The document controller has a copy of the file at that separate location and can keep using it until the write succeeds.

Good enough.

Now enter Mr Pinhead from above, the dimwit who was perplexed when he got those alarming alerts that his own file doesn't allow writes. This must of course be the rule in areas where Finder doesn't want you to go, but in your own home area, the main reason files are not marked as writeable is you, Mr Pinhead, you stupid oaf.

Never mind. Mr Pinhead wanted that changed pronto, and so the concerted and combined forces of Cupertino's 'next to best' were called in to find a solution. And this is the solution they found:

Just move that temporary file instead of copying it.

Then, if you run into permissions trouble, you can issue another alert instead. Like this.

Never mind that Unix doesn't have anything like a 'locked file' - this is solely an Apple concoction for Mr Pinhead.

Moving a file destroys the target file first, then copies in, as a two-part operation. The original inode is lost. The copy part of the operation should still be verified of course, but...

There are two things that uniquely identify a file on a Unix system.

  1. Device number
  2. Inode

A Unix file system, as presented to a user, can actually spread over several sovereign file systems. Apple's Finder will normally freak out when encountering such things, but more mature file managers can find them. And each such file system has a unique associated device number.

Each device has its own ilist - the array of iblocks for all resident files. The inodes (indexes) are of course also unique.

Naturally, as one file on one system can happen to have the same inode as a different file on a different system... So it's the two numbers together which uniquely identify a file anywhere.

Destroying a file to merely overwrite it is nuts - it's not Unix. This means that all attempts to track the file will fail. And why? Because the file, once overwritten by first stupidly destroying it and creating it again, is no longer the same file!

Which seems to defeat all adult purpose. But there is one advantage, at least for the less cerebrally fortunate computer users. And this has to do with how Unix structures its directories.

Contrary to the way most directory structures are created, Unix directories are first and foremost files like any other. There are distinctions made today in latter versions, but the distinctions are more semantic than technical. Directories are merely streams of bytes like any other files, as some of Brian Kernighan's earliest tutorials took pains to point out. Back in those days, file names were limited to 14 bytes, with 2 bytes used for the inode. Storage capacity has increased since then, and we have a more dynamic system as seen in GDE, but the idea is the same.

Directories contain only two items of essential information:

  1. The file name
  2. The file's inode (on the same device)

It's the iblock corresponding to the inode (index) in the ilist that has all the info on the file (save its name - important later on). It's here you'll find everything about the file (save its name). It's here you find its size, where it's located on the storage device, its ownership, its mode bits, and so on. The ACP Info sheet lists all this data for all Xfile System applications.

But here's the catch: directories are files too. Directories have read and write bits too, like ordinary files. Directories can be marked so they can't be read (true). Directories can be marked so they can't be 'executed' (so it's impossible to 'cd' into them). And directories can be marked so they can't be written.

So if you have a directory that's 'read only', how do you then save a file with Mr Pinhead's new system? You don't. You can't. That this hasn't gone and whacked the people behind this accommodation for Mr Pinhead is a wonder. For it'll do that all the time.

The trouble - pay attention now, Mr Pinhead - is that in the old days when you got an alert that you didn't have the proper permissions, the message you got made sense - by stating that you didn't have proper permissions, the system was trying to tell you (wait for it) that you didn't have proper permissions. But today when you get this:

WTF is that supposed to mean? Can the pitiable Finder even adjust directory permissions? Worse still: the alert doesn't say anything about the blasted directory, but instead about the item, which in this case is clearly denoted as 'Mr Pinhead's Private Porn Stash'. But poor Mr Pinhead! For if he tries to look at the permissions for that file (and the whole idea was he wouldn't have to deal with alerts like that anymore, poor thing) then he won't find zip - the permissions for the file are likely totally OK - it's the directory that's been hosed! And the genius alert from Apple, the same as before, says nothing about it: it misleads. And you the user, poor you, are up that classic creek without a paddle, and in fact aren't even told what kind of paddle you need.

So much for Mr Pinhead solutions. But this is an indication of where OS X is today and what's become of it. And yet this is only the first part.

The Two Towers

Yes, things get worse. We began to see this with 10.7 Lion. And it's amazing actually. For at the same time, we saw those poor Microsofties falling flat on their fat noses time after time as they tried to port their beloved Windows to every platform and device on the planet. The people at Apple, far more intelligent, understood that they needed a new UI and a new API for every device! So after OS X came what today is known as iOS, and then the watch got its own system, now there's tvOS, and so forth. Each of those devices works a different way. People use them differently.

So what did Apple do? They told us. They were going to take the best of the new ideas for iPhone (and iPad) and bring them back to OS X? They said so. They really did.

But who were those developers on the iPhone project? They were, according to Scott Forstall at his Stanford talk, the best of the OS X developers he could find.

[Homework assignment: if Apple put their best OS X programmers on iPhone, who's working on OS X? Hint: the transition from 10.4 to 10.5.]

Being able to quickly summon up a file, do a bit of 'tap tap tap', and then just leave things and put your phone back in your pocket - that makes a lot of sense on a phone... But it's batshit on an ordinary computer. And yet Apple pushed for it, and they continue pushing for it, as evidenced in the links above. Yes, things were supposed to get better with 10.8, but they didn't really get better, did they?

And yes, there's always a lot of thought that goes into changes, as the Fireball once pointed out when people first discovered everything in iPhone was running as root. 'If you people don't understand that Apple put a lot of thought into this', he boasted, someone who does not and will never understand file systems and system security... And yet at the very next 1.3 release, things were fixed and ordinary apps didn't run as root anymore. So despite all that thinking, mistakes are made, even by the company that can get so many things right, the company with the biggest market cap in the world.

The rush to Intel... Thermal grease... Screen remnants... All well known catastrophes. Mistakes do happen.

Return of the Ken

Ken Thompson should be an icon in everyone's world. The partner of the late Dennis Ritchie and lead developer and designer on the Bell Labs Unix project, Ken had a steadfast way of looking at systems. Files are only streams of bytes. Directories are files too. 'Keep your hands off the drivers', he once said - an aphorism Microsoft never took to heart. As another example.

Ken was also a stickler for the Doug McIlroy principle that programs are supposed to do one thing and do it well - as, for example, with command interpreters (or shells). All they bleating do is interpret commands. FFS. Try telling that to Microsoft. Their command interpreter told you the time of day, listed directory contents; they probably had a special fellating version for private use by Bill Gates...

But Ken's command interpreter did only one thing. Only one. And it's because Ken drew the line and kept his shells to that assignment and no others that it's so easy to interchange shells today. The original Bourne shell sh, csh from Berkeley, the 'Bourne again' shell bash, the Korn shell ksh, tcsh, and so forth. Add a new one any time you like. Come join the party.

Likewise there are file system operations that belong in file systems (and managers) and nowhere else. Look again at the menu on that first image from above. Now admittedly they've had a poxed PFY from Scandinavia there for a while, but that's no excuse.

  • Close All. You don't need close all if you can just exit the application. That's the way it's supposed to work. And with their new whiz-bang code running, an app that doesn't have any open documents will close on its own anyway. Pure bollocks.

  • Rename. No, this is a file system command, and does not belong on an application menu.

  • Move To. Same thing. Of course there's no point trying to explain that 2 and 3 are the same thing anyway. Mr Pinhead'll come around and complain he can't save a file again, this time in /dev.

Grey Havens

Back to the Apple discussion forums when this whole thing blew up four years ago.


You know Apple are watching. They sometimes say they're not patrolling anymore, but they are. You know they are. This thread finally went dead two years back after accumulating almost 70,000 posts. Clearly something is wrong.

Some of the comments are worth repeating. Dennis Burnham was very angry, but he was very articulate as well.


Save for the locked/unlocked bit (which is unforgivable) this all ties together. But to accomplish all this, Apple had to build an astoundingly top-heavy superstructure. There's .DocumentRevisions-VXXX at root; there's ~/Library/Containers; ~/Library you're going to want to hide, even though it's a user directory; and so forth - all to sustain an illusion of 'persistence'.


You can try putting the above in ~/Library/Preferences/.GlobalPreferences. Some say it works well; some say it even speeds things up on your system.

At application level, for the programmers that is, there's nothing - knock on wood - forcing them to use the new system. 'Save' and 'Save As' exist as before - it's the ISVs who switch out the old code for the new. They don't have to.

At the end of the day, it's nothing to get too worked up about. Or put it like this: most people don't have the time to get worked up about it. They figure out how to do things, then they get back to work.

But put it all together, and it's a lot of really disastrous design decisions.

See Also
Xfile: Free Test Drive
CLIX: Learn How to Fish

About | ACP | Buy | Forum | Industry Watch | Learning Curve | Search | Twitter | Xnews
Copyright © Rixstep. All rights reserved.