About | Buy Stuff | Industry Watch | Learning Curve | Products | Search | Twitter
Home » Industry Watch » The TechnologicalHall of Monkeys) (» Heroes Banquet)

Apple's File System APIs

'Shambolic' comes to mind. Again.


Buy It

Try It

Apple's file system APIs have never been good. But recent discoveries put things in an even more embarrassing light. Tom Karpik's discovery is the tip of an iceberg. It's a huge tip but it's only the tip.

Massive Data Loss Bug in Leopard

Tom Karpik was privy to an unfortunate incident on 10.5 Leopard - he lost a lot of data on an ordinary 'move' operation across volume boundaries. In this case to a remote destination. The workaround seems obvious: don't move - copy. But that's missing the point entirely.

To recreate the incident Tom set up a Windows share, created a 384 MB gibberish file on his own Leopard machine, and proceeded to 'move' it to the Windows share. As his Finder proceeded to report on the progress of the file operation Tom made the Windows share inaccessible. And Finder correctly reported the error and aborted the operation.

The punch line was that Tom's original file was already gone.

Any number of people have reported similar disasters going all the way back to OS X 10.2 Jaguar.

Primitive

Apple's file system APIs stand in a class of their own. They're primitive. Witness the 'trash' functionality: files literally get 'moved' to a 'secret' folder on the same volume and the ability to 'restore' them to their original location is nigh on nonexistent. Compare with Windows where each file in the Recycle Bin gets a database entry with a serial number and the original location - restoring is child's play. Not so on OS X with Apple's 'advanced' user interface.

Apple's file system 'trash' functionality is no more sophisticated than that found common in mail programs.

But it goes deeper still - way deeper.

Carbon — Be Careful!

Apple's Carbon file system APIs - today thankfully deprecated - are ripe with dangers, their document pages containing dire warnings to developers to be careful as misplaced bits can in fact hose the entire system. No file system worthy of the name would ever leave data security and system stability in the hands of third party developers but Apple's did.

And according to MacInTouch Apple are still using this Carbon API in Leopard.

According to Apple, Leopard's Finder uses a specific application programming interface (API) called FSCopyObject for all of its file copying/moving.

FSCopyObject is a Carbon API introduced in the original Carbon for OS 9 programming library to replace some older APIs and ease transition to Mac OS X (then known as Rhapsody). FSCopyObject is designed as a high performance copy engine that supports full HFS+ metadata and ACLs (access control lists) in Tiger and Leopard.

Both Finder and Time Machine use FSCopyObject exclusively; it is Apple's blessed copy engine.

Cocoa — Any Better?

Apple file system APIs in their NeXTSTEP Cocoa are found in two distinct places in the 'Cocoa framework'. 'Cocoa' is actually a wrapper for the two traditional NeXTSTEP frameworks 'Foundation' and 'AppKit'.

The Foundation framework is just as its name implies. It contains code classes which can be used by any software, on screen and GUI or not.

The AppKit framework is the framework for GUI tools and controls. Classes in this framework all have on screen representation.

The higher level NSWorkspace resides traditionally in the AppKit. It used to have a visual interface. There used to be a 'workspace manager' on NeXTSTEP. NSWorkspace took care of file management for GUI applications. It doesn't anymore.

The lower level NSFileManager resides in the Foundation framework where it too has always been. It has no visual interaction and never had. NSWorkspace calls can filter through to this class.

The key to NSWorkspace file management is the performFileOperation: method - a rough counterpart to Microsoft's SHFileOperation.

performFileOperation:source:destination:files:tag:

Performs a file operation on a set of files in a particular directory.

Although not mentioned in the online documentation the caveat becomes apparent when moving down to the Foundation framework's NSFileManager. Here an example with the movePath:toPath:handler: method.

movePath:toPath:handler:

Moves the directory or file specified by a given path to a different location in the file system identified by another path.

destination
The path to which source is moved. destination must not yet exist.

This is a key to the whole dilemma: Cocoa file APIs assume the destinations don't exist. If the destinations exist the APIs all fail. Apple's documentation for the counterpart copyPath:toPath:handler: says the same thing (albeit in another place).

The file specified in source must exist, while destination must not exist prior to the operation.

There is no 'overwriting' of files as there is in the 'rock solid foundation'; write-protecting a file doesn't stop the Apple file system APIs; the nature of these APIs forces the application using them to delete the destination first. And write protecting a file doesn't protect it from being deleted.

Essentially the protections of the 'rock solid foundation' are not so much overridden as they're circumvented.

Sanity Checks vs 'Works as Designed™'

You won't catch anyone messing with Dave Cutler - not if they value their physical health - and you won't catch anyone messing with one of his file systems either. File systems have to protect themselves - they have to protect themselves against wayward end user behaviour and 'wild pointers' in third party software. The 'computer' often called 'MY computer' by Microsoft is of course nothing of the sort - the system must protect itself. And Cutler's system does.

Cutler's system - like all systems worthy of the name - performs a lugubrious number of 'sanity checks' on operation requests. It can't allow 'cyclical' operations which would end up consuming the entire hard drive; it can't allow files and directory hives to be copied into/onto themselves; and so forth.

Try any of that on a Windows box and Cutler tells you where to get off. But in the world of Apple there is no sanity checking whatsoever: any line of rogue code in any program written by any programmer in the world can contain a bug that hoses the entire system. And that bug can be prevalent or latent and only once in a great while manifest itself. And users can go on for the longest time blissfully unaware.

But the rabbit hole goes deeper still. One of the obvious things file systems always look for is file type inconsistency: you can't overwrite a directory with a file or vice versa. Try it from a command line and see.

But again Apple don't worry about such niceties: inasmuch as they never overwrite files anyway it's sort of moot. If you get a prompt to replace one item with another and the one is a directory and the other is a file - who cares? Apple don't and they've said so: despite all the third party code from BBEdit to MS Office to Rixstep's Xfile forced to include such code to prevent user disasters Apple casually declare 'Works as Designed™'.

Five minutes of hacking is all it takes to create a simple Cocoa application that irrevocably destroys an entire hard drive by simply giving Apple's file system APIs 'rough data'. Conceivably no more than the following is needed - and if that doesn't do it then any one of a myriad of alternatives is at one's disposal.

[[NSWorkspace sharedWorkspace] performFileOperation:NSWorkspaceMoveOperation
        source:[@"/Users"]
        destination:[@"~/Documents" stringByExpandingTildeInPath]
        files:[NSArray arrayWithObject:[@"~" stringByExpandingTildeInPath]]
        tag:&tag
    ];

The point of course being that the file system should detect something is wrong and prevent it. After all, all other file systems do.

Self–Respecting Means Self–Protecting

The little file system logic Apple have is not in their system itself but in an application - where it doesn't belong. Cocoa has two classes dealing specifically with file management, the one of which used to have a visual interface, but the code that should be there is not there. And in the past Apple's 'Finder' has had a number of narrow escapes, not really understanding what's going on and thwarting destructive user commands in spite of itself.

Posters at the Slashdot thread discussing Tom Karpik's article suspect this 'strange' situation once more is related to the annoying 'spatiality' paradigm. It wouldn't be surprising if it were true.

If Apple want to be taken seriously in the enterprise they're going to have to learn to think in terms of enterprise computing. Otherwise all Leopard will be is a 64-bit beige box.

See Also
Tom Karpik: Massive Data Loss Bug in Leopard
Slashdot: Data Loss Bug In OS X 10.5 Leopard
Apple: NSWorkspace Reference
Apple: NSFileManager Reference
MacInTouch: Mac OS X 10.5 Leopard: Finder Data-Loss Bug
Industry Watch: Sanity Checks at Apple
Learning Curve: A Sanity Check for Apple
Learning Curve: 4893378 FAQ
Learning Curve: Rebel Scum: More Attacks on 'Expected Behaviour'
Learning Curve: 4893378: 'Expected Behaviour'

About | Buy Stuff | Industry Watch | Learning Curve | Products | Search | Twitter
Copyright © Rixstep. All rights reserved.