About | ACP | Buy | Industry Watch | Learning Curve | News | Products | Search | Substack
Home » Learning Curve » Developers Workshop


Don't take chances. Don't trust the next guy to be as careful as you.

Get It

Try It

Self-defence is a cornerstone of programming. Take an example from the Cocoa classes themselves. Following are a number of new class methods introduced with Tiger.

-(BOOL)readFromData:(NSData *)data ofType:(NSString *)typeName error:(NSError **)outError
-(BOOL)readFromFileWrapper:(NSFileWrapper *)fileWrapper ofType:(NSString *)typeName error:(NSError **)outError
-(BOOL)readFromURL:(NSURL *)absoluteURL ofType:(NSString *)typeName error:(NSError **)outError
-(BOOL)writeToFile:(NSString *)path atomically:(BOOL)useAuxiliaryFile encoding:(NSStringEncoding)enc error:(NSError **)error
-(BOOL)writeToFile:(NSString *)path options:(unsigned int)mask error:(NSError **)errorPtr
-(BOOL)writeToURL:(NSURL *)absoluteURL ofType:(NSString *)typeName error:(NSError **)outError
-(BOOL)writeToURL:(NSURL *)absoluteURL ofType:(NSString *)typeName forSaveOperation:(NSSaveOperationType)saveOperation originalContentsURL:(NSURL *)absoluteOriginalContentsURL error:(NSError **)error
-(BOOL)writeToURL:(NSURL *)absoluteURL ofType:(NSString *)typeName forSaveOperation:(NSSaveOperationType)saveOperation originalContentsURL:(NSURL *)absoluteOriginalContentsURL error:(NSError **)outError
-(BOOL)writeToURL:(NSURL *)aURL options:(unsigned int)mask error:(NSError **)errorPtr
-(BOOL)writeToURL:(NSURL *)url atomically:(BOOL)useAuxiliaryFile encoding:(NSStringEncoding)enc error:(NSError **)error
-(NSDictionary *)fileAttributesToWriteToURL:(NSURL *)absoluteURL ofType:(NSString *)typeName forSaveOperation:(NSSaveOperationType)saveOperation originalContentsURL:(NSURL *)absoluteOriginalContentsURL error:(NSError **)outError

What they all have in common is that final argument - the address of a pointer to an NSError object. Whether it be called outError or error or errorPtr or whatever. It's a pointer - and no more. But it's declared as pointing to type NSError. And its address is passed by the caller to the above methods.

That pointer to an NSError object is a stack variable. And it's not initialised either. At least not in Leopard. It's a garbage value. Its value is whatever else was on the stack there last time around.


Now the idea with tacking on this new argument is user friendliness. To be able to tell the user something more informative than 'sorry that didn't work'. So if someone further down the line can offer additional details into why the operation didn't work then great. The caller of course checks the return value before checking the (NSError *) variable. In all the above cases a 'zero' return means something went wrong.

Of course there's also the possibility (the probability) that in the case of an error nothing further can or will be obtained. So for that purpose the caller has to check the 'numerical' value of the (NSError *) variable first. If it's non-zero that means the callee has initialised it with good data and the caller can access that data.

But that assumes the caller initialised the variable to zero before initiating the call.

NSError *err = 0;

This was done with Tiger but someone forgot that line of code in Leopard. So an important level of self-defence was lost.

The caller has to know that if it accesses that variable it's a good value and he won't crash and burn. But he can't possibly know this if he didn't initialise the variable in the first place, can he?

[If you're not following: all addresses are paged. They're not real. Applications can't touch real memory. The page tables for a process contain the addresses that can be transformed by the virtual memory manager. If the address you're looking for isn't in the page tables then you crash and burn (or worse). A zero value normally means a pointer can't be used. If you want to know that no one has added data you have to give the variable a sentinel value (nil or 0) to start with - anything else assumes the address is valid. And accessing a garbage stack value as a pointer in the best case will crash you.]

So here's what happens when the callers call on Leopard.

  1. Get all the arguments ready.
  2. Declare an NSError pointer on the stack and initialise it to zero.
  3. Make the call.
  4. Check the return value.
  5. If return is zero then check the NSError pointer.
  6. If pointer is non-zero then go ahead and access it. You should be OK.
  7. If pointer is zero then display standard error alert text instead.

Tiger systems handled this properly but Leopard systems don't. In terms of self-defence this is very bad. In terms of common sense efficiency it also means one single line of code in a few places in a few frameworks has to be replaced by thousands of lines of code in thousands of client applications that access those frameworks.

The Launch Services

Apple have this fantastic thing called the launch services. They're actually pretty brilliant. The basic idea is you never need to search for any application by full path more than once. As soon as an application is run the kernel passes info onto the launch services. Mostly from the Info.plist. Such as icon, the file types it uses, the full path.

The launch services are interfaced directly by the NSWorkspace method openFile:withApplication:. This method, once a panel in Terminal.app in the late 1990s, is functionally pervasive in the system today.

-(BOOL)openFile:(NSString *)fullPath withApplication:(NSString *)appName
-(BOOL)openFile:(NSString *)fullPath withApplication:(NSString *)appName andDeactivate:(BOOL)flag

These methods are extremely flexible and useful. You can supply either the full path to a document file or the name of an application or both. If you only supply a full path to a document the system will open it with the default application. If you also supply the name of an application the system will open the document with the application you've specified. And if you only supply the name of an application the system will launch the application with no command line arguments.

You don't even need to add the '.app' extension to get it to work.


The appName parameter need not be specified with a full path and, in the case of an application wrapper, may be specified with or without the .app extension, as described in 'Use of .app Extension'. If appName is nil, the default application for the file's type is used.


    •  Available in Mac OS X v10.0 and later.

Note the return values of both methods - they're BOOL. That means that success returns non-zero and failure returns NO (zero). Both methods are prepared to handle situations where #1) documents aren't found according to the full paths given; and/or #2) the app name specified doesn't correspond to something already in the launch services caches.

As can be seen from the graphic this functionality has been around a long time. And worked flawlessly. Do what the caller asks but if you can't you return a zero value. Said and done and you're out of there.

Something changed with Tiger. For Tiger added Spotlight, fsevents, and a metadata layer. The metadata layer is called once the system and the launch services are through with a call.

But there's a problem. It can best be described in two parts.

  1. The launch services pass the input data down to the metadata layer regardless of whether the requested operation is successful or not. In other words regardless of whether the specified full path to a document file is correct or whether the application name actually corresponds to an existing 'known' application in the system.

  2. The metadata layer assumes the data is OK and has no self-defence of its own. If the full path doesn't exist or the application name isn't known the metadata layer goes berserk trying to process the data and eventually crashes the application.

Bad self-defence.

HFS Document Tracking

A good example of how self-defence works is the type of alert the Cocoa document controller will issue if an open file is renamed, moved, or outright disappears. This is only possible with the HFS file systems as these file systems track files not by path but by CNID.

  1. A file has been moved. The controller knows this because the path associated with the CNID has changed.

  2. A file has been renamed. The controller knows this because the file name associated with the CNID has changed.

  3. A file has disappeared. The controller knows this because the CNID itself is no longer in use. But the controller remembers the last full path and can let you start there to choose a new path when the file is to be saved again.

The above is not readily available in Unix because document control doesn't put any focus on CNIDs. On Unix systems the corresponding inode can be referred to by any number of totally separate full paths on the same volume. HFS can do this as HFS proved incapable of correctly implementing Unix hard links.

But this is a good (and welcome) feature. The day ZFS or some other file system finally replaces the whiskered HFS systems this feature will most likely no longer be available.

File Systems

For file systems themselves try the following experiment. Open Terminal.app and perform the following.

[Don't jump ahead and don't try anything until you're expressly told you can do it.]

$ cd ~/Desktop
$ mkdir test
$ cd test
$ mkdir foo
$ cd foo
$ echo file1 >file1
$ mkdir bar
$ cd bar
$ echo file2 >file2

Now see what you've got.

$ cd ~/Desktop/test
$ ls -ailnR

You should have this.

total 0
6951768 drwx------   3 501  20  102 Apr 26 07:41 .
  96136 drwx------   3 501  20  102 Apr 26 07:41 ..
6951764 drwxr-xr-x   4 501  20  136 Apr 26 07:40 foo

total 8
6951764 drwxr-xr-x   4 501  20  136 Apr 26 07:40 .
6951768 drwx------   3 501  20  102 Apr 26 07:41 ..
6951766 drwxr-xr-x   3 501  20  102 Apr 26 07:40 bar
6951765 -rw-r--r--   1 501  20    6 Apr 26 07:39 file1

total 8
6951766 drwxr-xr-x   3 501  20  102 Apr 26 07:40 .
6951764 drwxr-xr-x   4 501  20  136 Apr 26 07:40 ..
6951767 -rw-r--r--   1 501  20    6 Apr 26 07:40 file2

Now here comes the fun - don't try these operations but only ask yourself what they'll do. Ask yourself if the following file operations make any sense - and ask yourself what you think will happen if you try them. [And for goodness sake don't try them yet!]

$ cp -R foo foo/bar
$ cp -R foo/bar foo/bar

Cocoa has a number of file operation methods in NSWorkspace and NSFileManager. NSWorkspace is part of the AppKit framework which is ordinarily reserved for classes with a user interface; NSFileManager is part of the Foundation framework which normally is not seen. NSWorkspace file operations in the AppKit filter down through to NSFileManager in the Foundation framework.

-(BOOL)performFileOperation:(NSString *)operation source:(NSString *)source destination:(NSString *)destination files:(NSArray *)files tag:(int *)tag

-(BOOL)copyPath:(NSString *)source toPath:(NSString *)destination handler:(id)handler
-(BOOL)movePath:(NSString *)source toPath:(NSString *)destination handler:(id)handler

To what degree should these methods prevent operations that compromise the system?

Ideally all AppKit and Foundation code will devolve down to core kernel APIs and it'll be the file system that in such case determines what's to be done and what's not. But there's a problem with OS X: there are parallel APIs all over the place. Which APIs are really being used in any one given situation? At what level are they found? Consequently: do you know how well your file system is protected?

If you were to try either of the command line tests outlined above you'd soon see you ran out of characters in the paths needed. Unix file names can only be so long and Unix paths have a limit as well. There's nothing really 'wrong' with trying to copy a directory hierarchy onto itself. It's not exactly productive and things can only end one way but the operation itself isn't illegal and normally shouldn't hurt your file system per se. But how about this? What do you think will happen here?

$ cd ~/Desktop
$ echo file1 >file1
$ cp file1 file1

[This one you can go ahead and try. It's not going to hurt you. Really.]

cp: file1 and file1 are identical (not copied).

The file system flagged your request as a 'bad op'. It performed a 'sanity check' on it to see if it made any sense. If it was destructive. And it decided against accommodating you.

Of course file system sanity checks have to do a lot more. Error isn't always intentional. Any type of bad data can creep into an application and be used in a Cocoa file operation that potentially will hose your file system.

Apple have been on the hook for years for letting users unwittingly replace entire hierarchies of files and folders with single files (by mistake of course - who would ever intentionally want to do such a thing). Things like this can happen when people don't properly see what they're doing, work too fast, and click on those push buttons too fast and don't see what they've done until it's too late.

Sane systems won't allow such a thing. Never have, never will. Today any number of third party apps on OS X specifically disallow this to protect their users where Apple won't. Rixstep's Xfile goes even farther than disallowing file/folder mixups: it doesn't allow the mix of any Unix file types at all: you can't replace a socket with a symlink; or a symlink with an ordinary file; or a file with a folder; or a folder with a file; and so forth. But Apple's code doesn't stop this. And that's not a good thing. That's not good self-defence. That's not protecting the users.

The cavalier answer given by Apple has always been 'but the user is asked a question so we're off the hook'. As if 'do you really want to hose your entire file system and lose all your files' is to be considered a legitimate question with more than one plausible answer.

Bottom Line

Self-defence: every piece of code everywhere has to be aware that some other piece of code somewhere else can screw up. And be ready for it. And each 'computer within the computer' has to build up its own self-defence mechanism. And the most vital of these is the file system itself. The file system must never allow wayward code to be executed. It must always analyse the incoming requests and filter out the bad stuff through rigorous sanity checks.

Never take chances. Never assume anything. Always practice self-defence.

About | ACP | Buy | Industry Watch | Learning Curve | News | Products | Search | Substack
Copyright © Rixstep. All rights reserved.