Three Reasons 1/3

Apple's 'Leopard' OS remains a system punters can and will adopt but Apple are not within reach of the more important professional market.

Get It Try It

Apple are now into their fourth release of the world's major 64-bit operating system and according to pundits everywhere things still aren't right. Eventually the previous (Tiger) version will disappear and the single digit user base will use the new system almost exclusively. But that doesn't correlate to anything significant. Apple's OS will remain doomed to the margins.

There are at least three reasons for this. These are perhaps difficult for punters to understand but they're there still the same.

This article addresses the first of these.

The writeToFile:APIs. There's a whole slew of these methods in the Cocoa (NeXTSTEP) classes. They're used to integrate with the fabulous 'document controller'. This document controller is one of the reasons the system is so superior to anything Microsoft or the world of Linux have to offer.

Cocoa developers take what they're given from this controller and return data as they're asked to. There are actually a significant number of related APIs here.

-(BOOL)[NSArray writeToFile:(NSString *)path atomically:(BOOL)flag]
-(BOOL)[NSData writeToFile:(NSString *)path atomically:(BOOL)flag]
-(BOOL)[NSData writeToFile:(NSString *)path options:(NSUInteger)mask error:(NSError **)errorPtr]
-(BOOL)[NSDictionary writeToFile:(NSString *)path atomically:(BOOL)flag]
-(BOOL)[NSDocument writeToFile:(NSString *)fileName ofType:(NSString *)docType]
-(BOOL)[NSString writeToFile:(NSString *)path atomically:(BOOL)flag]
-(BOOL)[NSString writeToFile:(NSString *)path
        atomically:(BOOL)flag encoding:(NSStringEncoding)enc error:(NSError **)error]
-(BOOL)[NSFileWrapper writeToFile:(NSString *)node atomically:(BOOL)atomically
        updateFilenames:(BOOL)updateNames]
-(BOOL)[NSDocument writeToFile:(NSString *)path ofType:(NSString *)docType
        originalFile:(NSString *)originalPath saveOperation:(NSSaveOperationType)saveOperationType]

Several of the above APIs are deprecated and replaced by the now favoured writeToURL: family of APIs. There are also the dataRepresentationOfType: and loadDataRepresentation:ofType: methods found in Apple project templates.

Note in particular the pervasive method argument atomically.

-(BOOL)[NSArray writeToFile:(NSString *)path atomically:(BOOL)flag]
-(BOOL)[NSData writeToFile:(NSString *)path atomically:(BOOL)flag]
-(BOOL)[NSDictionary writeToFile:(NSString *)path atomically:(BOOL)flag]
-(BOOL)[NSString writeToFile:(NSString *)path atomically:(BOOL)flag]
-(BOOL)[NSString writeToFile:(NSString *)path
        atomically:(BOOL)flag encoding:(NSStringEncoding)enc error:(NSError **)error]
-(BOOL)[NSFileWrapper writeToFile:(NSString *)node atomically:(BOOL)atomically
        updateFilenames:(BOOL)updateNames]

atomically: is defined as follows.

If YES attempts to write the file safely so that an existing file at path is not overwritten, nor does a new file at path actually get created unless the write is successful.

Which is purely and simply a load of shit. The implication is that a new (temporary) file is used for output: the document controller contacts the application through one of the typical I/O methods and the application responds by using one of the above methods and if 'atomic' writes are desired the temporary file when I/O is complete will be 'moved' over the older existing file.

And things still work this way under Leopard. But the big lie is the atomically: flag is now ignored.

Do like this.

1. Fire up both Terminal and TextEdit.
2. Scoot Terminal to your ~/Documents folder.
3. Save your empty TextEdit document as Untitled.rtf.
4. Run the command ls -i Untitled.rtf from Terminal.

You'll get something like '910564 Untitled.rtf' back.

That number in step 4 is the file's inode. It's a unique identifier that represents the file Untitled.rtf. If Untitled.rtf is read from and written to (but not atomically) on all Unix systems - and all Apple's 'Unix' systems prior to Leopard - that inode won't change. It's still the same file.

Note that saving a file 'atomically' will necessarily change the file's inode because the file is first saved to a new file and the new file is then moved over the old one, removing the old one in the process. It is no longer the same file.

Now with the same Terminal and TextEdit windows still open click 'save' in TextEdit and see what happened to your inode. If you haven't done anything else between operations you should get a new inode back two higher ('910566') than the previous.

And why is this? One inode was probably wasted for temporary storage prior to the save; the second one's your new file - what you thought was your old (updated) file. But it's not your old file - it's a new file.

[Compare this behaviour with earlier versions and common Unix distros or just trust it's true for now. Because it is. Ed.]

Why is this important? The lowest level Unix I/O APIs are as follows.

NAME
     pwrite, write, writev -- write output

LIBRARY
     Standard C Library (libc, -lc)

SYNOPSIS
     #include <unistd.h>

     ssize_t
     pwrite(int fildes, const void *buf, size_t nbyte, off_t offset);

     ssize_t
     write(int fildes, const void *buf, size_t nbyte);

     #include 

     ssize_t
     writev(int fildes, const struct iovec *iov, int iovcnt);

DESCRIPTION
     Write() attempts to write nbyte of data to the object referenced by the
     descriptor fildes from the buffer pointed to by buf.  Writev() performs
     the same action, but gathers the output data from the iovcnt buffers
     specified by the members of the iov array: iov[0], iov[1], ...,
     iov[iovcnt-1].  Pwrite() performs the same function, but writes to the
     specified position in the file without modifying the file pointer.

Note the function argument fildes and how it's used.

attempts to write nbyte of data to the object referenced by the descriptor fildes

A separate function is called to return the descriptor. For existing files this function is open.

NAME
     open -- open or create a file for reading or writing

SYNOPSIS
     #include <fcntl.h>

     int
     open(const char *path, int oflag, ...);

DESCRIPTION
     The file name specified by path is opened for reading and/or writing, as
     specified by the argument oflag; the file descriptor is returned to the
     calling process.

The return value represents a file system descriptor: a unique number (not the inode) used for the subsequent I/O operations. There is no attempt to first destroy an existing file and then replace it with a new one. The returned value is a descriptor to an already existing file.

The APIs in the abstraction level immediately above the write family are as follows.

NAME
     fread, fwrite -- binary stream input/output

LIBRARY
     Standard C Library (libc, -lc)

SYNOPSIS
     #include <stdio.h>

     size_t
     fread(void *restrict ptr, size_t size, size_t nitems,
         FILE *restrict stream);

     size_t
     fwrite(const void *restrict ptr, size_t size, size_t nitems,
         FILE *restrict stream);

DESCRIPTION
     The function fread() reads nitems objects, each size bytes long, from the
     stream pointed to by stream, storing them at the location given by ptr.

     The function fwrite() writes nitems objects, each size bytes long, to the
     stream pointed to by stream, obtaining them from the location given by
     ptr.

Note the final argument in both functions: a pointer to a structure of type FILE. This structure will contain a reference to the file descriptor used above. Both the above functions presume use of another function corresponding to open to instead return a pointer to a FILE.

NAME
     fdopen, fopen, freopen -- stream open functions

LIBRARY
     Standard C Library (libc, -lc)

SYNOPSIS
     #include <stdio.h>

     FILE *
     fdopen(int fildes, const char *mode);

     FILE *
     fopen(const char *restrict filename, const char *restrict mode);

     FILE *
     freopen(const char *restrict filename, const char *restrict mode,
         FILE *restrict stream);

DESCRIPTION
     The fopen() function opens the file whose name is the string pointed to
     by filename and associates a stream with it.

In all the above cases - in the 'rock solid foundation' - the functions access an existing file and write to that file 'as is'. New files are not created and old files are not removed from the file system. Considering the difficulties Apple have had in securing file 'move' operations this should be a good thing.

So what happened?

First and foremost it must be understood that something really happened. Neither previous versions of Apple's OS nor any other self respecting Unix ever acted this way. Someone consciously went in and rewrote an extensive number of system APIs to achieve what must be called 'weird behaviour'.

So why do this? And what happens to the option to not save files 'atomically'? Why have the option at all if your entire API summarily ignores it?

A clue may be found in what Apple affectionally refer to as 'call support drivers' - in other words: what ordinary punters are complaining about. The pretext is the more and the louder you complain to Apple support the better your chances of getting something done - even if it's not a good thing.

And what could ordinary punters be complaining about? And note: professionals wouldn't be complaining about this - they'd know better.

A look at how Apple deal with write protected files in the new system reveals all.

There is no such thing in Unix as 'locked'. This is patently ludicrous. The file simply doesn't have write permissions. And what Apple are proposing with 'do you want to overwrite it anyway' is a stealth method for replacing protected files. Files presumably protected for a very good reason.

Note you don't have to authenticate here; the system isn't pulling root privileges out of its butt; you're deliberately taking the file system where it wasn't supposed to go.

And it gets worse. Unix file system security depends on use of proper permissions for both directories and files. Directories are files too. Users are not allowed to alter the contents of a directory if they don't have permission to write to it. But altering a file in a protected directory is something else - and intentionally so. Files must be protected separately.

It's very easy actually.

• If you want to prevent a file from being overwritten you limit the write permissions on the file itself.
• If you want to prevent the contents of a directory from being tampered with you limit the write permissions on the directory.

Tampering with the contents of a directory includes renaming files or subdirectories, adding files or subdirectories, and removing files or subdirectories.

It's a scheme that's worked admirably for the past forty years or so. It's one of the reasons Unix has come to the fore and even Apple are forced to use it.

• The reason Leopard can 'overwrite' files that are 'locked' is the files aren't overwritten at all. First lie.
• Your files are destroyed and new files are put in their place. [Knock on wood, dude!] But you're never told that.
• To make matters 'simpler' [ahem] Apple treat all files this way whether write protected or not. Second lie.

This means the file system didn't really have to ask you whether you wanted to 'overwrite' - it was going to do it anyway. And this in turn means the new Apple code is not working with the file system anymore.

The file system is supposed to protect itself. The new code directly subverts this protection. Use of this new code also means 'upper level' code must assume responsibilities previously held by low level file system code. And that's never a good idea.

And suddenly those common operations system administrators had organised over their LANs and WANs won't work as before. Common ways of assembling and protecting file systems are now for nought.

Picture a stronghold on a server where ordinary users are allowed to read and write to certain files but they are not allowed to remove rename or add files. [Will this become the new .DS_Store scandal? Ed.]

Users of Leopard will not be able to store files protected in this fashion and the diagnostic offered by the system is nothing short of confusing and misleading.

• The file has all the 'appropriate access privileges' [sic] it needs. Third lie.
• The directory is 'locked' - not the file. But you're never told that.
• You didn't do something wrong - Apple did. Fourth lie.
• The recommendation to change 'access privileges' on the file gets you nowhere - there's nothing wrong with the file. It's the directory. Punters won't figure this one out. The diagnostic sends them off in the wrong direction.

But what happens if you don't own the file - if it's for example owned by root? You get the same thing.

But try this while still in Terminal.

1. Turn off all access with 'chmod 000 Untitled.rtf'.
2. Change the ownership with 'sudo chown 0:0 Untitled.rtf'.

Now that sucker is locked - right? It's locked so no one can access it. Not even root. Can you still destroy it? Could you still theoretically replace it with another file with the same name?

1. With TextEdit do a 'save as' to the new file 'tmp.rtf'.
2. Use 'mv tmp.rtf Untitled.rtf to replace the original.

[On Leopard you'll get yet another 'silly putty' prompt but you won't need to authenticate. The file system doesn't require it.]

So suddenly you're looking not only at weird 'file system behaviour' - you're looking at disparate ways to deal with the same situation in the same operating system. To call this a bloody mess would be kind.

By giving into what presumably were a lot of totally demented 'call support drivers' Apple have essentially made fools of themselves yet again.