The OpenStep Standard

It's not an operating system but it expects one.

Get It Try It

NeXTSTEP, NEXTSTEP, NextStep, OPENSTEP - a lot of confusing names for what Apple today call Cocoa and even OS X. By now most Apple users understand their operating system comes from NeXT Computer (aka NeXT Software) but the abundance of outright contradictory and confusing names can get them not knowing left from right or up from down.

Appearances don't deceive.

Disregard the various ways of capitalising 'NeXTSTEP'; there is however an important distinction between NeXTSTEP (or NEXTSTEP or NextStep) and OPENSTEP.

NeXTSTEP is the original 'operating system' released by NeXT Computer aka NeXT Software. It ran with an underbody of FreeBSD, a variant of the Berkeley branch of Unix.

But already before Redwood City merged with Cupertino, a 'standard' had been agreed upon called 'OPENSTEP'. OPENSTEP was NeXTSTEP without the FreeBSD underbody.

OPENSTEP was ported to Sun Microsystems' SunOS; to Hewlett-Packard's HP-UX Unix; even to Microsoft's 'NT'. It could be ported because OPENSTEP did not rely on the FreeBSD underbody and because the three operating systems mentioned were all up to par for the task.

MacOS

If you look inside a typical Cocoa application package you'll find a folder called 'MacOS'. This is the folder that contains the actual program executable. However the name 'MacOS' is in this context a grievous misnomer.

With SunOS, HP-UX, and NT the basic job of providing a secure 32-bit protected mode platform had already been done, as had been done with FreeBSD before them. All the NeXT engineers had to do was rewire their NS classes into the new sets of APIs.

This is not the same as interacting with command line tools; it's identical with interfacing with all the operating system kernel hooks those command line tools use too. (And therefore it's expected to be very fast.)

But those platforms were ready for the task; anno 1997, when NeXT came to Cupertino, Apple had nothing to offer.

Great Expectations

Apple had tried for ten years to make the big step up to secure 32-bit computing - but failed miserably. Sun had their system; so did Hewlett-Packard; and so did Microsoft, thanks to former DEC engineer David Cutler and his DEC team. Apple on the other hand had nothing.

Had 'MacOS' been a viable support for OPENSTEP, then surely that's the way the NeXT engineers would have gone. But not only did MacOS remain a 16-bit system incapable of multiple users, secure memory, and proper virtual memory management, it had also become a hopelessly wobbly system incapable of even idling without a crash.

By the time Apple crawled with the olive branch to Redwood City Apple Computer were in an extremely sorry state. They'd already turned down an offer to acquire the excellent BeOS for less than one seventeenth what they would pay for NeXTSTEP and they were now only months away from dissolution, from chapter eleven, from outright bankruptcy. The merger with NeXT saved them - for a while.

And not only did they get a 32-bit operating system, they got arguably the best 32-bit system ever made and they also got NeXT's charismatic leader, a co-founder of Apple Computer. All for only 429 cold ones.

But now the challenge: as NeXT was already working with its OPENSTEP open standard and as the task at hand should be to port OPENSTEP to yet another platform, but because Apple did not have an adequate platform, what to do?

It's gone to history that the chief of software at NeXT Avie Tevanian argued against porting OPENSTEP to the Apple platform as it was just too difficult. Considering that Apple had just paid $429 million for the system, this wasn't a well received suggestion.

But because Apple had absolutely nothing to build on, NeXT would have to provide the basic operating system on their own. And it's no surprise they chose FreeBSD, as it was the platform they'd originally run and most of that code had survived.

But here comes the hard part: for this was ostensibly a port of OPENSTEP and yet it wasn't. It would have been easy enough if they'd just dispensed with the niceties and released the old NeXTSTEP and been done with it. In fact, it would have been admirably simple.

Maccies

Enter the Maccies - users of and developers for the old 'MacOS', the system that had been condemned as just too behind the times, wobbly, and obsolete. They'd ignored the events in Redwood City even though their fearless leader was there, and they'd ignored the writing on the walls that said they'd some day have to adjust. They didn't want to adjust.

Microsoft handled their transition from 16-bit to 32-bit admirably. Their Windows 9x team followed in the wake of David Cutler and the NTx team, emulating as much as they could, providing a 32-bit shell on the old 16-bit kernel, until the Windows 9x branch finally died out (with Windows Me it was a horrible death) and the two branches 'merged' into Windows XP.

Apple didn't do as well. They could have just scrapped the old APIs completely and told everyone 'OK, this is how Apple computers work today and this is how you write programs for them', but that's not the way things turned out.

What intentions the NeXT engineers had back then is doomed to speculation, but it is known that Steve Jobs was booed while onstage demonstrating NeXTSTEP for a crowd of Maccie developers back in 1998. They'd waited nearly ten years for something to happen; they must have understood that whatever happened it would be about change; and yet when they finally saw the product that would save them - they booed.

Madness

The strict architecture of OPENSTEP requires and demands a single underbody. You can't have two operating systems in the same box. With NT the case was clear; with HP-UX it was equally clear; with SunOS it was adamantly clear.

But now Apple had FreeBSD running underneath, and that should have been adamantly clear as well, but the Maccies weren't giving Apple any rest.

What Apple then did - evidently to appease this angry, unreasonable, and unpleasant crowd - was wire two operating systems side by side under the same hood.

Never before or since has any platform had two different sets of syntax rules to access items in its file system; never before has a fundamental philosophy of an operating system been compromised by the existence of two separate and radically opposed philosophies.

Never before has anyone been so wacky as to attempt to wire two radically different operating systems together under the same bonnet. (And hopefully no one will ever attempt it again.)

'MacOS' has nothing in common with Unix: the twain can never meet. Never.

And yet when you see your program executable in the 'MacOS' folder, this doesn't mean you're running OPENSTEP on top of MacOS; MacOS has ceased to exist; you're running OPENSTEP on top of FreeBSD.

And yet the cruft of old 'MacOS' is still in there. In one of the most confused projects ever in the history of computer science - far more confusing than the original Macintosh project - Apple wired in the old 'MacOS' APIs all over the place into FreeBSD and OPENSTEP. Sometimes they wired them in 'above' the FreeBSD layer, and sometimes below.

There was no method to the madness: it was only madness.

It would be one thing if this old stuff were just an alternate (albeit feeble) way to access the 'hooks' in FreeBSD, but the reality of OS X is not that simple. Any number of FreeBSD snippets were rewired to hook into this mongrel 'MacOS' layer instead of the other way around.

Toll-free Bridging

As any developer has noticed, the classes of NeXTSTEP are ultimately resolved today in a 'hybrid' often with the prefix 'NSCF' (standing for 'NeXTSTEP' and 'Core Foundation', an element of this old 'MacOS' API now wired into the OS). This is not OPENSTEP code, and it's not anything else either: it's officially called 'toll-free bridging' and it's begging for trouble.

Today it's almost impossible to rip the 'Mac' layer out of OS X. Almost the entire operating system - and OPENSTEP - would have to be rewritten. Either that or start again with the legacy NeXTSTEP anno 1997 and gradually build all the subsequent enhancements on top - perhaps a somewhat easier task but ultimately not much less time consuming and daunting.

And feature after feature of the space age OPENSTEP has been deprecated because it cannot be duplicated in old MacOS. Although this appeases the Maccie developers who would rather no one had these features if they can't, the Apple users of today are the ones losing out big.

And what's been left to the whining 'Maccie' developers isn't much either. Initially a few brave souls from NeXT figured they could placate the 'Maccie' contingent over time and gradually put things right again with a clean interface between OPENSTEP and FreeBSD, but things are simply not working out that way.

Apple's recent 10.4 release of OS X should crush all the hopes developers had that Apple would finally grab the rudder of this wayward ship and set things straight again. As Apple continue to sail the waves in these turbulent waters, the end grows nearer: namely the day when Apple will be forced to admit they've ruined yet another operating system and the ship will soon sink.

Colons & Slashes, CNIDs & Inodes

The inherent conflicts in merely assuming it's possible to have two underlying file system technologies coexist side by side serves to illustrate how hopeless the situation actually is.

• The file system of choice for the OS X workstation is HFS; colons are not allowed in HFS filenames as the colon is used as a path component separator.

• Unix uses the slash as the path component separator and allows colons in filenames.

• Unix also uses an elegant file system structure known as the ilist which contains file specific information in what are known as inodes.

• HFS doesn't have inodes; it has CNIDs (catalog node identifiers).

• With HFS it is possible to backtrack from a CNID to a physical file on disk. With a Unix file system it is possible to have multiple 'hard links' to the same physical file.

• Where a Unix file system is elegantly simple, the 'MacOS' file system is hopelessly wobbly, confused, and contradictory.

And yet these two are expected to live side by side. They can't - and neither can any of the other conflicting technologies bundled willy-nilly today into what is known as 'OS X'.

The battle is not between Nexties - the NeXTSTEP programmers who've migrated with their platform to Apple - and Maccies: the battle is between MacOS which no longer exists and FreeBSD which does exist. Apple continue to pretend these two can sleep side by side; they can't - and until Apple admit this things can only get worse.

About Rixstep

Stockholm/London-based Rixstep are a constellation of programmers and support staff from Radsoft Laboratories who tired of Windows vulnerabilities, Linux driver issues, and cursing x86 hardware all day long. Rixstep have many years of experience behind their efforts, with teaching and consulting credentials from the likes of British Aerospace, General Electric, Lockheed Martin, Lloyds TSB, SAAB Defence Systems, British Broadcasting Corporation, Barclays Bank, IBM, Microsoft, and Sony/Ericsson.

Rixstep and Radsoft products are or have been in use by Sweden's Royal Mail, Sony/Ericsson, the US Department of Defense, the offices of the US Supreme Court, the Government of Western Australia, the German Federal Police, Verizon Wireless, Los Alamos National Laboratory, Microsoft Corporation, the New York Times, Apple Inc, Oxford University, and hundreds of research institutes around the globe. See here.

All Content and Software Copyright © Rixstep. All Rights Reserved.