About | ACP | Buy Stuff | Forum | Industry Watch | Learning Curve | Search | Test Drive
Home » Learning Curve » Developers Workshop

Uncle Unix

The quintessential example.

Get It

Try It

Ken Thompson and Dennis Ritchie may have created Unix. They may have won the US Technology Award and other awards for it, and they may have been assisted by the likes of Joe Osanna, Al Aho, Steve Johnson, and many others - and of course Brian Wilson Kernighan... But it was their department chief Malcolm Douglas 'Doug' McIlroy who instilled in them the right attitude for building a system like that in the first place.

As of 2007, Doug McIlroy is at Dartmouth College, an Adjunct Professor of Computer Science. Doug's the one who ran the PhDs in the golden years at the AT&T Bell Labs facility in Murray Hill, New Jersey, when Unix and so many other incredible things came to pass.

'Come through the front doors, you're in the hallway, now veer off to the right, follow that corridor to the end, there's a staircase there, take it up to the first floor, there we are', Brian told us.

That stairway was part of quite the story years earlier. Ken and Den had succeeded in getting their Unix running on a PDP and were in the process of porting it to an Interdata I32. The Interdata was on the ground floor.

They'd write a bit of code, take it up to the 5th or 6th floor to the conversion hall, then jump back downstairs to the ground floor where they had the Interdata, of course the new code wouldn't work, so back up a flight to their offices to write it again.

'There's gotta be a better way', Steve Johnson told Dennis in the stairway. 'There's gotta be an easier way!'

Dennis' first C compiler used recursion straight through. Steve was new to the labs, but he'd been poking around at that Unix thing and was of the opinion that a lot of the code would actually be portable. With Dennis' help, Steve built the Portable C Compiler Machine pccm, quite a wonder. It ferreted out instructions suspected of being machine-dependent, and it was also self-optimising, automatically choosing the branches that took the fewest clock cycles. 94% of the code was portable, Steve concluded, and, with the pccm, Unix really got off the ground.

Dennis pulled in Brian to write the book on the C Programming Language. Brian was very keen. That book has got to be the most read book in all of Computer Science. The earliest editions were the best - incredibly compact, written in Brian's unique style, kindly but also without mercy. You might think his book was too difficult, but Computer Science is that difficult, and once you get accustomed to the terrain and the application domain, you're OK.

Dennis and Ken initially wrote their system in a language called 'B'. Ken disappeared into his office to write a Fortran compiler and came out with B instead. B was based on BCPL, from Martin Richards. BCPL in turn was based on the Algol family, largely a set of theoretical languages. B was interpretive.

Brian tells the story.

'So we all assembled in and outside Ken and Dennis' office for the grand opening. Ken starts up the console. Everything is fine. Now Dennis is supposed to login to a console. The system just stands still - it doesn't have the processing power. So I quip 'well one thing is for sure - you can't call this Multics anymore - you'll have to call it Unics - a multiuser system for at most one user!'

They kept the name. They changed the expected 'cs' to an 'X' and history was made. And always there, like an uncle, was Doug McIlroy.

'We need to be able to pipeline programs together', wrote Doug already in 1964, according to Brian who claims he still has the actual note Doug wrote: 'What We Want To Do'.

'Make each program do one thing well', he told the boys. 'To do a new job, build afresh rather than complicate old programs by adding new features.' And so forth.

'Expect the output of every program to become the input to another, as yet unknown, program', Doug said. 'Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.'

Doug wanted them to work on 'connecting programs together just like garden hose'. And, lastly for now, as Brian himself noted:

'Having smaller programs is better than having massive programs.'

For that's the key.

Various online encyclopaedias credit Doug with the development of the pipeline, but that is likely apocryphal. Ken and Dennis wrote it, initially using '^' as the pipe character rather than the '|' used today. But Doug wrote some fantastic software of his own, most notably diff.

'diff is the quintessential example of not giving up on an algorithm until you get it right', Doug wrote at the time. If you've seen Apple's dev tool FileMerge then you'll know. You can skip to the middle of a monster text file and insert or remove a single character and the program will find it immediately. diff later became the basis of SCCS, the Unix Source Code Control system, and almost all such systems are based on Doug's idea, if not parts of his actual code. You don't save entire versions when archiving: you save the differences between your new version and the previous one. It's likely that sites like Wikipedia use something similar.

Everipedia has a page on Doug.


'Malcolm Douglas McIlroy (born 1932) is a mathematician, engineer, and programmer. As of 2007 he is an Adjunct Professor of Computer Science at Dartmouth College. McIlroy is best known for having originally developed Unix pipelines, software componentry and several Unix tools, such as spell, diff, sort, join, graph, speak, and tr.'

'His seminal work on software componentisation makes him a pioneer of component-based software engineering and software product line engineering.'

'He taught at MIT from 1954 to 1958.'

'McIlroy joined Bell Laboratories in 1958; from 1965 to 1986 was head of its Computing Techniques Research Department (the birthplace of the Unix operating system) and thereafter was Distinguished Member of Technical Staff.'

'From 1967 to 1968, McIlroy also served as a visiting lecturer at Oxford University.'

'McIlroy is a member of the National Academy of Engineering, and has won both the USENIX Lifetime Achievement Award ('The Flame') and its Software Tools award. He has previously served the Association for Computing Machinery as national lecturer, Turing Award chairman, member of the publications planning committee, and associate editor for the Communications of the ACM, the Journal of the ACM, and ACM Transactions on Programming Languages and Systems. He also served on the executive committee of CSNET.'

Doug was a computer scientist before there were any computer scientists. Back then they came from departments of mathematics or electrical engineering. Although Ken Thompson and Dennis Ritchie get credit for creating Unix, the system in use by most of the world today, it's the grey eminence of Malcolm Douglas 'Doug' McIlroy that helped make the difference.

Doug took quite an interest in Linux in latter years, often heard to complain at the 'bloat' in that system. Which is ironic, considering why and how Linux was born. University student Linus Torvalds in Helsinki Finland longed to have a simpler, more powerful Unix to work with, petitioned his university to purchase a source code licence, was turned down, was unaware there was something called 'BSD' or 'FreeBSD' out there, and so began looking for people interested in writing this new 'lean and mean' Unix from scratch.

Doug McIlroy also dropped this immortal line on the Linux community:

'The day Linux gives the Mac diagnostic that you can't open a file because you don't have the application is the day Linux becomes non-Unix.'

And that's particularly relevant today. For whatever the woes and growing pains of Linux, they're nothing compared to what Apple's offshoot of FreeBSD is going through.

The impeccable framework classes brought over from NeXT back at the end of 1996 are no more. They've scrapped the programming language Steve Jobs bought from Brad Cox, and the monumental architecture, credited to Avie Tevanian and many others, is today a shambles.

Worst of all - symbolic of the ruin - is the following API, belonging to the once great and pivotal class NSWorkspace.



Surf to that page. Note that, although it's valid for all versions from 10.0 to 10.15, it's been deprecated.

openFile:withApplication: gives you the opportunity to choose a file to open and to choose which application to use to open it.

Scroll down that page. How many times do you see the label 'Deprecated'? It should be at least six.

Do you think the wider Unix community do this with legacy Unix code? Do you think Apple's brethren at Microsoft do this with the Windows API?

Look at the available functionality (methods) on this page. The only one that's not deprecated is openURL:. Today everything has to be in 'URL' format rather than filesystem path format (a simple UTF-8 character string) because... Well, 'because'.

Normally, when you deprecate something, you do it after considerable aforethought, and only when it's absolutely necessary. And it's rarely necessary. But at Apple they do it all the time. They rip out the rug from under myriad software products - and then, just like here, offer no replacement.

How is one to choose which application to use to open a given file? Can one open a SQL file with an ordinary text editor? Or a hex editor like Hex Fiend? Or to check for legible strings as with our Xstrings? Or to check attributes with our Xattrib? Or to check access control entries with our ACL?

No. You're not supposed to do that anymore. You double-click on a file and it will open for you. You will always have a default application to open any file. (Most often TextEdit - much good it will do you.) If you want to open a file any other way, you'll have to walk around the block and back again. And most likely this will involve their Finder, which is where all the good code from NSWorkspace went off to.

Except of course that this is not the way to go about it, and the Apple engineers should know that (and probably do). Except that Finder is the - by far - most awful, lacking, pathetic, powerless, and feature-deficient 'file manager' in the world. It won its infamous epithet - 'TFF' - justly.

But this is no accident. Look around at the online documentation. Yes, it's riddled with sloppy mistakes and omissions, but it also tells a vivid story.

Long ago, in the mists of the past history of Computer Science, a cinderella startup in California was in trouble. Their technology was outdated. They were using a paraplegic programming language like an obsession. A language that most universities had wisened up to. They had huge problems in their production line. Schools in the area, long amongst their most loyal customers, were thinking of abandoning them.

Their programmers wrote code in CAPS that looked more like COBOL than a modern procedural language capable of context-free parsing. They had ambiguous I/O routines and severe memory limitations. And now they needed to take the step up to protected memory 32-bit code but were getting nowhere fast.

They tried to buy BeOS but found the price too high - ironic, as what they finally punted for was over two thirds as much again. But they bought it - perhaps the pain was lessened because the chief seller got his $200 million in stock rather than cash.

Consultants were called in to teach their programmers the C Programming Language - you know, the language Brian presented to the world. From there they made the slight step up to Brad Cox' Objective-C, which was now owned by the seller and thereafter by them.

And this was all based on something known as FreeBSD, one of the systems Linus Torvalds hadn't heard of, an offshoot of BSD, which Unix creator Ken Thompson set in motion with Bill Joy and others at the University of California at Berkeley.

Here was the synergy: the company renowned for 'user friendly' marrying the de facto most robust popular system on the planet. And yes, it caused quite a stir. The things that could have been...

But, inside the secretive walls of their offices at One Infinite Loop in Cupertino California, things changed in appearance only. Deep down inside, the rank and file never warmed to the takeover, or to their new bosses, or to that new system.

They had friends on the outside supplying them with ammunition. Let's keep those creator codes, said one, and they brought it to their meetings. Never mind that the whole concept was hopelessly under-dimensioned. Can't we just go back to the old days?

But they're ruining the code they paid 400 million for, said another, and yet another said - quote - 'yes they are but let them continue, let them keep fighting the good fight'.

And gradually, piece by piece, inch by inch, micrometer by micrometer, they edged their way back again.

They must be extraordinarily happy.

Somewhere along the line, they lost the enthusiasts. They lost the people who marveled at the tools that came from NeXT and Unix. They marveled at the great Project Builder. They saw the incredible Interface Builder as the 'missing link'. And they appreciated that the engineers at NexT who'd designed those programs really knew what they were doing, really understood how to make really intelligent interfaces, unlike today. They tinkered. The road ahead was open to them, just as it had been in the 1970s, the 1980s, and the 1990s. Unlike today where everything is made not for inventive programmers but for piecemeal factory workers. The Black Art of Programming as Post-Modern Slavery v2.0.

Mark Pilgrim wrote about this long ago. Tinkerer's Sunset. The creativity is gone. The only ideas to come along now will come from on high. And you'll just have to deal with it.

They strangle the market, squeezing every last drop of ingenuity and life-affirming energy out of developers and users alike.

Remember what Malcolm Douglas 'Doug' McIlroy told the Linux people.

'The day Linux gives the Mac diagnostic that you can't open a file because you don't have the application is the day Linux becomes non-Unix.'

About | ACP | Buy Stuff | Forum | Industry Watch | Learning Curve | Search | Test Drive
Copyright © Rixstep. All rights reserved.