About | ACP | Buy Stuff | Forum | Industry Watch | Learning Curve | Search | Test Drive
Home » Learning Curve

Apple and Open Systems

A rise, a peak, a fall?

Get It

Try It

All good systems are open systems. This is without exception today.

Things weren't always this way, but they are today. More and more it's become obvious that the only systems that can be trusted are the open systems, and Ken Thompson showed us how even that may not always be enough, but 'open' at least gives users a fighting chance, where 'closed' is tantamount to totalitarian. Close the system - push to the gates of the walled garden - and people will run for it. Run for their freedom. As they should. As they must.

The original Unix, although not open source at the time, had a complete user manual that fit comfortably in an attaché case. The total number of lines of code was 13,500 - about the same as Microsoft's WINFILE for Windows 3.1 twenty years later. Graphical user interfaces are exorbitant, adding 400% more code to a project.

But that doesn't explain everything. A study of call graphs for comparative web servers showed us the rest, with Microsoft exposed as the most ridiculous software vendor in the world.

Apple had a great advantage twenty years ago. Right now they're betting their floricultural system on things continuing as before, even as they try to change the rules like never before. Yes, push will lead to shove. And, if history teaches us anything, it'll end in a backlash that history would rather forget.

IBM, the venerable 'Snow White' of the industry, found open source years ago. Their mainframes still run proprietary code, but that code can cradle Linux. Microsoft's foray into open source, after harassing both IBM and Linux for so many years, is feeble in the extreme, and the epithet 'fanboy', for those who still defend the Redmond monster, fits nicely.

Apple inherited a FreeBSD system from NeXT. The decision by NeXT to go with FreeBSD was a good one. Why waste time maintaining a system that others already maintain? But then Redwood City came to Cupertino and things changed radically.

It doesn't take five years to redo a system that's already being successfully marketed and used by corporations like Dell and WorldCom. It doesn't take even one night.

Apple made a feeble attempt to keep their new OS 'open' - it would have made sense if it had been done correctly and honestly (which it was not). FreeBSD already has a team to maintain the code. And FreeBSD is so good that Linus Torvalds admitted he wouldn't have embarked on his own project if he'd known about it.

That Unix is relatively stable and secure is the result of open vetting and years of testing in the real world. Apple's skunkworks don't qualify in that regard, as we've seen time and again. It's an added cost, opening the system to vulnerabilities that should not exist, and it's an additional expense that can't be defended.

Add to that the resulting hostility in the wider community.

Apple's iPhone changed everything. A company weeks away from receivership, they morphed into the richest corporation in the world, able to change the name of the game and make their own set of rules for this new game - up to a point at least, as monopolies too suffer from market constraints.

Open Darwin fell apart because no one at Apple gave a shit. Curator Rob Braun declared it had only been a PR stunt anyway. Parts of Safari went open, as the code came from from another open source project, but the system itself remains closed, and now it's increasingly antagonistic. The low-hanging fruit of a future where even third-party software is under their total control: that's the wet dream that's obsessed them for more than ten years.

But all good things must come to an end, mustn't they? Apple created the smartphone market, and naturally led it to start with, but the competition soon took over. Still, thanks to planned obsolescence and high prices, they were able to profit handsomely.

Until the market finally peaked.

The market was saturated and started to dry up. Apple saw this moment coming long before the pundits, despite its inevitability. Sales statistics are no longer published. Apple approached Hollywood. And they returned to their base OS, the one they'd relegated to picking up crumbs from the iOS table of feast. The time had come to start exploiting the desktop OS, the goal being to recuperate a steady flow of billions in annual revenue by turning free developers into indentured serfs.

Locking the gates to the walled garden may be a step too far, it may result in a backlash that ruins the remains of a crumbling empire, but can't people be induced to come inside of their own volition?

The technical details of Apple's 'gambit' are easy to understand. They deliver the OS, their OS controls the launching of software, and so forth. That's the gates, that's the 'kill switch'. The gates can be closed at any time - the 'kill switch' can be delivered at any time - through an otherwise innocent-looking OS update. But the repercussions are perhaps too dire for this option to even be considered.

Operating systems and file systems need ways to protect themselves, their users, and their resources. Almost all systems give such matters due consideration, save for Microsoft Windows of course, where things are mostly being done as an afterthought to save face and financial loss.

Access control is the name of the game. In broad terms, there are two types of access control models: discretionary and mandatory. Microsoft's is of the former, meaning that their systems may impose access control but don't need to. Unix systems are the latter - or the opposite - meaning that access control is always imposed. It's of course possible to grant unlimited access to an object, but this occurs only within the access control model, not in spite of it.

Unix was designed to be able to protect itself and its users. Unix gives users the means to protect themselves from one another and from themselves, and Unix sees that it can protect itself from its users as well - and, by extension, from itself.

Unix isn't in the business of hiding things. Unix has a more than adequate security model. Concealment isn't part of that model, nor should it be, nor can it ever be, in the long run.

Malware writers on Windows, and to a lesser extent on Unix systems, can try 'camouflage' as a means of protection. The infamous Windows Registry, unfathomable to mere mortals and even to some entry-level programmers, is a great place to hide stuff. Popular applications such as Paint Shop Pro have used the Windows Registry in this manner. Windows configuration (INI) files are another good hiding place. Opera's web browser for Windows used WIN.INI to hide things.

Software vendors can use 'camouflage' on Unix systems as well. A file 'browser' uses a number of concealment methods. And preferences files can be used for more than preferences - by vendors of FTP clients. Other attempts at camouflage can be found here, here, here, and here.

'Camouflage' isn't protection, and it will ultimately fail, but perhaps it's regarded more as a speed bump than a full barrier.

Aside from instances such as the above, neither Unix nor software running on Unix tries to hide anything, and the Unix system never tries to hide. For hiding is futile. It's always possible, and often necessary, to restrict access to system resources, but hiding the fact that such resources exist isn't part of the game, nor was it ever meant to be.

The dot file - a file whose name begins with a 'dot' - isn't really trying to hide. It's just politely kept out of the way in default file listings, unless a command is invoked with special switch. Apple's Finder has a similar option, where everything, including dot files, can be exposed issuing the command:

defaults write com.apple.Finder AppleShowAllFiles -bool YES

How Windows Protects

Can Windows protect its users? Can Windows protect itself? The short version: no it cannot. Standard file attributes say nothing about ownership or permissions. Access control lists, introduced by the Dave Cutler team for Windows NT, provide discretionary access control only, and are an arcane exercise.

How Unix Protects

Unix uses a base system of three permissions for three groups of users. In almost all cases, this is more than sufficient, and was used for years as the only protection scheme, even as the popularity of the platform grew into what it is today, all-pervasive on the Internet and the 'WorldWideWeb'.

The three permissions are Read, Write, and eXecute. The three user groups are owner or user, one of the owner's groups, and everyone else - 'other'.

Because execute permissions also apply to directories, directory owners can effectively restrict access to disk areas and thereby prevent malicious attacks. Check closely where your Unix system has its 'bin' directories and what permissions they and their parent directories have.

Unix systems of today can also have so-called 'file flags' to further augment control. Your root directory, for example, should be marked with a 'sunlnk' flag, meaning that your root directory cannot be unlinked, even by the root account, other than in single-user mode.

Going through a Mojave root reveals an assortment of 'sunlnk', 'hidden', 'uchg', and 'restricted' flags on subdirectories. The flags 'sunlnk' and 'uchg' are designed to protect. The flags 'restricted' and 'hidden'? Not so much.

The flags 'restricted' and 'hidden' are not 'Unix'. They're 'Apple'.

ls -@aCeilO
total 37
         2 drwxr-xr-x@ 31 root     wheel  sunlnk             992 Aug  2 22:11 .
	com.apple.backupd.unlockrecordUUIDs	  84
         2 drwxr-xr-x@ 31 root     wheel  sunlnk             992 Aug  2 22:11 ..
	com.apple.backupd.unlockrecordUUIDs	  84
    647151 d--x--x--x   9 root     wheel  uchg               288 Mar 11  2019 .DocumentRevisions-V100
        19 dr-xr-xr-t@  2 root     wheel  uchg,hidden         64 Jul 15  2017 .HFS+ Private Directory Data?
	com.apple.FinderInfo	  32
4320874042 -rw-r--r--   1 root     wheel  -                 1188 Aug  2 22:11 .OSInstallerMessages
    440239 drwxr-xr-x   2 root     admin  -                   64 Jul 15  2017 .PKInstallSandboxManager
    618010 drwxr-xr-x@  3 root     admin  -                   96 Sep 20 14:45 .PKInstallSandboxManager-SystemSoftware
	com.apple.rootless	   0
    624104 drwx------   5 root     wheel  -                  160 Jul 15  2017 .Spotlight-V100
4310250579 ----------   1 root     admin  -                    0 Aug 17  2018 .file
    624255 drwxr-xr-x   2 root     wheel  uchg                64 Mar 21  2019 .fseventsd
4310250580 drwxr-xr-x   2 root     wheel  hidden              64 Sep 24  2018 .vol
4310250581 drwxrwxr-x+ 44 root     admin  sunlnk            1408 Aug  2 22:10 Applications
 0: group:everyone deny delete
4310361351 drwxr-xr-x+ 63 root     wheel  sunlnk            2016 Sep 24  2018 Library
 0: group:everyone deny delete
4310396148 drwxr-xr-x   2 root     wheel  hidden              64 Sep 24  2018 Network
4310396149 drwxr-xr-x@  5 root     wheel  restricted         160 Sep 21  2018 System
	com.apple.rootless	   0
 0: group:everyone deny delete
    420280 drwxr-xr-x   4 root     admin  -                  128 May  1  2010 Users
    420284 drwxr-xr-x   3 root     wheel  hidden              96 Sep 21 07:12 Volumes
4310945599 drwxr-xr-x@ 37 root     wheel  restricted,hidden 1184 Jul 24 23:08 bin
	com.apple.rootless	   0
4310945670 drwxrwxr-t   2 root     admin  hidden              64 Sep 24  2018 cores
       305 dr-xr-xr-x   3 root     wheel  hidden            4308 Sep 15 22:11 dev
4310945672 lrwxr-xr-x@  1 root     wheel  restricted,hidden   11 Sep 24  2018 etc -> private/etc
	com.apple.rootless	   0
         5 dr-xr-xr-x   2 root     wheel  hidden               1 Sep 21 23:31 home
4310945673 -rw-r--r--   1 root     wheel  hidden,compressed  313 Aug 18  2018 installer.failurerequests
         3 dr-xr-xr-x   2 root     wheel  hidden               1 Sep 21 23:31 net
4310250552 drwxr-xr-x   8 root     wheel  sunlnk,hidden      256 Sep 24  2018 private
4310946809 drwxr-xr-x@ 64 root     wheel  restricted,hidden 2048 Jul 24 23:08 sbin
	com.apple.rootless	   0
4310946911 lrwxr-xr-x@  1 root     wheel  restricted,hidden   11 Sep 24  2018 tmp -> private/tmp
	com.apple.rootless	   0
4310946912 drwxr-xr-x@  9 root     wheel  restricted,hidden  288 Sep 21  2018 usr
	com.apple.rootless	   0
4310982149 lrwxr-xr-x@  1 root     wheel  restricted,hidden   11 Sep 24  2018 var -> private/var
	com.apple.rootless	   0

How Apple's Unix Hides

But as can be seen already above, Apple's Unix does more than protect - it hides. And hiding is not a good thing. Not ever.

AppleShowAllFiles isn't available as a menu option. You have to know this possibility exists, and it's not exactly being broadcast across the Internet. It obviously is there so Apple's people can get at it. If they wanted their users to get at it, they'd have it available as a menu option. But they don't.

Apple's 'Finder' (it's supposed to be a file manager) won't even reveal the third 'execute' permission bit, nor will it reveal access control entries or file flags - nor is it capable of reaching all destinations in the file system, 'AppleShowAllFiles' or no.

The directory listed above as '.HFS+ Private Directory Data?' doesn't have a question mark ('?') in its name, but is rendered as such by the Unix command line, because the final character in its name is the unprintable '\r', the 'carriage return' character, dating back to the days of teletype machines.

The directory is used by the system backup facility. In this case, Apple took things a step further, trying to make access impossible (at least for mere mortals).

This is akin to the system in use for many years to hide 'multi-linked files', a system that went even further to obfuscate: in that case, .DS_Store was employed to keep the display coordinates in the tens of thousands so no computer screen anywhere would ever find it.

The same goes for directories found under /private/var/folders. The names here used to begin with a 'dash' character ('-') in another attempt to thwart access, even for those who have access rights and perhaps reasons for being there.

Apple go even farther by perverting the low-level API getdirentries(). This API, a standard part of Unix, is supposed to be used to prune directory enumerations for the higher-level API readdir. What Apple did was - under file system level - mark files with the bogus inode value '0' ('zero') used in Unix to mark files scheduled for unlinking by the 'lazy write' system.

They did this so you wouldn't know that certain files actually existed. (The 64-bit version of getdirentries() is no longer available to user-land code, another attempt to hide things.)

The user directory ~/Library is today marked with the Apple-specific file flag 'Hidden'.

The file flag 'Hidden' is specific to Apple. This is a replacement for the old '.hidden' file used in directories on HFS file systems. The file flag variant works on all file systems, even the new APFS.

The file flags 'Restricted' and 'Tracked' are also specific to Apple. Their exact portent hasn't been broadcast either, but neither of them bode for more 'user-friendly'.

Access control lists can be used to augment protection but so far don't contain mechanisms to hide anything. File flags seem to rank higher than access control lists anyway.

Apple's extended attributes (XAs), on the other hand, can indeed be used to hide (and more).

Previous experience with their iPod showed Apple that, even if some things that have been hidden are suddenly exposed, and even if the barrier is no longer insurmountable, the overwhelming majority of users will either never learn how to gain access or never have the guts to try.

A key part of Apple's proposal to get record companies to sign off on iTunes was convincing the companies that file-sharing would not be a problem. Apple had to make it difficult for iPod owners to find their downloaded songs on their own devices. This was accomplished by making the core directory iPod_Control invisible.

And so forth. If history has anything else to teach us about Apple and their customers, it's that those customers can be fooled into almost anything, and that Apple are seldom adverse to taking advantage.

We will never do music. We will never do video iPod. We will never do phone. We will never go Intel.
 - '9toX'
For you Apple apologists claiming Apple will never lock down the Mac, step one is in place and you all let it happen.
 - Ron Gilbert

About Rixstep

Stockholm/London-based Rixstep are a constellation of programmers and support staff from Radsoft Laboratories who tired of Windows vulnerabilities, Linux driver issues, and cursing x86 hardware all day long. Rixstep have many years of experience behind their efforts, with teaching and consulting credentials from the likes of British Aerospace, General Electric, Lockheed Martin, Lloyds TSB, SAAB Defence Systems, British Broadcasting Corporation, Barclays Bank, IBM, Microsoft, and Sony/Ericsson.

Rixstep and Radsoft products are or have been in use by Sweden's Royal Mail, Sony/Ericsson, the US Department of Defense, the offices of the US Supreme Court, the Government of Western Australia, the German Federal Police, Verizon Wireless, Los Alamos National Laboratory, Microsoft Corporation, the New York Times, Apple Inc, Oxford University, and hundreds of research institutes around the globe. See here.

All Content and Software Copyright © Rixstep. All Rights Reserved.

John Cattelin
Media Contact
ACP/Xfile licences
About | ACP | Buy Stuff | Forum | Industry Watch | Learning Curve | Search | Test Drive
Copyright © Rixstep. All rights reserved.