About | ACP | Buy | Industry Watch | Learning Curve | News | Products | Search | Substack
Home » Learning Curve

On File Management (1)

Part one.

Get It

Try It

File management is an endemic part of computer system maintenance. But on Apple systems - and on Microsoft systems since Windows 95 - it's been pushed behind the curtains in an attempt to make things appear simpler than they really are. Even the open source GNOME user environment attempts to emulate the Apple experience, pushing all files onto the 'desktop' and forcing seasoned users to re-learn how to use computer systems.

Yet all systems save notably Apple's will flip into an 'advanced mode' if the user so desires. And none of them save Apple's will attempt to force use of the default manager on users. File management interfaces on Linux and Windows are independent of their file managers. The file managers are merely 'pretty faces' on the APIs used. In the case of Windows it's the 'shell' dynamic link library.

Apple's OS X also has a set of underlying APIs but they're not complete. And Apple's default file manager has no 'advanced button'. Users can either take what they're given (and successively get into more and more trouble) or take matters into their own hands (most often with command line utilities). But the latter is always going to be unwieldy - and far from the ideal of totally GUI-based tools.

Unix adepts running OS X can always try file management from the command line; Apple engineers are probably doing it too; but the advantage of GUI-based tools in this regard is not about pointing and clicking - it's about the overwhelming advantage of the total overview a graphic output makes possible.

Almost all file managers recognise the fact that modern file systems are extensive and anything but trivial. Today there are already more directories than there were total files on older systems. File managers such as the 'commanders' don't cut the muster anymore (if ever). Users don't need two directories - they need to see them all.

Apple's current file manager is based on NeXT's. It uses the NeXTSTEP NSOutlineView for basic navigation. This provides a basic 'lateral' navigation that works well enough for file I/O dialogs but doesn't come close to being good enough otherwise.

The NSOutlineView can also be used to hold an entire file hive, directories and files both, but studies have shown that this design leads to resource-intensive and ultimately crash-prone application code. NeXTSTEP and OS X both lack the simple but powerful 'tree' control found in Windows and the Linux graphical shells.

The modern division of labour - seen as well in blog engines - offers a basic current listing against an overview of the file system as a whole. The basic user interaction offers the opportunity to jump to a new location with a single click, all the more important with 20,000+ such locations to choose from.

Modern file systems are not trivial. Modern Unix systems - whereof OS X is one - have more directories than there were lines of code in the original version of Unix (13,500). Things have moved far beyond the elementary /bin, /etc, and /usr of old.

'Ordinary' computer users won't often need to venture beyond their home areas, but with 20,000+ directories out there with 100,000+ useful files, there's bound to be something they need. The CLIX command arsenal comes mostly from these areas - and was made possible by using a file management system that didn't try to hide it all away.

For OS X tools, unlike those found on other platforms, attempt to hide the complexity of the file system from the user. More: they attempt to thwart attempts to 'get out there' even if said user discovers the home area is really a 'walled garden'. Various mechanisms have been used by Apple over the years to make those 20,000+ directories with their 100,000+ files invisible and unattainable.

Early OS X systems used the 'dot file' .hidden to store lists of file and directories in each location that should not be listed by Apple's one and only file manager. But there was (and is - and remains to this day) an 'advanced button' of a sort.

default write com.apple.finder AppleShowAllFiles -bool YES

Crashing and restarting Finder makes the invisible files visible.

That's a step in the right direction, but it still leaves the user with 'TFF' to do major system maintenance with a kiddie toy. Apple's file manager is still unwieldy and lacks the ability to address most of the maintenance issues important to the harried user.

Following is a list of properties applying to all files and directories on OS X systems.

PropertyDescriptionApple Support?
File permissionsFile permissions are divided into three groups and are a combination of read access, write access, and execute access. The three groups are user, group, and other, where 'user' means the user who actually created and owns the file, 'group' is the default user account group assigned to the file, and 'other' is everyone else.

File permissions also include the 4th bit reserved for the 'sticky bit' and 'set ID' use.
Somewhat. Apple's Finder won't get to the 'executable' bit, meaning users can't enable script files they create. Nor can they remove executable bits they discover shouldn't be there. Apple's Finder doesn't even hint at the 4th bit. OS X users can have dangerous files on their system and will never know it.
File typeUnix admits of several file system file types. These types are not modifiable but knowing of them can be of vital importance to users and file managers both.None. Neither is there much protection in the application (or the system per se) against mishaps caused by mixing these file types on operations.
File flagsThese flags apply to files in addition to (and overriding) the ordinary file permissions. They're used to lock and protect files in various ways and today used by Apple to still keep files and directories hidden.None. Apple have today moved their 'hidden' attribute into this collection but otherwise none of it is available.
Extended attributesExtended attributes are a mostly innocuous (and potentially very useful) enhancement to file attributes. Apple are still using them in two cases: the archaic 'Finder info' (which is used amongst other things to place and hide files) and the somewhat arbitrary 'resource fork'. The API is now generalised and third party houses can use it to enhance the user experience on OS X. (Note: this API doesn't translate to other Unix platforms.)None. Apple will use existing information but there are no provisions for accessing it or modifying it. Command line capabilities exist but are severely limited.
Access control entriesAccess control entries are a powerful (and dangerous) alternative to ordinary file permissions. The verdict is still not in on how good they are (and may never be) but they exist, and to the extent they're already in use, the user needs to see and control how they're used.None. Go instead to the command line and use new switches with chmod and ls instead. Good luck.
Time stampsUnix file systems traditionally have three time stamps for when files or directories were created, last modified, and last accessed. The 'created' field has often been used to indicate when status information (such as time stamps themselves) have last been modified. Starting with the latest versions of FreeBSD (and OS X) there is now a dedicated 'created' field. All fields are modifiable save for the 'status information' field.Somewhat. The fields are played down and none are directly modifiable.
File sizeThis goes without saying. But given how Apple attach extraneous data to files, not necessarily as simple as one might think. Files can be considered to have both a 'data' size (the actual file data) as well as an aggregate for all associated extended attributes. Users need to see when their files have extended attributes attached and they need to be able to access and modify these extended attributes.

'Black hat' use of these extended attributes has led to numerous system hacks over the years. These issues remain unresolved.
Apple display an aggregate size for both the 'data fork' and all existing extended attributes, including 'Finder info' and 'resource fork' data. Nothing turns up individually until files on an OS X system are copied to a non-OS X system; then suddenly discrepancies arise, often puzzling the user.
File inodeNot modifiable, the inode is assigned at file creation time. The inode is an unsigned integer of a size commensurate with the native integer size of the underlying system. New 64-bit versions of OS X can therefore have 64-bit inodes. Inodes are assigned when files are created and left unused when files are removed. The file system driver will fill the empty slots only when overflowing and starting from zero again. Inodes are important because Unix systems admit of multiple links to the same physical file (hard links) and the inode together with the device number is the arbiter of identity on Unix file systems.Absolutely nothing. Apple like to play down the significance of the inode, in part because they were never able to twist their HFS into full Unix mode for this vital feature.
Number of linksThis correlates to the above ('inode'). The number of links shows the number of actual file paths associated with the same physical file (multiple links). This field can be used to see how many hard links need to be found for the same physical file. Apple's file system driver has a design bug whereby directories show a value corresponding to the HFS 'valence' rather than the number of links. (Unix typically will not allow multiple links on directories to protect against endless recursion; Apple have lifted this prudent and otherwise universal guideline to implement Time Machine. Cross your fingers.)The same as above. This is rocket science to most Apple engineers and totally bewildering to their so-called 'user experience engineers'.
File ownerWho created the file. Only the superuser can change this value. The name is an arbitrary value for a signed integer denoting the user ID.Yes. What capabilities are offered for changing this field (even with proper permissions) can vary over time.
File groupThe group assigned to the file. Only the user and superuser can change this value. The name is an arbitrary value for a signed integer denoting the group ID.Yes. What capabilities are offered for changing this field (even with proper permissions) can vary over time.
File device numberA unique integer denoting the 'device' (hard drive or virtual volume) on which the file resides.No. Apple user documentation (even for admins) won't touch this topic.
File device typeUsed exclusively by members of the /dev hive. Device type numbers are divided into major and minor fields.No. Apple's file system APIs and 'file manager' can't even get in there, much less make any sense out of it, this despite the hive being an endemic (and important) part of Unix since Day One.
Number of blocksThe actual disk real estate the file takes, expressed normally in 512-byte blocks. Blocks are the smallest I/O unit on the hard drive and a basic cluster (basic I/O on the system) is usually 8 blocks or 4,096 bytes.Indirectly. Apple aggregate file system use and round up things to the nearest cluster value. This gives an accurate portrayal of how much disk space is used but there is little further information.

About | ACP | Buy | Industry Watch | Learning Curve | News | Products | Search | Substack
Copyright © Rixstep. All rights reserved.