More File Stuff (2)

All the wonderful things you can do to a bunch of your poor innocent files.

Get It Try It

Three previous articles (here, here, and most recently here) discussed Unix file permissions, file ownership, how this applies to both files and directories, Unix file types, sticky bits, and set ID bits.

And that's the Unix end of it. Save for one possible final detail. The system and user flags.

These additional file access modifiers are accessible through the command line with chflags. The standard ACP file info sheet deals with them and so do Xattrib and Xstat. Some are found on other platforms, some are not. Some are found only on Mac OS X.

What They Do

The extended flags are commonly expressed in hexadecimal and are a doubleword (32-bit) value. This means there are eight (8) hexadecimal digits used to express them.

The leftmost four digits represent the higher values and 'system' settings whilst the rightmost four digits represent user settings with lower values. You can add and remove user flags at any time; you may try to escalate to root to set system flags; but system flags can't always be removed as easily - even by root. So be careful - Xfile will warn you; other programs may not.

What are these flags, what do they mean, and what do they do? They're all used to further protect files or limit their use.

User No Dump		The file cannot be dumped.
User Immutable		You can't modify the file no matter what the permissions say.
User Append		You can add to the file but you can't change what's already there.
User Opaque		Used to make directories opaque in certain circumstances.
User No Unlink		You can't unlink (almost the same as remove) the file.
User No Hidden		An Apple-only setting also found in the legacy fileinfo.
System Archived		The file's been backed up.
System Immutable		This is the system saying you can't change the file.
System Append		Same as for the user setting but again: it's the system talking.
System No Unlink		The system says you can't unlink (remove) the file.

Remember: you normally need to be root to set the system flags. And again: being root won't always be enough to remove them. The alert panel mentions something called 'single-user reset' - what's that?

Single-User Reset

Single-user reset is a reset performed in single user mode - it's like root but even more powerful. You hold down ⌘S as you reboot your Mac. You get a console mode screen instead (with a lot of jabberwocky). After a while the system settles down and the screen output comes to a halt. You're in single user mode.

There are no accounts in single user mode. There is no root and there is no you. There is only whoever's at the keyboard together with the file system itself. To do anything of any import at this point (and be really careful and preferably don't try this at home if you're not really comfortable with it) you first have to check to make sure your file system is healthy and then remount your file systems as writable.

The first of the two commands below runs the file system check program fsck and feeds it 'yes' answers for prompts it otherwise would give you as well as taking into consideration the fact you're running a journaling file system. The second command updates the mount on your root file system so it's writable.

This is what you have to do to get rid of system flags, so keep this information at hand or learn it by heart.

# /sbin/fsck -fy
# /sbin/mount/ -uw /

Once you've got this far, you then proceed to removing the system flags you couldn't do as 'only root'. This part is easy. Prepend 'chflags 0 ' to the path of your file (of course using quotes if the path contains spaces or other weird characters).

# chflags 0 'stubborn file.txt'

Now check back with the ls command. The extended flags (or a dash character if none are found) will appear between the group name and the file size.

# ls -lO 'stubborn file.txt'

Now issue the following command (hands off the keyboard) to bring your Mac back up the way you're used to seeing it.

# reboot

Access Control Lists

Access control lists are a mixed blessing. They afford the administrator more granularity and flexibility in meting out permissions on network objects but they can be bewildering as well, inasmuch as they can override ordinary POSIX file permissions (and mostly anything). Today there's a FreeBSD standard which Apple are borrowing from and the FreeBSD standard probably borrows from Dave Cutler's standard on VMS/WNT (where they can be sorely needed).

So if something is wrong with your files and you can't figure out why - it could be an ACL.

Xfile automatically alerts you about files with either extended flags or ACLs on them - the 'mode' field of the Xfile display then uses an 'alternate' colour. The standard info sheet reveals the extended flags (and lets you modify them). If nothing's out of the ordinary with those flags, you then try Rixstep's ACL utility.

Apple set access control lists on at least four directories on Snow Leopard, three of which are at root level: /Applications, /Applications/Utilities, /Library, and /Volumes. It's likely these ACLs don't add anything the extended flags couldn't do easier. But cut them some slack - everyone likes to play around with new toys.

You otherwise detect ACLs from the command line with the ls command using the switch -el. The entries (ACEs or access control entries) in an ACL are listed under their file names. First comes the index (because the order of ACEs is very important) and then comes the group to which the ACE applies. Then comes the type of ACE - either an 'allow' or 'deny' - and then the precise operations to be allowed or denied.

$ ls -el /
drwxrwxr-x+ 26 root     admin       884 Feb 16 21:39 Applications
 0: group:everyone deny delete
drwxrwxr-t+ 53 root     admin      1802 Jan 24 08:29 Library
 0: group:everyone deny delete
drwxrwxr-x+ 26 root     admin       884 Feb 15 19:45 Utilities
 0: group:everyone deny delete

Some caveats about access control lists. The file system looks to the ACLs first before anything else. And it looks only until it has an answer to its question - namely if you're going to be allowed to do what you want to do. The second ACE in the listing below has no effect whatsoever - the operation 'delete' has already been denied.

 0: group:everyone deny delete
 1: group:everyone allow delete

ACEs can be inherited too - making it rather difficult for administrators to see what's going on and who causes it.

Generally it's better to leave ACLs alone - and to studiously avoid giving yourself exceptional privileges in places you shouldn't really go (such as Snow Leopard's Finder is evidently all too keen to help you with).

Extended Attributes

Once upon a time Apple's file systems had forks. They had data forks and resource forks on every file. This made it cute and clever to associate a lot of cool 'Mac' stuff with your files. Then came OS X and changed things.

Once upon a time Apple had 'fileinfo' too. Fileinfo is sometimes known as 'Finder info' and there's a very good reason for this. Actually it should be called 'HFS+ info'. For this type of info resides not in the ordinary file system but in the HFS control blocks themselves.

Fileinfo includes very non-Unix things like colours, legacy file creator and file type IDs, pixel location in Finder windows, the coordinates of folders on screen - lots of lovely stuff that really doesn't belong in a system file control block anyway.

There's been a war raging for years over these two critters with people declaring victory when they had no clue what they were talking about and others just shaking their heads because those forks and fileinfo bits don't transport too well on the Internet.

But things started changing with OS X 10.4 Tiger, released 29 April 2005. Now all of that stuff falls into the category extended attribute and has its own Unix-like (but not POSIX) API. Starting in April 2005 it was no longer necessary to use legacy (archaic) Carbon APIs to get at (eradicate) this stuff. And a lot of this new technology has proven eminently useful.

Extended attributes are controlled with the Xattr utility. They're also accessible (and manageable) from the command line but not as easily and with no ability to see what's actually in them.

$ ls -@l
-rw-------@  1 rixstep  staff   545 Apr  1 02:00 h2->h1.xx
	com.rixstep.XX	  89

Files can still have fileinfo (denoted with the name 'com.apple.FinderInfo') and resource forks (denoted with the name 'com.apple.ResourceFork') but now they're managed by a new Unix-like API. What makes things really interesting is the fact the same API can be used to glue any extended attribute onto a file. Several ACP applications use this advantageously to store document-specific settings such as screen coordinates and table view column widths, something that wasn't possible before despite the fact OS X is supposed to be an object-oriented user environment.

Xattr can also be used to manually add extended attributes to a file. The command line xabatch tool can simultaneously add, remove, and replace extended attributes to a group of files all at once.

Extended attributes other than com.apple.FinderInfo don't affect file access (and even com.apple.FinderInfo can't affect file access with Xfile) but they're an important part of the OS X file system of today. Xfile displays sizes of files with extended attributes in an alternate colour; the standard ACP info sheet shows the full file size, the size of the 'data fork', and the total size of all extended attributes; whilst the Xstrings utility dumps data found in all these areas in different colour codings.

See Also
ACP Users: More File Stuff
Learning Curve: Files, Ownership, Permissions, Stuff Like That
Learning Curve: Files, Ownership, Permissions, Stuff Like That (2)