|Home » Learning Curve
Files Folders & Documents: Organisation/Adaptation
Keep things separate.
TRIESTE (Rixstep) — There's a good piece, featured again at MacSurfer, on how best to organise files.
It's a bit wordy, but it's a big topic. It starts with:
'If you're reading this, there's a good chance your computer is a mess.'
Yes, it's addressed to people who have files all over the place - all over their desktop (~/Desktop) and all throughout ~/Downloads. Of course that isn't you, or any of us, of course not, but still and all...
'Crammed with app installers from two years ago...'
Yup. Great language.
So what does the author, Harry Guinness, have to say about it?
Quite a lot actually. But he's methodical.
'Your first priority, then, is to implement a system you can actually stick to.'
Decide on a Structure
Decide on a structure, says Harry. Absolutely. Where would we be without /bin, /usr/bin, /sbin, /usr/sbin? Nowhere at all. Decide on a structure - easy enough, right?
'There are three main ways you can structure your file system: project or client-based, date-based, and file type-based', says Harry. To add: if it's file type-based you're after, you probably don't have to organise too much, as sorting by extension will take care of that for you. (Your file management suite can of course sort by extension?) (But Harry means something slightly different with 'file type-based' - see below.)
Here's where Harry gets in a bit of hot water.
He first explains client-based organisation. As in subdirectories for each client, then files in projects which become subdirectories of the first subdirectories. And so forth and so on.
And presumably the odd hard link will take care of crossovers...
'What makes a project or client set up work so well is that it's brainless. If file A is to do with client X, it goes in folder X. If file B is to do with client Y, then, shockingly, it goes in folder Y.'
True, Harry, but that only reasonably works if the clients don't amass too many documents. Otherwise you'll need further subdirectories, per project, or per fiscal year, or something. No?
'If you've got multiple projects for the same clients, you can either give each project its own top-level folder or have individual project folders within each client folder.'
Yes, and this should be so obvious for anyone already capable of managing clients. No?
Harry finds something to trip things up.
'Where a project or client-based file system starts to fall apart is when you deal with a lot of general files that have to do with multiple projects or the organisation as a whole.'
Then something far worse.
'The other time you might run into difficulties with a project or client set up is when there are lots of different files so each folder is a total mess.'
You mean like the desktop? Or the downloads folder?
For if your business is good, you'll get there sooner or later. For which Harry has a further solution.
Date-based - like an accountant. Each date gets its own directory. From there you can branch further in clients or projects. Or, for that matter, you can start with clients or projects, then create subdirectories with dates under them.
(Surely by now even the distracted reader sees where this is going, where it begs to be taken?)
'With a date-based structure, you normally have a folder for each year with a subfolder for each month.'
Ah. Sounds good!
'Depending on how many files you work with, you can also have further subfolders for each week although it's probably over kill [sic].'
'The nice thing about a date-based structure is it makes it very easy to find files from a certain period, for example, to look at last year's financials for January.'
Thank you, Harry.
But a few caveats.
'Unless you've got a large number of similar files then it's overkill and you won't be bothered to stick with it. Also, it doesn't work very well if you're working on the same file for an extended period of time.'
This is all true.
File Type-Based Organisation
Harry's final organisation type is 'file type-based'. He divides things in such case into, for example, marketing, presentations, and financials. He cautions that this organisation is not good at an upper level.
'Again, think about what kind of work you do. If it's just a few things over and over again, then a file type method of organising folders might be right for you. Otherwise, stick to using it for subfolders.'
All good advice. From someone who's given the matter serious thought, someone who's had to give it serious thought.
But let's compare the challenge with that of devising the filesystem layout for, say, an operating system. With an operating system, you know what files and file types you'll have on board, what 'projects' they belong to, and so forth. Once you've finalised your operating system (and it's taken you some time, and you've undoubtedly been moving files helter-skelter all over the place) you don't need to worry too much anymore. You have macros or environment variables (or tools such as whereis) as reference for particular locations. Your system is mostly static from that point onward. No, nothing is 100% perfect, you'll be adding features as time goes on, but 90-95% is etched in stone (knock on wood). Your system software will know how to find files, how to refer to them. You won't have to know yourself. You're OK in that regard.
Much of the above is what people run into when trying to organise their correspondence. Instead of cluttered desktops, we have cluttered inboxes. And so forth. Users create folders to store the correspondence they want to save. And they ultimately run into the same snags as Harry. For which method is best? Project-based, client-based, date-based? Or why not something else that no one's even thought of before? What the weather was like the day you got the contract? Whether the Yankees won or lost that day? What suit you wore to work? What tie? Anything at all. The problem is you have three ways to organise - three very good ways, it must be pointed out - and that's it. You begin creating your directory structure. Now it's etched in stone.
What happens if your original design fails? Harry's piece is peppered with innuendoes and hints that these designs can, in many cases, ultimately fail. What do you do then? A week or a whole month of drag-and-drop?
It's likely Paul Buchheit thought of this close on 20 years ago. He designed Gmail, which, in addition to utilising the message ID field for the first time to create 'conversation threads', dispensed with the older 'folder' paradigm and gave things labels instead.
The internal architecture of Gmail must be fascinating. It takes a few seconds to load things, probably because the label layout isn't completely known until the account is accessed. But once you're inside, and you can look around, you see why Buchheit chose labels over folders.
True, Gmail today admits of a limited 'hierarchy' when it comes to labels, but this can be as illusory as the original non-hierarchical Macintosh. The main thing is it's labels that rule, not folders. Labels that are (mostly) not hierarchical.
You can tag a message with anything you like. You can make up a new label on the spot. You can remove labels. The files are not affected.
The files don't reside inside labels as they would do with the traditional directory structure. The files are just there. Add labels, rename labels, remove labels, from individual messages/files, or completely: it's all the same. The files aren't touched.
Nor will it matter in which order you apply a label. You can tag a message with 'Client 1', 'Jan 2018', 'Financials', 'Marketing', anything you like, in any order you like.
Finding is King
The key to a good organisational scheme is not only that it's tidy, but that it's easy to find things when you go looking later on. We may have 30,000 or more files in our user areas, but we're familiar with the organisation - we know where to go looking if we need to find something.
But situations like that can quickly get out of hand. Following a long assignment that lasted several years, we had terabytes of unexpected (and unwelcome) files with no attempt ever made to have them organised and sorted. And, long ago, we worked with staff at Reuters of London (before the 'New Millennium') to find fast and efficient ways to wade through millions of documents. It's not easy. And you learn a thing or two along the way.
First, you have to have good tools. By 'good' is meant 'fast', 'reliable', 'no BS'. You don't want doodads - you want to get the job done. By yesterday.
Second, you need an on-disk basic structure. To start with. The default macOS home area may be sufficient - you've already got Documents, Movies, Music, Pictures. And that about covers it.
Documents - text files, PDFs, spreadsheet, what ever. Text stuff.
Movies - just what it sounds like. Movies as in pictures that move.
Music - music (MP3, MP4, overlapping here).
Pictures - JPEG, PDF perhaps, PNG, TIFF, and so forth and so on.
And yes, get everything off the desktop. And out of ~/Downloads. Now let's skip back to Harry for a moment. (Just a moment.)
Harry's list of best practices is excellent.
√ 'Never ever store files on your Desktop. It just looks messy and cluttered.'
√ 'Don't let files sit in your Downloads folder. Either file them where they belong or delete them.'
√ 'File things immediately.'
√ 'Sort everything once a week.'
√ 'Use descriptive names.'
(Ah. There we go again.)
√ 'If you can't find a file by looking, try searching for it. If you've named your files and folders correctly, it will be easy to find.'
√ 'Don't use too many folders.'
What Harry says after that is only double-confirmation that the idea's in trouble.
'Top down systems are stupid; they rarely work. Start with one of the structures I recommend and then tweak it as you go. Adapt it to your workflow rather than trying to force your workflow to adapt to a rigid file structure.'
Yes, Harry, we know - but 'adaptation' with those paradigms means a lot of moving of files, of deleting directories, of creating new directories, only to... Remember Ali Ozer at the WWDC? How they wanted to RENAME new macros they'd only created a few months earlier?
So let's set up a little test hierarchy as Harry suggests. Let's mix directories for clients with directories for years and directories for months and see what we come up with. Granted that for Harry's business (he's an excellent photographer, check out his site) this might be sufficient. And it might be sufficient for you too. But how long will the system hold? How flexible is it?
OK, we've got 'Jan Financials' and 'Jan Marketing' in '1 - Jan' under '2018'... How do we organise according to client?
This method will work well enough for a one-man enterprise. But when things get going, when more clients drop in?
A few seconds of fooling around shows what we got - for a few seconds. We can now organise (and search for anything) according to year, client, financials, or marketing. We tag stuff, and in it goes - and the on-disk file hierarchy, the on-disk structure, doesn't really matter.
Better still: because this is a Mac, those labels will follow their files around wherever they go, as long as we transfer from one Apple filesystem to another.
You select a label (tab) on the left - just as you'd do in Gmail - and you immediately see all the items - folders, files, and otherwise - that share that label.
Labels (tags) are always textual. (Apple's colours of the rainbow won't get you far, wouldn't even get you to the end of Harry's 2018 calendar year.)
The above application - macTag - is only a suggestion. It's one way of - hopefully - many ways to accomplish this goal, to deal with the situation, the situation being, to put it simply and succinctly: we have too many files. Harry has his business to organise. Reuters needed to sort incoming mail in the millions. We have (or had, thanks very much) terabytes of unsorted file data.
And, as with Gmail, we don't have to etch anything in stone. We can change the 'structure' very easily. There is no hierarchy to speak of. Tags (labels) attach to files, not the other way around. Files and tags are separate. Whether it be your fancy to sort your files by year, month, day, or by client, or by file type.
Bottom line: there's no 'either/or'. Or any lugubrious backtracking. You don't have to choose between client-based, date-based, and file type-based. You can choose all three. At once. Or none of them at all. Or new methods. You've kept things separate.
macTag: 'This is how'
Learning Curve: Personal Storage, Google, Tags
Industry Watch: prMac: macOS Tag Editor Offers a Better Way
Stockholm/London-based Rixstep are a constellation of programmers and support staff from Radsoft Laboratories who tired of Windows vulnerabilities, Linux driver issues, and cursing x86 hardware all day long. Rixstep have many years of experience behind their efforts, with teaching and consulting credentials from the likes of British Aerospace, General Electric, Lockheed Martin, Lloyds TSB, SAAB Defence Systems, British Broadcasting Corporation, Barclays Bank, IBM, Microsoft, and Sony/Ericsson.
Rixstep and Radsoft products are or have been in use by Sweden's Royal Mail, Sony/Ericsson, the US Department of Defense, the offices of the US Supreme Court, the Government of Western Australia, the German Federal Police, Verizon Wireless, Los Alamos National Laboratory, Microsoft Corporation, the New York Times, Apple Inc, Oxford University, and hundreds of research institutes around the globe. See here.
All Content and Software Copyright © Rixstep. All Rights Reserved.