Rixstep
	`About \| ACP \| Buy \| Industry Watch \| Learning Curve \| News \| Products \| Search \| Substack`

Home » Industry Watch

CLIX, CSV, DSV, ESR & The Meaning of Life

CLIX 1.7.2.g - out now - represents a full update of this award winning program.

Get It

Try It

Database utilities have long been plagued with a supposed need to be compatible with 'CSV' - whatever that is. 'CSV' stands for 'comma separated values' and as such doesn't sound bad - but it gets bad when attempting to achieve 'cross platform compatibility'.

The 'yardstick' here, as it turns out, is none other than Microsoft. Time to flash the warning lights and sound the alarm.

And attempting compatibility with Microsoft - who unofficially hold the CSV standard - is futile: as CSV never was a reliable 'standard' anyway, Microsoft's own programs aren't compatible with each other and different versions of the same Microsoft program (eg Excel) will founder.

Clearly there has to be a better way of doing business. And when the alternative is Microsoft, it's a given.

And the answer is 'DSV', a Unix standard - and furthermore a standard that's been around for the longest time and is eminently stable. 'DSV' stands for 'delimiter separated values' and at first glance can appear to be a generalisation of CSV. But it's not. DSV takes the 'Unix' approach; CSV takes the 'Microsoft' approach - with easily foreseeable consequences.

As Eric Raymond points out in a discussion of data file metaformats, the system is eminently simple: a character, mostly at random, is chosen to be the field separator. (The colon is often used in Unix - see /etc/passwd.) If this character should be needed within a field, it is escaped with another character, most often the backslash ('\'). To input the escape character, just double it.

That's all there is to it. Would that all things were so eminently simple.

DSV the Microsoft Way

For reasons unknown to both acquaintances and experts in abnormal psychology, Microsoft decided to not use this eminently reliable and stable system when registering their RFC. When Microsoft's field separator is needed within a field everything goes pear shaped. Instead of doing the obvious and escaping it, the entire field is enclosed in double quotes instead.

Which of course gives rise to the next snag: what if the double quote character is needed within a field? Microsoft have an answer to that all right - double the double quotes - but of it Eric Raymond writes the following.

In fact, the Microsoft version of CSV is a textbook example of how not to design a textual file format. Its problems begin with the case in which the separator character (in this case, a comma) is found inside a field. The Unix way would be to simply escape the separator with a backslash, and have a double escape represent a literal backslash. This design gives us a single special case (the escape character) to check for when parsing the file, and only a single action when the escape is found (treat the following character as a literal). The latter conveniently not only handles the separator character, but gives us a way to handle the escape character and newlines for free. CSV, on the other hand, encloses the entire field in double quotes if it contains the separator. If the field contains double quotes, it must also be enclosed in double quotes, and the individual double quotes in the field must themselves be repeated twice to indicate that they don't end the field. The bad results of proliferating special cases are twofold. First, the complexity of the parser (and its vulnerability to bugs) is increased. Second, because the format rules are complex and underspecified, different implementations diverge in their handling of edge cases. Sometimes continuation lines are supported, by starting the last field of the line with an unterminated double quote - but only in some products! Microsoft has incompatible versions of CSV files between its own applications, and in some cases between different versions of the same application (Excel being the obvious example here).

CLIX 1.7.2g

Starting with version 1.7.2g CLIX uses DSV exclusively and drops all ambitions and pretensions to be compatible with a non-standard from Microsoft that doesn't even exist. Even within Microsoft. The result is better, leaner, and faster running code, but it also means all current CLIX users must update all their command files as well.

As the internal data model for all versions is the same, the way to update one's own commands is to copy and paste (or drag and drop) them from the old version to the new one. You can also use a stream editor or a text editor to replace all instances found of '\' with '\\'.

All currently packaged command files have already been updated.

1,410 Commands

ACP users currently have at least 1,410 CLIX commands at their disposal; users of the free version have somewhat less. In both cases the move to DSV means a vast improvement.

CLIX/Security Review

CLIX represents a safer and more convenient way to use the Unix command line. You can set TTY tickets and set your sudo time stamp timeout to zero and you'll still be all right - and comforted knowing you've thwarted the black hats from hijacking your computer.

CLIX accepts your admin passphrase like many applications, but unlike the rest your passphrase is never stored on disk. So when you exit CLIX, that passphrase is 'gone'.

But CLIX goes even further: as soon as your computer goes to sleep, CLIX clears your passphrase - and obliterates your eventual grace periods as well. Any hijacker intent on taking over your machine will be left with an unsympathetic prompt.

More Commands Than You Can Shake A Stick At

With 1,410 unique Unix commands, CLIX has it all - certainly more than any of the other incidental AppleScript system optimisers out there. If you still can't find what you want in CLIX, you can probably write it yourself.

For that's the genius of working with CLIX: it's extensible. You're not locked into a limiting complex of GUI doodads - how would you add a click box yourself? You add the commands you want all by yourself. And they work instantly, first time every time.

The Organiser

And because CLIX is an authentic native Cocoa application, you can make all the command files you want: you can organise the commands you need in files of your own. CLIX comes with a full dozen starter command files of its own - you can either edit these to suit your tastes or copy the commands you like to new files. Put the desktop enhancement commands in one file, the system and security commands in another file, and so forth.

Staring Down the Threats

CLIX 1.7.2d 'pk' had new commands to address the insidious 'input managers hole'. You can lock down your system so no trojan, no matter how clever, can ever again exploit you in this way - and all at the click of your mouse.

Rescuing Your Hard Drive

Your system may be riddled with way too many 'language project' hives, eating away at your precious disk space - a click of the mouse and again CLIX makes them gone. It can't be easier.

Discover the World Around You

A default OS X installation comes with 15,000 directories and nearly 80,000 files, most of which you never see. And navigating outside the harsh confines of your own backyard with your default system tools isn't going to get you very far. But with CLIX you can see it all - and use it all.

After all, you paid for it, didn't you? So find out what's there and learn to use it.

Free As In Beer

As always, CLIX 1.7.2g 'gee' is free as in beer. It's free because knowledge like this should be free - and must remain free. It is your right to know how to manage your computer and to be in control of it.

It's a labour of love from Rixstep done to make you a safer and happier OS X user.