Xbench

Spiny Software
Rating:@

Xbench - or Xstench as someone on a discussion forum dubbed it - is a program that's supposed to duplicate a lot of what Apple's own System Profiler already does - but for what purpose no one can fathom. Still, Xbench comes up embarrassingly short in almost every category.

After several intense minutes of disk crunching and swap abuse, the program produces a half page of numerical statistics, with a grade at the top for the overall performance - but there's no explanation for the relevance of any of the numbers shown. You can stare at the numbers all day long and not be the wiser.

The idea behind the program was that everyone would volunteer their statistics to a database held by the program author, so they could log in and compare (for what purpose no one knows), but the sorry fact is the author never succeeded in getting enough people to show interest in the project.

So the database never came online. And without that database, all the numbers the program spits out are that and no more: just numbers, meaningless statistics, and to quote Carl Sandburg, there are three kinds of lies...

The program opens with a tiny dialog in the upper left hand corner of your screen - a touch that is bound to annoy most people - and with a number of checkboxes for all the tests you want to perform. A sheet slides down to admonish you to turn off as many processes as possible so you get the highest possible rating. After that, it's just press the Start button and kick back and watch the movie.

The author puts your system stats in an outline view, dividing them up into different categories - unfortunately, he forgets to extend the final column to the right border of the window, which gives the program an ugly feeling - the question begged being 'why didn't he organise the UI better?'

As the tests continue, up again in the upper left hand corner of your screen, there is yet another window on top (the initial window is still there, it's just cluttered over by the two new ones) with a simple stop button - so you can stop the test at any time (but it's wisest to go to the end, at least once, so you see what you're getting).

About midway through this extravaganza the program opens yet another window and attempts to dazzle you with graphics. It performs a number of rudimentary Quartz operations to test your video card. There are line tests, and rectangle tests, and circle tests, and bezier tests, and text tests, and there are even OpenGL spinning squares tests - eye candy, but what does it all mean, wonders the innocent user?

No clue. It's just a window with some special effects, and no one to compare with.

Perhaps the worst crime comes when the tests are complete - it is then that the gargantuan mistakes of what should be a simplistic UI come to light.

For it is then that you notice that this programmer has gone to the considerable expense of creating a toolbar with ONE BUTTON: 'Submit' - the author wants you to send your completed report to him.

How is that going to work? Who knows who this Xstench character is? As it's so easy to get a MAC number off a Mac, how does anyone know personal, sensitive information will not be sent along?

Evidently enough people doubted the entire affair that the touted database, with 'thousands' of contributions from Xstench users, would never come into being.

There are other UI bloopers of course - this is far from a polished program from a professional engineer - but they're too numerous to mention here, and the point is obvious already: this is a slipshod program by an author who thinks the program is complete - which is exactly why you can never trust him.

And while a lopsided UI might only be a misdemeanor, touting a program as freeware when in fact it's nagware is certainly a felony.

And not only that, but the poor bastard can't count! For no matter how many times you go into Preferences and delete the poor program's plist, and carefully start it so it only runs once, you always get the following nagalog on the second start.

What the author was probably thinking of is 'you're about to use it a second time', but that's not what he says. He says 'You've used Xbench 2 times.'

If he can't count to 1, how are people supposed to trust the statistics he culls from your machine?

Which is exactly the worry of the following poster. Read it and weep.

Hello, everyone.

Before I go on, I would like everyone to know that I am posting this article without any negative intention against the development team of Xbench.

However, so-called the benchmark results that I received from Xbench 1.1 (and even the past versions of Xbench) left me confused.

Two users of 17" PowerBook had Xbench run on their PowerBooks. The results not only contained very different numbers between the two PowerBooks, but also contained 'wrong' information on the hardware setup. For example, according to the Xbench 1.1, one person had a PowerBook with dual G4 at 800MHz each. He also had 1,152MB RAM while 1,024MB is the maximum amount of RAM the PowerBook can have. Information on L2 cache, L3 cache, graphic card and the bus speed were also incorrect.

The performance results were not consistent between the two PowerBooks, either.

If the inconsistent results were limited to hard disk, it would have been easier to understand (although the margins were too far to be credible). Yet even the CPU, FPU, AltiVec, and memory performance had differences of over 20%. The Quartz graphics test had a difference close to 20%, and the OpenGL test had a difference of over 20%.

The thread test and computation test had differences of over 150%.

The disk test may vary depending on the model and manufacturer; yet a difference of 194% is a little hard to believe.