About | Buy Stuff | Industry Watch | Learning Curve | Products | Search | Twitter
Home » Learning Curve

Every Bundle Counts!

Deploy your software.


Recently there's been more discussion about cleaning universal builds to reclaim disk space. As has already been pointed out this can often be but the tip of the iceberg. Here follow some general tips on how you can successfully deploy your own software and save (sometimes) copious amounts of disk space as well as a real life example of the dramatic results that can be expected.

Why Don't Apple Do It?

The quick answer to the above is 'but Apple do do it'. Stripping binaries is not what they want or can do: their OS must run on all platforms and in addition support software written for all platforms - sometimes for the wrong platforms.

But that's only about the 'universal' aspect of Apple builds anyway - in all other respects Apple engineering represents a yardstick third party vendors should hold themselves to.

Why Don't Third Party Do It?

Good question, next question. It's perhaps understandable Apple do the best job as they're most intimately acquainted with how the system works, but it's not beyond a third party vendor understanding the methods and the exigencies involved.

And it's what they don't do you have to do - if you want to reclaim the disk space they're so graciously wasting on your behalf.

Drilling Down!

To get at all of the flotsam and jetsam in your applications you have to be able to 'drill down' inside the application bundles - and recursively into further embedded bundles if need be. You can do this in one of several ways.

  1. Get a decent file manager such as this or this. [Although if you choose the latter you'll waste the first few hours cleaning that one instead of the bundle you originally planned on cleaning. Ed.]

  2. Get to a command line and stay there. Terminal (or the equivalent) will show you what you really have on disk and you need to see that. You might use a command line for other purposes anyway.

  3. Tell yourself 'Finder's spatial - and the best thing since sliced bread' and get used to ctrl-clicking for 'Show Package Contents' all the time. The overview won't be as good and you'll have difficulties integrating with the command line but with a bit of perseverance (and perhaps several weeks arduous labour) you'll make it.

What You Don't Need

There are any number of typical elements of an OS X application package that represent 'flotsam and jetsam' - that have nothing to do with the running of the application you just downloaded.

PkgInfo
This is an eight byte file that consumes four kilobytes on disk because of the clustering of your file system. It contains but a creator code and a file type, each four bytes in length.

This information is also found in the essential file 'Info.plist' and therefore is basically redundant - but for authentically native Cocoa applications only. If you are not sure if you're looking at a real Cocoa application then don't touch it. If you still do about the worst that can happen is you lose your dock icon but don't touch it anyway.

If however you know you're looking at a really native OS X application then by all means get rid of it. You just reclaimed 4 KB disk space.

pbdevelopment.plist
Why this file is ever created is a mystery but it's not created on real 'install' builds. Unfortunately too few third party vendors seem to know how to create these install builds. If you find this file inside your application package, just remove it.

You might be curious as to what it contains, and if you feel the urge then by all means take a peek. It's a straightforward property list file with one key value pair only that archives the disk location of the development project that was used to create your application. As this pertains to the developer computers and not yours you see how irrelevant it is for you. Remove it and you reclaim another 4 KB.

NIBs
NIBs - NeXTSTEP Interface Builder files - are a chapter unto themselves. For starters they're not files at all (but may appear to be - especially if the file manager you're stuck with is Finder). They're directories and in turn contain files that are used by your application at runtime.

Unfortunately there's only one file in a NIB that's ever used by the running application: objects.nib (or keyedobjects.nib). All the other files can and should be removed - in fact Apple themselves make quite a point of doing this for all their products. It's just that third party vendors either #1) don't know; or #2) don't bother.

See for yourself: search /System/Library for any of the other files such as classes.nib, info.nib, or data.dependency. The first two are plain text files used by Interface Builder and the third is a clue to Interface Builder what other so called 'palettes' it needs to load to manage your interface (at edit time only).

And yes, if you have the ADC developer tools installed, you will find quite a few of the above - and you should not remove them. [You should not ever muck about in /System/Library at all.] But these files are used for development: they interface developer tools where they're needed - they're not used in third party apps such as the ones you're looking at right now.

The classes.nib, info.nib, and data.dependency files each take at least one cluster each on disk. That means every time you remove one of them you reclaim at least another 4 KB of your disk space.

As they can be found in every NIB your application has and as the first two are almost always found there, you reclaim at least 8 KB for every NIB in your application. And remember: you need objects.nib (keyedobjects.nib) as that's the code that's run by your application.

[Note: NIBs can also contain image files. These files are essential and should not be removed.]

lproj
Localisation files are found in directories with the lproj extension. One of these directories will be the one used by your system when you run your application. The method the system uses to find the right files is rather sophisticated: you have a preferred order of languages already specified in your system preferences; the app at runtime will access its resources first and foremost through its Resources directory; if the files needed are not found there it will then look through Resources in search of an lproj subdirectory corresponding to your list of preferred languages in your system preferences.

If English is your preferred language and the NIBs your application needs are not found directly in Resources, the system will look first for English.lproj or en.lproj. If these are not found the system will then check for your second preference. If it's French the app will now search for French.lproj or fr.lproj. And so forth.

The point being that unless you plan to flip your language preferences all the time you're probably going to run your application in a single language. And if you know where those files are located in your application package you can safely move them to Resources and thereafter remove all extraneous lproj files.

Check Safari as an example: Apple's web browser has numerous NIBs directly in Resources. It can have others in language specific directories but the point is the system itself will never go down below Resources if it doesn't have to. If you have all your resources directly in Resources you not only save disk space but you help your application load and run marginally faster.

Most applications will have at least two NIBs and one localisation directory. Removing classes.nib and info.nib from both and moving the NIBs up to Resources (and removing your now empty localisation directory) saves you 20 KB just like that.

[That's because no matter what your trusty Finder tells you those directories do consume disk space like any other file in your system - and again the minimum they can take is 4 KB. Ed.]

Image Files
TIFFs (tagged image format files) are unusual in that they permit non-lossy compression. But using this compression is not always the default. If you have a tool that can compress your application's TIFFs you can save quite a lot of disk space. OS X's ICNS (NeXTSTEP icon) files are OK - when TIFFs and other images are imported they're automatically compressed. PNG files too are automatically compressed. It's just TIFFs you need care about.

Stripping Binaries I

You should always clean your app as well as you can according to the above guidelines before attempting any work on the binaries themselves. When you've got the application bundles whittled down you should archive (zip) the application up and set it away for safekeeping. What you've done up to this point works for any hardware platform - what you do now is hardware specific.

[And you should always test your application before proceeding: make sure it still performs according to spec - you might have made a blooper and you want to catch it now instead of later. If the app is functioning correctly, then yes - save your new archive and dispense of the old one.]

There are two steps in stripping binaries.

  1. Get rid of the binary image you never use.
  2. Get rid of the debug info you won't want either.

They must be performed in the above order.

There are several tools available for step 1, perhaps most notably Monolingual. Be careful when you use the program as you don't want to remove the wrong images.

You can also do this manually but with a great number of targets all at once it becomes tedious.

You might want to first check and see if your target is in fact a universal ('fat') binary. Drill down to its directory and use the following.

/usr/bin/file <target>

If you're looking at a fat binary, file will tell you so.

The next step (no pun intended) is to strip out the unused binary. There are several methods to do this but one does it the way you really want and the others don't. In all cases you use the program /usr/bin/lipo.

lipo <target> -thin <arch> -output <target>.lipo; mv <target>.lipo <target>

The 'lipo' extension is arbitrary; you can use any extension you want. The idea is to avoid possible renaming conflicts. You can of course reduce all this to a script as well.

lipo $2 -thin $1 -output $2.lipo; mv $2.lipo $2

Where $1 is the architecture you want to use and $2 is your target. [The architecture you want to use will mostly be either 'ppc' or 'i386'.]

So if your script were called 'unilipo' a typical invocation would look like one of the following.

unilipo ppc <target>
unilipo i386 <target>

Where '<target>' again is the binary you want to perform the lipo on.

Where is this binary located? Your application bundle may have several - you need to keep 'heads up' and 'eyes open' for this phase. The primary binary will be in a 'MacOS' directory under 'Contents' but there can be other binaries as well embedded in frameworks and plugins and whatnot inside your application bundle in the Resources directory.

And any file marked with the eXecutable bits (aside from legacy which often have all bits set as they didn't use the present POSIX paradigm) may be a binary you'd want to strip. Run file on each of the suspects and proceed accordingly.

A universal binary will normally be about twice the size of a single architecture binary. So if your universal binary is 500 KB you'll reclaim about 250 KB. And so forth.

Stripping Binaries II

But that's only phase one; phase two can often have more dramatic results. For a reason unknown to most platform observers an alarming number of third party vendors ship so called 'debug binaries' - these binaries, several times the size they need to be, are chock full of - for you - totally useless information.

This information is to be used (sparingly) during software development to track down bugs and other unforeseen occurrences prior to declaring a program fit to ship and creating a good 'install' build. Apple's development environments of course see the developer can set things as needed both for the development stage and finally the deployment stage.

If your third party vendor ships you a debug image there will be vestiges of junk left inside the binary no matter what you do but stripping away the most of it will still save you incredible amounts of disk space.

To do this you need the ADC developer tools - in particular pbxcp found at /Developer/Tools. This program works a funky way - and here's a script that will make it work.

/Developer/Tools/pbxcp -resolve-src-symlinks -strip-debug-symbols <target> ..; mv ../<target> .

Where again '<target>' is the binary you want to strip.

pbxcp assumes you're moving a non-stripped binary to another directory; as it copies the binary it strips it. To get your binary back where it belongs you simply move it back.

You can synthesize all the above into a script you can invoke from your command line.

/Developer/Tools/pbxcp -resolve-src-symlinks -strip-debug-symbols $1 ..; mv ../$1 .

Calling this script 'adcstrip' your command line might look like the following.

adcstrip <target>

That's all you'd need. The results from stripping debug info are often far greater than from merely removing an unused binary image. To see how much - read on.

A Real Life Example

Taking a generic Cocoa document based application and seeing what's there to be weeded out before the 'universal strip' can be instructive.

  1. Create a new Cocoa document based application. Call it 'Fatso'.

  2. Build the application 'as is'. Do not change any build settings. This will result in a Cocoa document based application and a binary with a 'debug image'.

  3. This results in the following.
    17 items, 136321 bytes, 336 blocks, 0 bytes in resource forks.
    
    Fatso.app/Contents
    Fatso.app/Contents/Info.plist
    Fatso.app/Contents/MacOS
    Fatso.app/Contents/MacOS/Fatso
    Fatso.app/Contents/pbdevelopment.plist
    Fatso.app/Contents/PkgInfo
    Fatso.app/Contents/Resources
    Fatso.app/Contents/Resources/English.lproj
    Fatso.app/Contents/Resources/English.lproj/InfoPlist.strings
    Fatso.app/Contents/Resources/English.lproj/MainMenu.nib
    Fatso.app/Contents/Resources/English.lproj/MainMenu.nib/classes.nib
    Fatso.app/Contents/Resources/English.lproj/MainMenu.nib/info.nib
    Fatso.app/Contents/Resources/English.lproj/MainMenu.nib/objects.nib
    Fatso.app/Contents/Resources/English.lproj/MyDocument.nib
    Fatso.app/Contents/Resources/English.lproj/MyDocument.nib/classes.nib
    Fatso.app/Contents/Resources/English.lproj/MyDocument.nib/info.nib
    Fatso.app/Contents/Resources/English.lproj/MyDocument.nib/objects.nib
  4. Set to work: clean out classes.nib, info.nib, pbdevelopment.plist, PkgInfo, and bring the NIBs up to Resources.

  5. This results in the following.
    10 items, 134581 bytes, 288 blocks, 0 bytes in resource forks.
    
    Fatso.app/Contents
    Fatso.app/Contents/Info.plist
    Fatso.app/Contents/MacOS
    Fatso.app/Contents/MacOS/Fatso
    Fatso.app/Contents/Resources
    Fatso.app/Contents/Resources/InfoPlist.strings
    Fatso.app/Contents/Resources/MainMenu.nib
    Fatso.app/Contents/Resources/MainMenu.nib/objects.nib
    Fatso.app/Contents/Resources/MyDocument.nib
    Fatso.app/Contents/Resources/MyDocument.nib/objects.nib
    The bytes saved don't tell the story - the blocks saved do. From an original 336 blocks things are down to 288 blocks. In other words the bundle originally took 172032 bytes but now takes only 147456 bytes - the savings at this point are 24576 bytes (24 KB). Considering this particular (minimal) bundle was only 168 KB to start with the savings are already 15%.

    But that's only the start - it's the debug image which really spooks. It's 126376 bytes and has nary a line of code in it - it's purely a skeletal app generated by Apple's ADC tools. But stripping it saves almost three quarters of its on disk size.

  6. adcstrip Fatso

    The resulting stripped binary is only 18788 bytes. 107588 bytes (86%) have been stripped away.

  7. What you're now left with is the following.
    10 items, 26993 bytes, 80 blocks, 0 bytes in resource forks.
    
    Fatso.app/Contents
    Fatso.app/Contents/Info.plist
    Fatso.app/Contents/MacOS
    Fatso.app/Contents/MacOS/Fatso
    Fatso.app/Contents/Resources
    Fatso.app/Contents/Resources/InfoPlist.strings
    Fatso.app/Contents/Resources/MainMenu.nib
    Fatso.app/Contents/Resources/MainMenu.nib/objects.nib
    Fatso.app/Contents/Resources/MyDocument.nib
    Fatso.app/Contents/Resources/MyDocument.nib/objects.nib

Which is pretty incriminating. The original bundle took 136321 bytes/336 blocks; after your work on it all you have left is 26993 bytes/80 blocks. The results are nothing less than dramatic - from an original 168 KB you're down to 40 KB. You just reclaimed 128 KB - which is over three times what remains. Nearly 80% of the bundle was pure junk.

And were this only a question of stripping a universal binary (with all else already taken care of) the difference would have been in the 20 KB range rather than the above figure of 128 KB, over six times that.

And of course the issues compound when application bundles have further bundles within - frameworks and plugins each with their own binaries and resources that in turn might contain further bundles in need of similar cleaning and each with their own NIBs in need of trimming. The above example is only the merest of examples of how dramatic the differences can be - between what you need and what you get.

A relevant question is: are there any tools that automate this process? And the answer has to be 'not up to now there aren't'. Until something comes along to save users from these issues it's still all up to that developer you trust.

See Also
Rixstep Reviews: The Very Ugly
Learning Curve: Intel, Not Universal!
Mac OS Hints: Strip x86 code from fat binaries

About | ACP | Buy | Forum | Industry Watch | Learning Curve | Search | Twitter | Xnews
Copyright © Rixstep. All rights reserved.