The xargs Speed Boost

Good stuff for the command line and for CLIX.

Get It Try It

The ACP utility xabatch offers OS X users and administrators unparalleled control of extended attributes.

Yet straightforward use of this program - 'ceteris paribus' so to speak - might yield unexpected results.

xabatch

xabatch works from a command file specified at the first command line argument.

xabatch <command file> <file list>

Every argument after the first - the 'file list' - is a file xabatch is supposed to act on.

An xabatch command file can look like this. The listed attributes are in this case all to be removed from target files.

-:com.apple.diskimages.recentcksum
-:com.apple.FinderInfo
-:com.apple.metadata\:kMDItemAttributeChangeDate
-:com.apple.quarantine
-:com.apple.TextEncoding
-:com.apple.XcodeGenerated

And should one wish to remove them from all files in a user area it seems straightforward to pipe find into xabatch.

find ~ -exec xabatch <command_file> {} \;

But find sends and xabatch inspects all files - running find into xabatch in this way means starting and exiting xabatch a number of times equal to the total number of files in the user home area find finds. xabatch doesn't take any time to run; but the system takes a lot of time starting and stopping the program thousands of times - which is where xargs comes in.

Running find without xargs may take 2-3 minutes to complete; running it with xargs will normally take less than a second.

[Yes it's that dramatic a difference.]

xargs

The manpage for xargs is typically frightening.

XARGS(1)                  BSD General Commands Manual                 XARGS(1)

NAME
     xargs - construct argument list(s) and execute utility

SYNOPSIS
     xargs [-0pt] [-E eofstr] [-I replstr [-R replacements]] [-J replstr]
           [-L number] [-n number [-x]] [-s size] [utility [argument ...]]

DESCRIPTION
     The xargs utility reads space, tab, newline and end-of-file delimited
     arguments from the standard input and executes the specified utility with
     them as arguments.

     The utility and any arguments specified on the command line are given to
     the utility upon each invocation, followed by some number of the argu-
     ments read from standard input. The utility is repeatedly executed until
     standard input is exhausted.

But what xargs is doing is assembling command line arguments and giving a lot of them to the next application all at once - as many as possible without the argument list growing too long. Naturally newlines are parsed out as well.

A snag can be separating arguments: using find with the -print0 switch and xargs with the -0 switch takes care of that.

     -print0
              This primary always evaluates to true. It prints the pathname of
              the current file to standard output, followed by an ASCII NUL
              character (character code 0).

     -0       Change xargs to expect NUL (``\0'') characters as separators,
              instead of spaces and newlines. This is expected to be used in
              concert with the -print0 function in find(1).

The same operation as above but now with xargs.

find ~ -print0 | xargs -0 xabatch <command_file> {}

Using standard CLIX commands to remove extraneous language files, .DS_Store files, and other assorted cruft - commands combining find with rm for example - will go significantly faster with xargs.

For ordinary everyday maintenance work xargs can be the way to go.

See Also
The ACP: CLIX
The ACP: Tools
The ACP: XaBatch
Industry Watch: xabatch