Rixstep
	`About \| ACP \| Buy \| Industry Watch \| Learning Curve \| News \| Products \| Search \| Substack`

Home » Learning Curve » Developers Workshop » JA/WL

The ABCs of XYZ

Who's the idiot? Who's the creep?

Get It

Try It

By now most people have found out how the secret location of the Cablegate tranche was discovered online. There are myriad mirrors of those files but one in particular made its way onto Twitter.

For a more extensive coverage of how this transpired, see the link below.

http://nigelparry.com/news/guardian-david-leigh-cablegate.shtml

For a (yes) complete explanation of the exigencies involved (and why Leigh and the Grauniad have again stuck their noses in it) see the link below.

http://unspecified.wordpress.com/2011/09/03/wikileaks-password-leak-faq

But when all is said and done, when you realise the extent of the perfidy Leigh and the Grauniad have been involved in:

Being the editor's brother-in-law means David Leigh never has to say sorry for publishing the secret password to the largest intelligence leak in human history as a chapter heading in a book with the newspaper's logo on it.

Then you still have to wonder how anyone could connect the one with the other: the absolutely facepalming unforgivable publication not only of the means of distribution of 251,287 embassy cables, not only the methodology used to adequately protect those cables, not only an insight into the infrastructure used to protect those cables, not only the algorithm used to construct the encryption key used to protect those cables, but the sacrosanct 'salt' that must never be written down (but which Guardian Books and Leigh not only wrote down but published in a book and even used as a chapter header for said book) with that final vital piece of information needed to complete the puzzle: the location of the files themselves.

The files in question were found on a server in Croatia. Whether this server is part of an extensive internal WikiLeaks backup network, part of the volunteer mirroring movement that's assembled thousands more servers around the world, or part of the seeder network that cropped up as a result of the torrent being posted at The Pirate Bay is not known. But it's there.

descr:          Multimedijani institut
descr:          Preradoviceva 18
descr:          10000 Zagreb
country:        HR
admin-c:        CIa22-RIPE
tech-c:         CIa22-RIPE
status:         ASSIGNED PA
mnt-by:         AS2108-MNT
source:         RIPE # Filtered

The RRL in question is /wiki/file/xyz. That means that from server root level there are three nested levels of directories needed to access the holy grail. The final level 'xyz' is visible in the directory listing of parent directory 'file'. And 'file' is visible in the directory listing of its parent 'wiki'. But 'wiki' is not visible in the root directory listing at all.

A bit about how Apache works. For Apache is the web server used on that server (and most good servers today). Apache has default behaviour for directories that don't have what's known as a directory index file. The directory index file is a file Apache uses as the default file to read when accessing a directory. A typical array of file names can appear in the file .htaccess in the directory itself.

DirectoryIndex index.shtml index.html index.cgi index.php

The names are tested in order given. So if both 'index.shtml' and 'index.html' are found, Apache will render 'index.shtml' and won't even proceed to look at 'index.html'. And so forth.

But what happens if Apache can't make a match at all? Ah. Apache then reverts to internally programmed behaviour to offer a rudimentary listing of the directory's contents on its own terms. Which is what you seem to see when you access that Croatian server at root.

Don't give any credence to the 'modified' times listed. They don't have to mean anything as you'll see below. But do note that the all-important subdirectory 'wiki' is not listed.

That isn't a Mac over there in Croatia. It's not Apple's file system HFS and it's certainly not Apple's 'FF' spewing out the data. It's Apache. And it's running on Ubuntu. It's not some concoction from Redmond either.

You can't hide anything on a Unix file system. You can make things inaccessible so the user can't see what subdirectories and further files are hidden within but you can't do something like Apple and Microsoft do. Nothing is invisible on Unix.

We can access the magical directory 'xyz' so it's definitely accessible. And yet its parent directory isn't listed at root. What's going on?

There are probably myriad ways to accomplish this but here's one.

Take the output generated by Apache for a directory without an index file and redact it. Take out the reference to the directory you don't want anyone to see ('wiki').
Save this to disk with an unforgivably incomprehensibly long and funky name. Use every possible character and make the name as long as the system permits - perhaps up to 1024 characters long. Don't even bother with a file extension - or for that matter use several nested file extensions. It doesn't matter - do what you want.
Use a random character generator for even better results.
Add this file name to the directory's .htaccess file and upload both.

The results should be exactly what you see above - sans the reference to the now hidden and for all practical purposes nonexistent subdirectory 'wiki'. That subdirectory for all practical purposes now ceases to exist.

Unless you of course already know of its existence.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
<head>
  <title>Index of /</title>
</head>
<body>
<h1>Index of /</h1>
<table><tr><th><img src="/icons/blank.gif" alt="[ICO]"></th><th><a href="?C=N;O=D">Name</a></th><th><a href="?C=M;O=A">Last modified</a></th><th><a href="?C=S;O=A">Size</a></th><th><a href="?C=D;O=A">Description</a></th></tr><tr><th colspan="5"><hr></th></tr>
<tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="collateralmurder/">collateralmurder/</a></td><td align="right">06-Dec-2010 15:17  </td><td align="right">  - </td><td> </td></tr>
<tr><td valign="top"><img src="/icons/image2.gif" alt="[IMG]"></td><td><a href="favicon.ico">favicon.ico</a></td><td align="right">19-May-2010 02:34  </td><td align="right">9.4K</td><td> </td></tr>
<tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="file/">file/</a></td><td align="right">02-Sep-2011 17:34  </td><td align="right">  - </td><td> </td></tr>
<tr><td valign="top"><img src="/icons/text.gif" alt="[TXT]"></td><td><a href="robots-out.txt">robots-out.txt</a></td><td align="right">12-Dec-2010 16:02  </td><td align="right"> 53K</td><td> </td></tr>
<tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="torrent/">torrent/</a></td><td align="right">25-Aug-2010 18:28  </td><td align="right">  - </td><td> </td></tr>
<tr><th colspan="5"><hr></th></tr>
</table>
</body></html>

Of course there's an easier way. As pointed out by Sean Collins of Core IT Pro - at least on Apache servers (and the above server is an Apache server). It's an .htaccess directive called 'IndexIgnore'.

But the fact remains that no one external to the site would know of the use of the directive, and only people with internal access would know anything was being hidden.

Only Three

Only three people purportedly knew of the existence of that directory: Julian Assange, David Leigh, and an unnamed third other. Daniel Domscheit-Berg reputedly blagged that third other to reveal its location. So the story goes.

But David Leigh never revealed the location of that directory. And Julian Assange certainly didn't. Which leaves only one person remaining. The Cablegate tranche might as well have been protected by Rubberhose - it was probably protected even better.

Daniel Domscheit-Berg contacted the people at Der Freitag to tell them the good news.

[According to Andrew Fowler's documentary, it was Herbert Snorrason who knew of the the file's existence. He contacted the one person he felt he could trust immediately he corroborated the encryption key in David Leigh's book: he contacted Daniel Domscheit-Berg.]

Can the existence of the disingenuous index file be detected? Yes if you know the name. Otherwise you're SOL.

HTTP/1.1 404 Not Found
Server: Apache/2.2.14 (Ubuntu)

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /
BGWw7u8mx2oLXkltEXG1uiAgK07N1CYAS44XxCKsD8Dkrxb5Us4OYC2qC7DBJlJjoNHj\
XbbkhmVYj4bCwfYSrbI1gWA0FRj1oyiKZH4F4XllcNxW0Uzq3HpjlyhzPPyCNhWvwaA0\
8tkiGfEF9D5CTsuCqbBG0ASMpyHlYPkeIUMX9bmHmpRFhLQWvbmvjEFYCWKiv4LCYhhf\
GTw3GMGvhgRaIBUzPjYZGqG9ubJQI03XRyafKqbzV0alC4kZnI718LB0wSOCTSjuQHak\
69hb7HMHL6p7Nw8THJSBkqLDI57gMfOSm61uNNBhSyonswG9EgIyXe9nhGUSgwTYI.html was not found on this server.</p>
<hr>
</body></html>

And no point trying to detect (or read) .htaccess.

HTTP/1.1 403 Forbidden
Server: Apache/2.2.14 (Ubuntu)
Vary: Accept-Encoding
Content-Length: 293
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /.htaccess
on this server.</p>
<hr>
</body></html>

Note that Apache is smart enough to not even reveal if .htaccess is there.

HTTP/1.1 403 Forbidden
Date: Tue, 06 Sep 2011 01:19:29 GMT
Server: Apache/2.2.14 (Ubuntu)
Vary: Accept-Encoding
Content-Length: 326
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /.htaccess-oh-yeah-come-on-this-is-bullshit
on this server.</p>
<hr>
</body></html>

There's no way Julian Assange explained any of this to David Leigh. Assange might have thought Leigh capable of managing a 7-zip but there's no way he didn't see him as a mental midget. Leigh never knew the directory could be hidden from view. And the former comment editor at the Grauniad is too ignorant to be curious about something like that.

But the directory was hidden. Leigh had the URL. Julian put the directory there. One other person knew of its existence. And that person was blagged by Daniel Domscheit-Berg.

This isn't security through obscurity. It's security through 'it doesn't exist!'

DDB was out to do damage at any price. Publishing the key and the salt was colossally stupid. Already there the organisation's security is compromised as people now know how they build passwords and encryption keys. Stupid stupid stupid.

But Daniel Domscheit-Berg's act was one of pure evil. But does that make David Leigh the idiot?

Backtrack

Backtrack to that fateful 2010 summer night. Take the appropriate passages right out of David Leigh's book.

The cables

Near Lochnagar, Scotland
August 2010

ACollectionOfDiplomaticHistorySince_1966_ToThe_PresentDay#
ASSANGE'S 58-CHARACTER PASSWORD

It had been a struggle to prise these documents out of Assange back in London. There were repeated pilgrimages to the mews house belonging to Vaughan Smith's Frontline Club near Paddington station before Assange reluctantly turned them over. Leigh used his ever-shifting demands as a negotiating lever. Assange capitulated. Late at night he started the process on one of his little netbooks that would enable Leigh to download the entire tranche of cables. The Guardian journalist had to set up the PGP encryption system on his laptop at home across the other side of London. Then he could feed in a password Assange wrote down on a scrap of paper.

Hold it right there. Leigh knew he had to set up a 'PGP encryption system' on his own computer when he got home. The book deliberately uses the term 'password' even though Leigh at the time of download at the latest knew it was nothing such. Leigh didn't use the 'password' to download - all he needed was the URL.

Leigh is using the term 'password' all over the place. He knows the additional software isn't called a 'PGP password system': he calls it a 'PGP encryption system' right there in the same paragraph. Nobody's that dumb. Leigh knows it's not a password. And that's why he keeps using the term. Nor could he miss the fact he didn't need the 'key' to access the download URL.

David Leigh deliberately plays dumb - that's his escape strategy if he's threatened with exposure.

The book doesn't mention that Assange wrote down the URL. He might not have: he could have communicated the URL over the Internet (although given who Assange is, this would be unlikely). Assange writes down most of the encryption key as he did years earlier for Nick Davies. Then he tells Leigh there's one more word and asks Leigh if he can remember it.

'I can remember that', Leigh claims he replied.

Leigh set off home and successfully installed the PGP software. He typed in the lengthy password and was gratified to be able to download a huge file from Assange's temporary website.

Uh NO. That's a deliberate lie. Leigh is intentionally prevaricating here. The 'password' isn't a password at all. The 'password' is an encryption key used to decrypt the download. The download is freely accessible without authentication. Leigh knows this.

Leigh downloads the file first - without password, without encryption key - and then and only then does he use his brand new PGP encryption software - he's deliberately shuffling the chronology for the readership.

There's no mention of a second bit of bits, a real password in addition to the encryption key. And normally there won't be as the purpose of encryption is to let files pass in the clear. Read that section of Leigh's book again.

He typed in the lengthy password and was gratified to be able to download a huge file from Assange's temporary website.

Uh - 'temporary'? Who said anything about 'temporary'? What would 'temporary' matter anyway? The file's encrypted with high grade technology. Why would 'temporary' be even said in their meeting? Assange is famously tight-lipped - and he's not about to say something like that to Leigh anyway. If the file were truly at a 'temporary' location, Assange would have demanded Leigh ring back when he had it.

'OK David, here's your super-secret important encryption key. You add the word you promised you would't forget and then use that to decrypt the file once you've downloaded it. And oh by the way? That URL? Yeah that's a temporary one. And the file's on a server from HP that we painted blue last month.'

There's no way that plays.

Moreover: Leigh is deliberately confusing the readership here. And he has a long history of doing that. He deliberately confuses 'key' with 'password' and then hints he needed the key to do the actual download. Which is nonsense.

Leigh got the download a full six months before his book was published. Countless Guardian journalists and technical gurus gained access to the file during that period, often in strict violation of the memorandum of understanding. Leigh had technical staff such as Harold Frayman helping him with the file. As promiscuous as the Guardian turned out to be, it's impossible they didn't download the file several more times - perhaps creating the 'bogus' Heather Brooke copy.

There is no way Leigh's book went to print without everyone at the Guardian understanding the score.

So is DDB a rat bastard because he told someone where the file was located? No. Assange told Leigh and Assange is definitely not a rat bastard. Files protected with high grade encryption are expected to be out in the open. A great many people knew of the file's existence and some even knew of its significance.

But nobody could do a thing because they didn't have the encryption key. David Leigh published the key - he even published the part that was never supposed to be written down. And everything else he wrote is a deliberate lie all designed to confuse the readers and to ultimately get him off the hook if he were ever found out.

Daniel Domscheit-Berg's actions were pure evil. But they can't compare to those of the master. David Leigh almost hoodwinked an entire world into believing he was just congenitally stupid. A few seconds reflection on what he actually wrote shows him to be the consummate liar.

Twas bryllyg and ye slythy toves did gyre and gymble in ye wabe.
- The Jabberwock
We have technical software controllants reviewing the code right now.
- Daniel Domscheit-Berg