I write about NSA's surveillance program. I do not write about its blatant unconstitutionality, nor about its obvious potential for abuse, nor its uselessness in preventing the Marathon bombing, nor even about the institutional incompetence of outsourcing data that hot; I write to point in amazement at a yottabyte.
A yottabyte of storage! That's what the NSA claims to have in that facility in Utah:
Now maybe they're lying or mistaken; but for now let us charitably assume that an information agency is giving out valid information. Yotta is a septillion, or ten to the 24th power: 1,000,000,000,000,000,000,000,000. I've only just gotten used to tera, and was preparing for peta, but now they leap past exa and zetta and land smack on yotta! A yottabyte is a trillion trillion bytes; that is, a thousand million billion megabytes. Even reciting that number makes me sound like an excited child. Or put it this way; 10^24 is 1.6 times Avogadro's number. One point six moles of bytes! That's not an astronomical number; it's a chemical number! Did the NSA plan to track individual molecules?
Now I ask you; how come we paid for 1.6 moles of storage? Divided amongst 6 billion people that's about 166 trillion bytes per person, or 166,666 gigabytes each. I've just put a Dr. Who CD in this computer's disk drive; it informs me that the Doctor takes up 7 gigabytes; so the NSA is taking the equivalent of 23,809 Dr. Who CDs, for each and every person now alive. Are we all really that interesting? I personally have been alive for less than 21,000 days, and the sum total of all my best writings fits onto a single thumb drive!
How do you process a yottabyte? A petaflop machine will take a gigasecond just to scan it; that's 31 years! And as for analyzing it... think of all the waste heat. Think of all the spurious correlations! Suppose they outdo the particle physicists and analyze data to 7 sigmas; that's still a false-positive rate of about one in a trillion; times a septillion is a trillion false positives!
How do you secure a yottabyte? I suppose sheer inertia will help; but to make any use of that inert mass, they'd need an army of smart hackers, and some of them will be Snowden. It's as Assange predicted; the security state must either tighten up internal communications, and become stupid, or keep internal communication open, and leak. Snowden proves that they've achieved the middle ground; stupid enough to outsource data handling to Snowden.
So what's going on here? Is the NSA super-stalking everyone in the world? Or do they need a molar buffer for Captain Kirk's transporter? Or is somebody high up a hoarder?
Or maybe, just maybe, some well-placed high-tech company got a really sweet no-bid deal on constructing a massive boondoggle?
Or maybe even: spies tell lies, and it's not a yottabyte, but it is 2 billion dollars?
Since it's all secret, how do I know that it's not corrupt?