The waves of controversy and outrage following former CIA and NSA technical officer Edward Snowden’s leaks of details that reveal the US and British governments had been secretly deploying mass surveillance programs on their citizens are far from over. Recently further details as to the extent and reach these programs possess have been uncovered, after a top secret National Security Agency program called XKeyScore was recently presented to the general public.

“I, sitting at my desk,” said Snowden, could “wiretap anyone, from you or your accountant, to a federal judge or even the president, if I had a personal email”.

This statement made a lot of people at Washington unhappy, and unsurprising Snowden’s claim was quickly refuted by officials. On another note, Snowden – a whistle-blower and hero by one side, and a dangerous traitor from another – is currently residing in Moscow, Russia, after fleeing the US via Hong Kong.

Wouldn’t you know it, however, most recently training files for a secret software called XKeyScore have been leaked which show and demonstrate that any analyst has access to a user’s internet history (e-mails, phone number, browsing, chats, just about anything) WITHOUT the need for a warrant. US laws dictates that the NSA is required to obtain an individualized Fisa warrant only if the target of their surveillance is a ‘US person’, however XKeyScore offers the technological capabilities (legal authority is blurry) for an analyst to perform any kind of digital surveillance on any user, US citizen or otherwise.

These training files show how an analyst can easily access just about any digital history for a particular user through a simple point and click interface that can return valuable and relevant information to their queries. The request is not reviewed by a court or any NSA personnel before it is processed.

XKeyScore has access to such a wealth of information that the analyst can retrieve relevant information simply by searching using one or a combination of the following: name, telephone number, IP address, keywords, the language in which the internet activity was conducted or the type of browser used. E-mail is of little interest, since much of the activity on the web is anonymous.

The NSA documents assert that by 2008, 300 terrorists had been captured using intelligence from XKeyscore.

The XKeyscore program also allows an analyst to learn the IP addresses of every person who visits any website the analyst specifies. As one slide indicates, the ability to search HTTP activity by keyword permits the analyst access to what the NSA calls “nearly everything a typical user does on the internet”.

It’s all very easy. For instance, Snowden, who was authorized to use XKeyScore, recalls that to search for emails, an analyst using XKS enters the individual’s email address into a simple online search form, along with the “justification” for the search and the time period for which the emails are sought.

What this means – and these latest insights come as a confirmation – is that the NSA and surrounding agencies are storing immense amounts of data. One NSA report from 2007 estimated that there were 850bn “call events” collected and stored in the NSA databases, and close to 150bn internet records. Each day, the document says, 1-2bn records were added ( emails, phone calls and other type of communications). William Binney, a former NSA mathematician, said last year that the agency had “assembled on the order of 20 trillion transactions about U.S. citizens with other U.S. citizens,” an estimate, he said, that “only was involving phone calls and emails.”

A few years ago, I reported on ZME Science on the matter that IBM was planning to build the largest data array in the world capable of storing 120 petabytes – 60 downloads of the entire internet or 10 times bigger than any other data center in the world at present date. The contractor is unknown, but it’s easy to guess who. ZME Science also reported how every six hours the NSA collects data the size of the Library of Congress.

Still, even with these resources the NSA can’t store all your data for too long, considering the humongous amount of internet connections. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: “At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours.” In 2012, there were at least 41 billion total records collected and stored in XKeyscore for a single 30-day period.

In a statement to the Guardian, the NSA said:

“NSA’s activities are focused and specifically deployed against – and only against – legitimate foreign intelligence targets in response to requirements that our leaders need for information necessary to protect our nation and its interests.”

“XKeyscore is used as a part of NSA’s lawful foreign signals intelligence collection system.

“Allegations of widespread, unchecked analyst access to NSA collection data are simply not true. Access to XKeyscore, as well as all of NSA’s analytic tools, is limited to only those personnel who require access for their assigned tasks … In addition, there are multiple technical, manual and supervisory checks and balances within the system to prevent deliberate misuse from occurring.”

“Every search by an NSA analyst is fully auditable, to ensure that they are proper and within the law.

“These types of programs allow us to collect the information that enables us to perform our missions successfully – to defend the nation and to protect US and allied troops abroad.”