The Guardian has released more information about a program that appears to let the NSA access almost any part of a web user's digital life. A set of 2008 and 2010 training documents discuss a program called XKeyscore, which acts as a central interface for email, Facebook chat, web browsing history, and more. According to The Guardian, XKeyscore is available not only to members of the NSA but to outside analysts like Edward Snowden, who worked as a contractor for Booz Allen before his flight to Hong Kong, and its vast database allows users to find people by email address, name, phone number, type of browser, language used, IP address, or specific keywords. Metadata is apparently used to narrow searches, but actual content like email text is also included in the database.
According to the laws that govern NSA surveillance, this data can only be used to target non-Americans without a warrant, but it's not clear whether there are any technical restrictions or guidelines in place, though one slide shows analysts being required to justify their surveillance with a short web form. The NSA, nonetheless, insists that the potential for abuse is low. "XKeyscore is used as a part of NSA's lawful foreign signals intelligence collection system," it told The Guardian in a statement. "Allegations of widespread, unchecked analyst access to NSA collection data are simply not true ... There are multiple technical, manual and supervisory checks and balances within the system to prevent deliberate misuse from occurring." The NSA claimed in documents that by 2008, XKeyscore had been used to help capture 300 terrorists.
The clearest limit on the system is simply how much information it can store. The Guardian reports that in 2012, 41 billion individual records were stored in XKeyscore over one 30-day period. Because of the high volume of incoming records, however, the content of things like emails are stored for only three to five days, with metadata stored for thirty days. If content is deemed "interesting," though, it's moved to other databases, where it can be stored for up to five years. Some of the training documents suggest ways to look for specific users, but others show how to do things like track the IP address of everyone visiting a particular website.
Some of XKeyscore's capabilities were already known. O Globo published slides from an XKeyscore presentation earlier this month, suggesting that the NSA had gotten access to Google Maps records and web searches. More recently, Der Spiegel reported that German intelligence agencies also had access to XKeyscore data. These new documents show further capabilities and hint at how it's been actually used by analysts, showing a system that makes it simple to connect the dots between every part of someone's online activity.
Update: The Guardian has also published a full slideshow from 2008 online. In addition to the slides shown, the report gives multiple examples of how to use the database, including specific people searches based on language and country and document scans that would reveal the origin of terrorist training material or propaganda. Some pages have been redacted, including two slides of "success stories" from the program that presumably would have pointed to specific cases or names.