social rating, which is monitored by the government. A low social rating could prevent her from working or traveling abroad.
China’s social rating system, which was announced by the ruling Communist Party in 2014, will soon be a fact of life for many more Chinese.
By 2020, if the Party’s plan holds, every footstep, keystroke, like, dislike, social media contact, and posting tracked by the state will affect one’s social rating.
Personal “creditworthiness” or “trustworthiness” points will be used to reward and punish individuals and companies by granting or denying them access to public services like health care, travel, and employment, according to a plan released last year by the municipal government of Beijing. High-scoring individuals will find themselves in a “green channel,” where they can more easily access social opportunities, while those who take actions that are disapproved of by the state will be “unable to move a step.”
Big Brother is an emerging reality in China. Yet in the West, at least, the threat of government surveillance systems being integrated with the existing corporate surveillance capacities of big-data companies like Facebook, Google, Microsoft, and Amazon into one gigantic all-seeing eye appears to trouble very few people—even as countries like Venezuela have been quick to copy the Chinese model.
Still, it can’t happen here, right? We are iPhone owners and Amazon Prime members, not vassals of a one-party state. We are canny consumers who know that Facebook is tracking our interactions and Google is selling us stuff.
Yet it seems to me there is little reason to imagine that the people who run large technology companies have any vested interest in allowing pre-digital folkways to interfere with their 21st-century engineering and business models, any more than 19th-century robber barons showed any particular regard for laws or people that got in the way of their railroads and steel trusts.
Nor is there much reason to imagine that the technologists who run our giant consumer-data monopolies have any better idea of the future they’re building than the rest of us do.
Facebook, Google, and other big-data monopolists already hoover up behavioral markers and cues on a scale and with a frequency that few of us understand. They then analyze, package, and sell that data to their partners.
A glimpse into the inner workings of the global trade in personal data was provided in early December in a 250-page report released by a British parliamentary committee that included hundreds of emails between high-level Facebook executives. Among other things, it showed how the company engineered sneaky ways to obtain continually updated SMS and call data from Android phones. In response, Facebook claimed that users must “opt-in” for the company to gain access to their texts and calls.
The machines and systems that the techno-monopolists have built are changing us faster than they or we understand. The scale of this change is so vast and systemic that we simple humans can’t do the math—perhaps in part because of the way that incessant smartphone use has affected our ability to pay attention to anything longer than 140 or 280 characters.
As the idea of a “right to privacy,” for example, starts to seem hopelessly old-fashioned and impractical in the face of ever-more-invasive data systems—whose eyes and ears, i.e., our smartphones, follow us everywhere—so has our belief that other individual rights, like freedom of speech, are somehow sacred.
Being wired together with billions of other humans in vast networks mediated by thinking machines is not an experience that humans have enjoyed before. The best guides we have to this emerging reality may be failed 20th-century totalitarian experiments and science fiction. More on that a little later.
The speed at which individual-rights-and-privacy-based social arrangements collapse is likely to depend on how fast Big Tech and the American national security apparatus consummate a relationship that has been growing ever closer for the past decade. While US surveillance agencies do not have regular real-time access to the gigantic amounts of data collected by the likes of Google, Facebook, and Amazon—as far as we know, anyway—there is both anecdotal and hard evidence to suggest that the once-distant planets of consumer Big Tech and American surveillance agencies are fast merging into a single corporate-bureaucratic life-world, whose potential for tracking, sorting, gas-lighting, manipulating, and censoring citizens may result in a softer version of China’s Big Brother.
These troubling trends are accelerating in part because Big Tech is increasingly beholden to Washington, which has little incentive to kill the golden goose that is filling its tax and political coffers. One of the leading corporate spenders on lobbying services in Washington, DC, in 2017 was Google’s parent company, Alphabet, which, according to the Center for Responsive Politics, spent more than $18 million. Lobbying Congress and government helps tech companies like Google win large government contracts. Perhaps more importantly, it serves as a shield against attempts to regulate their wildly lucrative businesses.
If anything, measuring the flood of tech dollars pouring into Washington, DC, law firms, lobbying outfits, and think tanks radically understates Big Tech’s influence inside the Beltway. By buyingThe Washington Post, Amazon’s Jeff Bezos took direct control of Washington’s hometown newspaper. In locating one of Amazon’s two new headquarters in nearby Northern Virginia, Bezos made the company a major employer in the area—with 25,000 jobs to offer.
Who will get those jobs? Last year, Amazon Web Services announced the opening of the new AWS Secret Region, the result of a 10-year, $600 million contract the company won from the CIA in 2014. This made Amazon the sole provider of cloud services across “the full range of data classifications, including Unclassified, Sensitive, Secret, and Top Secret,” according to an Amazon corporate press release.
Once the CIA’s Amazon-administered self-contained servers were up and running, the NSA was quick to follow suit, announcing its own integrated big-data project. Last year the agency moved most of its data into a new classified computing environment known as the Intelligence Community GovCloud, an integrated “big data fusion environment,” as the news site NextGov described it, that allows government analysts to “connect the dots” across all available data sources, whether classified or not.
The creation of IC GovCloud should send a chill up the spine of anyone who understands how powerful these systems can be and how inherently resistant they are to traditional forms of oversight, whose own track record can be charitably described as poor.
Amazon’s IC GovCloud was quickly countered by Microsoft’s secure version of its Azure Government cloud service, tailored for the use of 17 US intelligence agencies. Amazon and Microsoft are both expected to be major bidders for the Pentagon’s secure cloud system, the Joint Enterprise Defense Initiative—JEDI—a winner-take-all contract that will likely be worth at least $10 billion.
With so many pots of gold waiting at the end of the Washington, DC, rainbow, it seems like a small matter for tech companies to turn over our personal data—which legally speaking, is actually their data—to the spy agencies that guarantee their profits. This is the threat that is now emerging in plain sight. It is something we should reckon with now, before it’s too late.
In fact, bigtech and the surveillance agencies are already partners. According to a 2016 report by Reuters, Yahoo designed custom software to filter its users’ emails and deliver messages that triggered a set of search terms to the NSA.
The company’s security chief quit in protest when he learned of the program. “Yahoo is a law-abiding company, and complies with the laws of the United States,” the company said in a statement, which notably did not deny the activity, while perhaps implying that turning over user data to government spy agencies is legal.
While Google has stated that it will not provide private data to government agencies, that policy does not extend beyond America’s borders. At the same time as Yahoo was feeding user data to the NSA, Google was developing a search engine called Dragonfly in collaboration with the Communist Party of China. In a letter obtained byThe Intercept, Google CEO Sundar Pichai told a group of six US senators that Dragonfly could have “broad benefits inside and outside of China” but refused to release other details of the program, which the company’s search engine chief, Ben Gomes, informed Google staff would be released in early 2019.
According to the documents obtained byThe Intercept, Dragonfly would restrict access to broad categories of information, banning phrases like “human rights,” “student protest,” and “Nobel Prize” while linking online searches to a user’s phone number and tracking their physical location and movements, all of which will presumably impact social ratings or worse—much worse, if you happen to be a Uighur or a member of another Muslim minority group inside China, more than 1 million of whom are now confined in re-education camps. China’s digital surveillance net is a key tool by which Chinese authorities identify and track Muslims and others in need of re-educatio