China’s social credit policies have populated news headlines recently, and it’s easy to see why. The social credit score (SCS), which assigns users a three-digit score, seems to come directly from an episode of Black Mirror. Pulling together reams of data, the SCS supposedly incentivizes good behavior and punishes bad. Yet, the horror stories of the system are only a Google search away: there is the dissident journalist whose low score prevented him from traveling the country on anything other than the slowest trains; the woman who accidentally swiped her young son’s transit card and saw her score drop; the thousands of Chinese citizens whose names and low scores were placed on billboards to be publicly shamed.
But, while mass-coverage of the Social Credit Score (SCS) is recent, the story of the system stretches back to 2014, when the Chinese Communist Party (CCP) released an official planning document laying out a vision for a national social credit score. In the documents the planning committee names four main goals: honesty in government affairs, commercial integrity, societal integrity, and judicial credibility — suggesting a timeline with trials beginning in 2014 and ensuing nationwide implementation in 2020.
Right on schedule, in 2015 –a year after the document was issued — eight private companies, including some of the largest in China, were authorized to begin developing social credit systems. The largest and most successful of this bunch has been Sesame Credit, driven by Ant Financial, one of the wings of the tech giant Alibaba. Using heaps of data from Alibaba’s other subsidiary, Alipay, Sesame Credit was able to construct a detailed, but opaque, system based on purchasing history. Consumers might be punished for late night web browsing or rewarded for buying diapers –an indication of parental responsibility. A video game might indicate a lazy personality and trigger a lower score. Good scores allow lower interest rates on loans and item discounts.
While pernicious and invasive, Sesame Credit was a private company with limited ability to reward and punish. In the past two years, that has changed as provinces have begun to develop their own applications. Honest Shangai, an app in China’s most populous city which debuted during “honesty week” in November of 2016, has led the charge. While the actual mechanisms of Honest Shanghai’s scoring remain opaque, the basic architecture of the app is simple. Citizens sign up using their national ID. Then, using facial recognition, the app tracks interactions with government entities and taps private companies for their data. At the end of this process, users receive a public honesty score of very good, good, or bad.
It is a program like this that the CCP eventually envisions implementing on a national scale. The current timeline is on track to have a compulsory Social Credit Score by 2020 nationwide that, according to planning documents, “allow[s] the trustworthy to roam everywhere while making it hard for the discredited to take a single step.” The three digit score will take into account everything from social media posts, online friends, and purchases. A good score will allow citizens to skip lines, stay at VIP hotels, and find cheaper loans. At 750 points, score holders will receive a fastrack application for the European Union’s Schengen visa. Those with low scores, on the other hand, will have to travel on the slowest trains, stay in the worst hotels, and potentially have their access to government services significantly limited.
The danger of such a system is clear. If successful, the SCS will further consolidate power in China. Xi Jinping recently abolished term limits, paving the way to rule for life. With the ability to withhold benefits at will, he has the tools to suppress dissent. This technological totalitarianism could tighten significantly the grip that the CCP already has on its citizens. One particularly dangerous portion of a national system is its self-policing network effect. Users can see their scores affected if the scores of their close friends drop. As a result, they have a responsibility to self-police, carefully selecting acquaintances, and nudging those in their circles to state approved behavior.
Yet, while it is hard to overstate how draconian the SCS might be, it is important to note two points of nuance. First, though Chinese citizens have concerns about privacy, they still seem to favor the system. Citizen trust in China is significantly lower than in the U.S. or Europe. In a national poll conducted in 2013, less than half of Chinese citizens said that “most people can be trusted” and only about thirty percent said that they trust strangers. Nonetheless, a 2017 survey indicated that “leaking personal data” was the primary concern of consumers. This is to say that, although a number of Chinese consumers see the system as draconian, they are also uniquely poised to benefit from its implementation. For a country that has thrived on Guanxi — personal relationships — and distrust of strangers, a social credit system, even if its categories are arbitrary and opaque, may democratize the social economy. Even though China’s consumers have fierce privacy concerns, they may see the system as a net positive.
There are also reasons that businesses are excited about the social credit score. China has never before had a national credit score. The Ministry of Commerce has estimated that the average loss due to credit misinformation runs nearly $97 billion annually. For lenders, an efficient social credit system is a boon, no matter how unfair. Moreover, while “social integrity” has received the lionshare of international coverage, a social credit system for businesses could reshape an economy that still relies on kickbacks and corruption. The system could produce a degree of transparency and accountability into China that exists nowhere else in the world. That’s not necessarily a bad a thing.
Some commentators have stressed that the West, too, is closer to a social credit score than we think. However, such conclusions are based on a naïve assessment of global dynamics . Unlike China, the United States has a longstanding, widespread, and generally fair national credit score, FICO, that most people have used since the 1950s. Citizen trust in the U.S. is higher, and the artificial intelligence necessary to deploy such a system isn’t as developed. The mobile payment market in China is huge, clocking in at more than $5.5 trillion annuals, more than 50 times the volume of the U.S.
Moreover, at the same time that China has moved into dystopian data, Europe has implemented a groundbreaking and progressive set of rules to govern data use. The General Data Protection Regulation (GDPR) returns the power to users. Article 12 requires “transparent information, communication and modalities for the exercise of the rights of the data subject,’’ setting up clear lines of communications between those who provide data and those who collect it. Article 5 (1b) requires that companies “collect [data] for specified, explicit and legitimate purposes and not further process [it] in a manner that is incompatible with those purposes.” This concept, known as contextual integrity, returns a legitimate idea of consent in data ethics, because users know the destination of their data.
This is all to say: politics in the 21st century are grappling with how to use data. China has developed a system with totalitarian possibilities. But that shouldn’t suggest that data will inevitably work against democracy.