/ 3 January 2020

The cost of surveillance capitalism

The dominance of digital: People play the Pokémon Go game at a park at Tin Shui Wai on July 26 2016
The dominance of digital: People play the Pokémon Go game at a park at Tin Shui Wai on July 26 2016

 

 

As we enter a new decade, we are also entering a new era of political economy. Over the centuries, capitalism has evolved through a number of stages, from industrial to managerial to financial capitalism. Now we are entering the age of “surveillance capitalism”.

Under surveillance capitalism, people’s lived experiences are unilaterally claimed by private companies and translated into proprietary data flows. Some of this data is used to improve products and services. The rest is considered a “behavioural surplus” and valued for its rich predictive signals.

The predictive data is shipped to new-age factories of machine intelligence, where it is computed into highly profitable prediction products that anticipate your current and future choices. Prediction products are then traded in what I call “behavioural futures markets”, where surveillance capitalists sell certainty to their business customers.

Google’s “clickthrough rate” was the first globally successful prediction product, and its ad markets were the first to trade in human futures. Already, surveillance capitalists have grown wealthy from these trading operations, and ever more companies across nearly every economic sector have shown an eagerness to lay bets on our future behaviour.

The competitive dynamics of these new markets reveal surveillance capitalism’s economic imperatives. First, machine intelligence demands a lot of data: economies of scale. Second, the best predictions also require varieties of data: economies of scope. This drove the extension of surplus capture beyond likes and clicks into the offline world: your jogging gait and pace; your breakfast conversation; your hunt for a parking space; your face, voice, personality and emotions.

In a third phase of competitive intensity, surveillance capitalists discovered that the most predictive data comes from intervening in human action to coax, tune, herd and modify behaviour in the direction of guaranteed outcomes. This shift from knowledge to power transforms technology from a means of production to a global means of behavioural modification in order to achieve “economies of action”.

Data centre manager Joel Kjellgren walks in one of the server rooms at the Facebook data centre, its first outside the US, in Luleå, in Swedish Lapland. (Jonathan Nackstrand/AFP)

I call this power to shape human behaviour remotely and at scale towards others’ ends “instrumentarian power” because it works through the medium of digital instrumentation. Instrumentarian power will not threaten you with terror or murder. No soldiers will appear to drag you to the gulag or the camp. This new species of power works remotely, engineering subliminal cues, social comparison dynamics, rewards and punishments, and varieties of enforcers to shape behaviour that aligns with its commercial interests.

Economies of action have been a focal point of experimentation. Facebook’s “massive-scale contagion experiments” discovered how to engineer subliminal cues and social comparison dynamics on Facebook pages to change users’ real-world behaviour and emotions, while steadfastly bypassing user awareness.

The Google-developed augmented-reality game Pokémon Go drove experimentation to a new level. Those participating in the Pokémon Go craze a few years ago were not just playing a smartphone game. In fact, the rewards and punishments of gamification were used to herd people to restaurants, bars, fast-food outlets and shops that had paid for guaranteed “foot traffic”.

Just as industrial capitalism continuously intensified the means of production, surveillance capitalism is intensifying the means of behavioural modification. Surveillance capitalism dismantles the early digital dream, which envisioned the internet as a liberating, democratising force.

One should no longer harbour any illusions that networks are inherently moral or that “connection” is intrinsically social, inclusive and democratic. On the contrary, digital connection is now merely a means to someone else’s commercial end. Surveillance capitalism is parasitic to the core, reprising Karl Marx’s depiction of capitalism as a vampire that feeds on labour, only now labour has been replaced by private human experience.

Surveillance capitalism has been embraced by Facebook, Microsoft, Amazon and many others, but it was first perfected by Google (now Alphabet), just as managerial capitalism was perfected by General Motors a century ago.

As the pioneer, Google rapidly colonised the unmapped spaces of the unregulated internet, where it thrived like an invasive species in an environment free of natural predators. It developed its business model at breakneck speed, far outpacing public institutions’ or users’ ability to follow it. But it also benefited from historical events. After the attacks of September 11 2001, the US national-security apparatus was inclined to nurture, mimic, shelter and appropriate, rather than regulate, surveillance capitalism’s emergent capabilities.

Surveillance capitalists quickly realised that they could do anything they wanted, so that’s what they did. While paying lip service to the emancipatory power of technology, the real action – the relentless secret extraction of private experience as free raw material for production and sales – was hidden offstage. Emboldened by vast and growing revenue streams and a wild, uncontested theatre of operation, they were also protected by the inherent illegibility of automated processes. People simply didn’t realise what was happening, and how the new economic logic really worked.

Having started with the major internet companies, the mechanisms and economic imperatives of surveillance capitalism have become the default model for internet-based businesses generally. Moreover, today’s prediction products extend far beyond targeted online ads to many other sectors, including insurance, retail, finance, health, education and an ever-widening range of goods and services.

These goods and services are not made for constructive producer-consumer reciprocities. Rather than being the objects of a traditional value exchange, they are the “hooks” that lure users into extractive arrangements, where their personal experiences can be harvested and packaged to serve others’ ends. A digital-age truism holds that, “If it’s free, you are the product.” But that is not right: you are the raw material in a much larger extraction process.

This implies a Faustian compact. We have all accepted that the internet is now essential for social and economic participation. Yet to reap its benefits, we must subject ourselves to surveillance-capitalist exploitation. Because we are so dependent on the digital world, we have inured ourselves to the realities of being tracked, parsed, mined and manipulated. We rationalise (“I have nothing to hide”) or simply submit to the loss of privacy and agency, without realising that a fundamentally illegitimate choice is being imposed on us.

How did it come to this? One of the most important features of surveillance capitalism is that it is unprecedented. By definition, unprecedented developments are initially unrecognisable. When we encounter something new, we try to understand it by placing it in the context of familiar categories. For example, when the first automobiles appeared, many relied on the term “horseless carriage” to make sense of the new technology.

When we interpret an unprecedented development in the present as a mere extension of the past, we risk normalising the abnormal. This can be a dangerous error. When indigenous people on the pre-Columbian Caribbean islands first encountered armoured Spanish soldiers marching onto their shores, they assumed that they were being visited by gods and sowed the seeds of their own destruction by welcoming the newcomers.

Because it is unprecedented, surveillance capitalism exceeds the scope of our existing economic concepts. For example, we reach for words like “monopoly” and “privacy” when objecting to surveillance-capitalist practices. But while these terms certainly apply, they do not capture the essence of the new operations.

Surveillance capitalism is not just about corporate governance or market power; it is about an entirely new logic of accumulation, one with its own original mechanisms, methods, imperatives and markets. The consequences reach beyond the conventional territory of the private firm and undermine democracy from above and below.

From above, surveillance capitalism operates through unprecedented asymmetries of knowledge and power, increasing rather than diminishing social inequality. From below, surveillance capitalism’s imperatives take aim at human autonomy, individual sovereignty, and agency – capabilities without which democracy is unimaginable.

Although surveillance capitalism is an entirely new force in economic history, we have already seen enough to know that it relies on flagrant contempt for the social norms and rights that are essential to a properly functioning democratic society. Just as industrial civilisation flourished at the expense of nature, surveillance capitalism thrives at the expense of human nature.

We are now confronting the legacy of industrial capitalism in the global fight against catastrophic climate change. By giving free rein to surveillance capitalism, what will our own legacy be? — © Project Syndicate

Shoshana Zuboff is professor emerita of business administration at Harvard Business School and the author of The Age of Surveillance Capitalism