The Cult of Optimization

You awake to the sun rising, glance at the tracker on your wrist. You smile at your high score: an optimal night’s sleep. You step onto your exercise bike for the ten minutes allotted by the AI-optimized schedule on your phone—just enough time to finish the podcast recommended for you yesterday. Breakfast is filtered spring water, rice cakes, and kimchi—optimized to give you just enough energy for the morning ahead. You brush your teeth, though they’re gleaming white already, and start your 5,476-step walk to work. Your first priority will be to make sure someone tweaks the incentives in the app that coordinates your company’s couriers. They need to speed up.

Accounts of daily routines that sound something like this appear regularly in the media; there is plenty of advice on how to enhance your working life along such lines on YouTube. As ludicrous as much of this may be, it speaks to a wide and growing phenomenon: the belief that wherever possible, we should optimize.

Optimization means achieving a measurable objective—that is, maximizing or minimizing a given number. It requires controlling the inputs and processes that affect the objective, in order to find the most efficient way to achieve it. It draws on centuries of mathematical discovery, but it emerged in its modern form under the pressure of World War II and the need to manage the byzantine complexity of US military logistics.

In the summer of 1947, a Stanford-trained mathematical scientist named George Dantzig sat in his office at the Pentagon, laboring over US Air Force planning issues with a desk calculator. The problems he had dealt with during the war and since involved “an astronomical number of feasible solutions to choose from,” making it impossible to calculate which was the best.¹ As he recalled, “Those in charge often do a hand-wave and say, ‘I’ve considered all the alternatives,’ but this is so much garbage.”² All those leaders had to offer was that their “‘experience’ and ‘mature judgment’ would guide the way” by laying down rules that would limit the options.³

The problem, Dantzig realized, was that “you could never find any direct relationship between the stated goal and the actions to achieve the goal.”⁴ The solution, he believed, was to formulate a complex real-world problem as a mathematical model. This could then be solved by what became Dantzig’s “simplex algorithm” (or “simplex method”), provided a precise goal was set as its “objective function.”

The simplex algorithm radically reduced the number of feasible solutions. It soon became clear that it could be brought to bear far beyond the military: “All one had to do,” Dantzig remembered, “was change the names of the columns and the rows, and it was applicable to an economic planning problem or to an industrial planning problem.”⁵

In engineering, optimization was put to use in designing rockets and aircraft, and the shape of cars, wind turbines, and hydrofoils. It redefined manufacturing, circuit design, and the management of supply chains. Its impact is still visible in measures of the occurrence of the word “optimization” itself. Before 1950, the term was barely in use at all; thereafter, its frequency soared.

Economists embraced optimization too.⁶ An early application was in the development of “portfolio theory,” which, as the aerospace engineer Joaquim R. R. A. Martins and the computer scientist Andrew Ning put it, “formalized the idea of investment diversification, marking the birth of modern financial economics.”⁷ One important element of optimization in economics is the inclusion of constraining factors: Given a set level of income or cost, how can we maximize utility? But economics is not quite as scientifically determinable as engineering—it’s more exposed to messy, contradictory fellow humans. Here, optimization starts to look rather suboptimal.

The earliest known use of the word “optimize” is in an 1817 article by the literary critic Leigh Hunt—and it suggests an awareness right from the start that this idea, if misapplied, might do more harm than good. Attacking the mannered style of established poets, Hunt wrote that poetry should be like a true “cornucopia.” It should not be “insidiously optimized at the top, like Mr. Southey’s stale strawberry baskets.”

As computers have become ubiquitous, optimization has spread ever deeper into human life. In 2021, a trio of Stanford academics published a book titled System Error: Where Big Tech Went Wrong and How We Can Reboot. They observed: “What begins as a professional mind-set for the technologist easily becomes a more general orientation to life. … The paramount goal becomes removing friction from everyday activities, automating repetitive tasks, and finding ways to save time while improving outcomes.”⁸ US tech companies, for instance, are often led by software engineers, who manage their staff accordingly, measuring results against precisely set objectives. An over-dominant engineering mindset, System Error argued, is extending optimization beyond the areas where it can be effective.

This might surprise the tech analyst Dan Wang, whose 2025 bestseller Breakneck: China’s Quest to Engineer the Future argued that “an American elite, made up mostly of lawyers, excelling at obstruction” had much to learn from China’s “technocratic class, made up of mostly engineers, that excels at construction.”⁹ But even Wang admitted that engineering logic can be taken too far. “Sometimes, it feels like China’s leadership is made up entirely of hydraulic engineers,” he wrote, “who view the economy and society as liquid flows, as if all human activity—from mass production to reproduction—can be directed, restricted, increased, or blocked with the same ease as turning a series of valves.”

Given its mathematical foundations, optimization depends on having numerical data that can be adjusted to achieve the numerically expressed objective. As the American historian of science Theodore Porter showed in his 1995 study Trust in Numbers: The Pursuit of Objectivity in Science and Public Life, governments began to gather data at scale and to rely on it for decision-making for reasons similar to those set out by Dantzig—to get away from the subjective judgment of leaders.

However, Porter warned that while using numbers to exercise power objectively might be an attractive idea, it is also impossible to do. Even governments can’t count everything, and choosing what to leave out is an intensely political decision. Worse, Porter wrote, “numbers have often been an agency for acting on people, exercising power over them,” even turning people “into objects to be manipulated.”¹⁰ As Wang noted, in China the drive to meet numerical targets has sometimes taken a crushingly simple form, as with the government’s “one child” or “zero Covid” policies.¹¹

Even when optimizers aren’t sealing sick people in their homes, as the Chinese state did during the pandemic, they are often so focused on their objective that they don’t notice the damage they’re doing. Whatever is not relevant to the objective can be shrugged off as a so-called externality. Witness corporations optimizing their operations to maximize profits or the price of their shares. Some squeeze pay or working conditions; others pollute with abandon, or exploit their dominant market position to force down their suppliers’ prices, regardless of the impact.

And the problem with optimization is not just a matter of unfortunate side-effects. We are seeing the emergence of what we might call “social optimization”—the belief that this idea offers a way to transform society as a whole. But as Porter’s work suggests, this is not a matter of neutrally making things better. Optimization privileges the measurable over the unmeasurable. And it places the onus for improving society on the ever-striving individual rather than asking more fundamental, structural questions about why systems work as they do and whom they empower and disempower.

This is not an explicit ideology. No doubt, businesses and governments often are simply following the logic and opportunities implicit in new digital technology, from smartphones to the cameras and sensors that can now cheaply and efficiently monitor a wide range of activities. Nonetheless, as new technology has made it possible to gather ever more numerical data, optimization has begun to embed its implicit values into our lives.

In the workplace, this can swiftly make people’s lives worse. Particularly in sectors such as logistics, new technology allows employers to optimize more and more rigorously for maximum productivity and minimum cost. It has become commonplace to give employees an ongoing score, with the aim of incentivizing them to compete continually. This goes beyond even the monitoring of worker efficiency that the management consultant Frederick Taylor pioneered in the early twentieth century and the numerical key performance indicators that his successors promoted.¹² The intensive quantification of employees’ performance has come to be known as “digital Taylorism.”

Optimization has refocused the media around the measurable preferences of the individual, as tallied in clicks, page views, unique browses, and similar metrics. This erodes the shared moments that build a culture and the shared truths that underpin democracy. Social media takes this even further: Algorithms are optimized to maximize attention, incentivizing people to respond to political issues not with thought but vivid expressions of feeling, rewarding users numerically in follows, likes, and shares. Meanwhile, tracking apps increasingly normalize the optimization of health metrics.

Yet technologists are keen to go much further. Off the back of their successes producing software, they are raising their sights to the horizon, optimizing for a few grand objectives at all costs, in pursuit of an ever more perfect world. They have formed an alliance with philosophers and philanthropists in the Effective Altruism movement, which aims to purge generosity of the influence of feeling in favor of calculable reason—even as it confidently prophesies the far future. Other tech leaders support the principles of the “network state.” According to the journalist Gil Duran, this concept proposes to create “private, corporate-controlled cities” that will liberate innovators from the constraints of the democratic state and its messy, unmeasurable trade-offs.¹³

And most of all, the dream of social optimization reverberates through promises of an AI-transformed future, in which once unthinkably efficient tech will supposedly liberate individual human potential. In “The Techno-Optimist Manifesto” (2023), the venture capitalist Marc Andreessen proposed using technology and the free market to maximize abundance to the point of infinity. Though Andreessen insists he does not believe in “the Unconstrained Vision of Utopia,” he dismisses the “Precautionary Principle” as an “enemy.”

The problem here is obvious to anyone not immersed in the culture of Silicon Valley. Not every worthwhile objective can be measured. How do we quantify social peace, for instance, or the health of our arts and culture, or the concentration of power? Or the worth of work itself, or a truly enriching education, or kindness? We might hope the realization that not everything can be measured would prompt the promoters of social optimization to accept its limitations and appreciate the qualities of more deeply rooted systems, such as democracy. Alas, they tend to conclude that if a goal or a problem has no measure, it is not worth bothering with. Kevin Kelly, a technology journalist and an apostle of the Quantified Self movement, has faced criticism, as he puts it, that “only intangibles like meaningful happiness count.” His response: “Meaningfulness is very hard to measure, which makes it very hard to optimize.”¹⁴ Similarly, in a critique of the US anti-monopoly movement, the journalist Matthew Yglesias has protested that “‘corporate power’ doesn’t mean anything” on the grounds that it “doesn’t add up to anything measurable or actionable.”

But this is not the first time similar-sounding criticisms have been raised against attempts to perfect society. Where they chose to focus their fire, and where they didn’t, reveals what’s distinctive about the phenomenon of social optimization.

In the 1940s and the early postwar period, the primary target of such critiques was not the private power of corporations. In 1945, the philosopher Karl Popper published The Open Society and Its Enemies, which attacked the utopian certainties of “social engineering,” the belief that politics can and should rebuild society for the better. Popper was taking aim not at private companies but at the state. Invoking the free market philosopher Friedrich Hayek, he wrote, “‘Utopian engineering’ corresponds largely, I believe, to what Hayek would call ‘centralized’ or ‘collectivist’ planning.”

Similarly, in Dialectic of Enlightenment (1944), the critical theorists Max Horkheimer and Theodor Adorno criticized the growth of bureaucratic administration and the emergence of state-managed capitalism. These developments, they argued, displayed a narrow, instrumental reason and a tendency to consider only what could be quantified: “Whatever does not conform to the rule of computation and utility is suspect.”¹⁵ Once again, however, they were focused on the tendency of the state and other powerful centralized organizations to treat individuals as part of a mass. Even when Horkheimer wrote about shopping, in a mid-1960s essay titled “Feudal Lord, Customer and Specialist,” his concern remained broadly the same. He likened customers confronting the “standardized brands” in a department store to factory workers: “To the extent that the individual does not disappear entirely, he is a marginal figure.”

The subtitle of Horkheimer’s essay was “The End of the Fairy Tale of the Customer as King.” Since the 1980s, the fairy tale has returned, and customers have been re-enthroned as kings and queens once more. They are forever on show. There are incessant demands on their attention. And they are constantly told how empowered they are—but this is an illusion they would be suckers to believe.

As the state was weakened and the market became more dominant, new critical thinking emerged in response. The neoliberal Standard Economic Model assumes a world of self-interested, ultra-rational, atomized individuals, all pursuing measurable goals in a world of perfect information. One criticism of “market rationality” is that its model offers a pretty thin account of being human.¹⁶ But this argument took shape before the crucial innovation that underpins social optimization.

Like software engineers’ use of optimization, the Standard Economic Model is predicated on the pursuit of measurable goals, disregarding negative side-effects. Both approaches have a common root in earlier innovations in mathematics. But engineers’ use of optimization only became enmeshed with economists’ use of optimization in the 1990s, as neoliberal economics triumphed geopolitically, and the internet was taking shape.¹⁷

In 2013, Evgeny Morozov identified some of the implications of the ways that tech has promoted the logic of optimization. His book To Save Everything, Click Here is a takedown of what he calls “technological solutionism” and the long-standing belief that “everything that could be fixed should be fixed.” In recent years, he wrote, digital technology made this drive to fix everything “easier, cheaper, and harder to resist”—to the point that “the very availability of cheap and diverse digital fixes tells us what needs fixing.” Drawing on Porter, he identified the role that quantification plays in this process, reducing what counts to what can be counted:

Recasting all complex social situations either as neatly defined problems with definite, computable solutions or as transparent and self-evident processes that can be easily optimized—if only the right algorithms are in place!—this quest is likely to have unexpected consequences that could eventually cause more damage than the problems they seek to address.

However, Morozov did not address optimization’s fusion with the Standard Economic Model. He suggested that Popper’s intellectual comrade Hayek was one of the figures who stood against solutionism—but Hayek was only resisting it in the form of Marxist-style state planning. Hayek’s alternative vision relies on a perfectly free market to optimize an economy through the numerical information that is perfectly communicated by prices.¹⁸ This reasoning follows a similar, potentially utopian logic, particularly when adopted by libertarianism, the favored faith of much of Silicon Valley.

Rather than minimizing the visibility of the individual, as Horkheimer once lamented, social optimization now maximizes it. By gathering huge amounts of personal data, corporations can provide individuals with a highly personalized service. This tacitly encourages people to see corporations as ultra-responsive to their needs, while the sluggish state remains tangled up in the demands of democratic compromise. Software design has begun to coax actual human beings into becoming more like the fully rational self-interest maximizer imagined in the Standard Economic Model.Today, social optimization has brought engineering and libertarianism together to try to impose a complete, privately owned utopian system on the rest of us.

There is a precedent for this that suggests how dangerous and destructive social optimization could become and how we could challenge it. But it lies further back, well before the rise of the centralized state, in a period when, as today, it was corporations that wielded excessive power.

After the Civil War, the US entered the era of the American robber barons, such as the steel magnate Andrew Carnegie and the oil tycoon John D. Rockefeller—men to whom business leaders today, particularly in tech, are often compared.¹⁹ This period offers an earlier example of private power seizing on a neutral theory and overextending it to justify its dominance and its claim to remake the world. The robber barons and their cheerleaders took Charles Darwin’s theory of biological evolution and fallaciously applied it to modern society, with a range of damaging consequences. This was the way of thinking that later became known as “social Darwinism.”

The English philosopher Herbert Spencer incorporated evolutionary theory into his comprehensive system, which claimed to explain how humankind was gradually moving toward perfection. This process required freedom of enterprise, in line with Spencer’s concept of the “survival of the fittest.” In his 1853 essay “Over-Legislation,” for example, Spencer argued that “the apparent insufficiency of private enterprise is a result of previous State-interferences.”²⁰ As the historian Richard Hofstadter put it in 1944, these ideas helpfully provided American capitalists with a high-end intellectual rationale for the laissez-faire approach to economics they wanted to pursue anyway:

With its rapid expansion, its exploitative methods, its desperate competition, and its peremptory rejection of failure, post-bellum America was like a vast human caricature of the Darwinian struggle for existence and survival of the fittest. Successful business entrepreneurs apparently accepted almost by instinct the Darwinian terminology which seemed to portray the conditions of their existence. Businessmen are not commonly articulate social philosophers, but a rough reconstruction of their social outlook shows how congenial to their thinking were the plausible analogies of social selection, and how welcome was the expansive evolutionary optimism of the Spencerian system.²¹

As Hofstadter showed, the argument ran that millionaires were rich because they were fine examples of the fittest surviving, and thriving. Poverty would soon be over if everyone worked as hard as the likes of Carnegie—who, perhaps unsurprisingly, was one of Spencer’s most ardent admirers. And there were genuine parallels between Darwinism and classical laissez-faire economics. According to Hofstadter, both assumed “the fundamentally self-interested animal pursuing, in the classical pattern, pleasure or, in the Darwinian pattern, survival.”

So up to a point, Spencer’s worldview was an argument for free market competition, but it also served to justify the absorption of smaller businesses by huge trusts such as Rockefeller’s Standard Oil, a process that concentrated wealth and power in ever fewer hands. And Hofstadter set out how social Darwinist ideas informed further expressions of damaging dominance: white supremacism, imperial adventures, the eugenics movement.

Yet soon enough, other American thinkers showed that Darwin’s ideas were wide open to a range of interpretations. Some rejected the basic premise that Darwin’s theories about the evolution of all earthly life over eons could shed much useful light on, say, industrial relations in Pittsburgh. Other ideological tendencies also extended Darwinism—the Social Gospel movement, Marxists, the pragmatist philosophers—but reached radically different conclusions.²²

The waning of social Darwinism that followed suggests one possible future for social optimization. Perhaps its overextension of a useful idea will generate similar damage, mockery, and resistance—and prompt a search for other, less hubristic ways to make use of the basic concept. One important element of optimization in economics is the inclusion of constraints, so perhaps this could help motivate a move away from the full-speed, at-all-costs race into the AI future in favor of an approach that includes more consideration of the risks.

It might also be useful to remember that when Dantzig created the simplex algorithm, he was not running a start-up. He was a planner for the state. There is no reason that the objective of optimization need be individualistic. The role of constraints in economics’ use of optimization is flexible. It is perfectly possible to set stern limits on economic inequality and accept significant efficiency losses in return for more egalitarian outcomes. Perhaps if optimization can be decoupled from the notion of the individual ruthlessly pursuing their self-interest, it would be less of an ideological entrenchment mechanism. Amid a rising climate emergency, optimization might, for instance, prove vital to minimizing the waste of resources.

Governments today do have something to learn from Dantzig’s insistence on the importance of having a clear objective. But his overly dim view of leadership needs to be constrained. It is increasingly clear that, in the less calculable areas of life, a leader exercising human judgment is preferable to an implacable optimization algorithm. Without such human-centered constraints in place, social optimization won’t make things better—only more extreme.

Phil Tinline is the author of The Death of Consensus: 100 Years of British Political Nightmares and Ghosts of Iron Mountain: The Hoax of the Century, Its Enduring Impact, and What It Reveals About America Today.

1. George Dantzig, “Reminiscences About the Origins of Linear Programming: Technical Report 50L 81-5” (Stanford University, April 1981), 3. ↩︎
2. Donald J. Albers, Gerald L. Alexanderson, and Constance Reid (eds.), More Mathematical People: Contemporary Conversations (Harcourt Brace Jovanovich, 1990), 73. ↩︎
3. Dantzig, “Reminiscences,” 3. ↩︎
4. Albers et al, More Mathematical People, 73. ↩︎
5. Albers et al, More Mathematical People, 73. ↩︎
6. Francisco J. André, M. Alejandro Cardenete, and Carlos Romero, Designing Public Policies: An Approach Based on Multi-Criteria Analysis and Computable General Equilibrium Modeling (Springer, 2010), x. ↩︎
7. Joaquim R. R. A. Martins and Andrew Ning, “A Short History of Optimization” in Engineering Design Optimization (Cambridge University Press, January 2022), Chapter 2.3. ↩︎
8. Jeremy Weinstein, Mehran Sahami, and Rob Reich, System Error: Where Big Tech Went Wrong and How We Can Reboot (Harper, 2021), 11. Kindle. ↩︎
9. Dan Wang, Breakneck: China’s Quest to Engineer the Future (Allen Lane, 2025), xv. Kindle. ↩︎
10. Theodore Porter, Trust in Numbers: The Pursuit of Objectivity in Science and Public Life (Princeton University Press, 2020), 76-77. ↩︎
11. Wang, Breakneck, 6. ↩︎
12. Adrian Hon, You’ve Been Played: How Corporations, Governments and Schools Use Games to Control Us All (Swift Press, 2023), 66-78. ↩︎
13. Gil Duran, “Trump’s Gaza Fantasy and the Network State: The Tech-Fueled Future of Privatized Sovereignty,” Tech Policy Press, June 6, 2025. See also, for example, Reich et al, System Error,22; and Peter Thiel, “The Education of a Libertarian,” (Cato Unbound, April 13, 2009), in which he writes: “I no longer believe that freedom and democracy are compatible.” ↩︎
14. Evgeny Morozov, To Save Everything, Click Here: Technology, Solutionism and the Urge to Fix Problems That Don’t Exist (Allen Lane, 2013), 244. Kindle. ↩︎
15. Max Horkheimer and Theodor Adorno, Dialectic of Enlightenment (Verso, 1997), 5-6. ↩︎
16. See, for example, some of the arguments summarized in Jane B. Baron and Jeffrey L. Dunoff, “Against Market Rationality: Moral Critiques of Economic Analysis in Legal Theory,” Carozo Law Review (Vol. 17, 1996). There are elements of this argument in James C. Scott, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed (Yale University Press, 1998). For a later, conservative example of this critique, seeJesse Norman, The Big Society: The Anatomy of the New Politics (University of Buckingham Press, 2010). ↩︎
17. Phil Tinline, The Digital-Democratic Doom Loop: Social Media and the Breaking of the State-Citizen Relationship (Demos, February 2026), 12. ↩︎
18. Friedrich Hayek, The Road to Serfdom (Routledge, 2001), 38, 97. ↩︎
19. See, for example, Tim Wu, The Curse of Bigness: Antitrust in the New Gilded Age (Columbia Global Reports, 2018) and Steve Fraser, Mongrel Firebugs and Men of Property: Capitalism and Class Conflict in American History (Verso, 2019). ↩︎
20. Herbert Spencer, “Over-Legislation,” in The Man and the State: With Six Essays on Government, Society, and Freedom (Liberty Fund, 1982), 302. ↩︎
21. Hofstadter, Social Darwinism in American Thought, 44. ↩︎
22. Hofstadter, Social Darwinism in American Thought, Chapters 6 and 7. ↩︎