Nicholas Carr:

Evaluating works of art, or any creative products of skill and imagination, requires a mind, which the machines of media automation lack. Identifying a pattern requires only a statistical procedure, which is what the machines have. Indeed, it’s all they have. People like MrBeast didn’t crack the code of virality. Machines did. MrBeast’s great strength as a contemporary creator is that he has no ambition beyond repeating a pattern. He’s a machine-listener. He attends to the machine, and he does what it tells him to do.

Complete value capture — no ambition beyond chasing what can be measured.

Nicholas Carr:

As the older generations die, they take with them their knowledge of what was lost when the new technology arrived. Only the sense of what was gained remains. It’s in this way that progress covers its tracks, perpetually refreshing the illusion that where we are is where we were meant to be.

Instagram is a media production company:

The story of social media ever since has been a story of the refinement of feeds as a media product aimed at capturing and holding an audience. The platforms have invested billions of dollars in designing those feeds—what they contain, how they look, how they work—to make them as “engaging” as possible. To argue that the companies are still in the business of transmitting “user-generated content” is absurd. They’re not carriers anymore; they’re media companies. Yes, users still contribute posts and comments—though even those, in today’s era of influencers, creators, and AI, are often subsidized and actively shaped by the companies—but the essential content of social media is now the feeds produced by the platforms, not the individual messages posted by users. Go to Instagram and scroll through your feed. It’s obvious that what you’re experiencing is not discrete bits of user-generated content. It’s an elaborate, finely tuned media production manufactured by Instagram for an audience of one: you

What I hate most is when companies try to turn my media into a feed for their product.

The feed is the content, and the social media company is its publisher. Period.

You’ve heard of Nick Bostrom’s paperclip maximizer? A super-intelligent AI is tasked with making paperclips and, in single-minded pursuit of that goal, converts all available resources — including humanity — into paperclips.

Nicholas Carr puts a spin on that thought experiment and concludes we’re already living in that reality — except it’s us who have become the maximalizers.

Bostrom’s story, I would argue, becomes compelling when viewed not as a thought experiment but as a fable. It’s not really about AIs making paperclips. It’s about people making AIs. Look around. Are we not madly harvesting the world’s resources in a monomaniacal attempt to optimize artificial intelligence? Are we not trapped in an “AI maximizer” scenario?

Oh.

Elon Musk, having abandoned his earlier misgivings about AI, announced last week that he was merging xAI into SpaceX. The combined companies were “scaling to make a sentient sun to understand the Universe and extend the light of consciousness to the stars!” he declared. “In the long term, space-based AI is obviously the only way to scale.” It’s exactly what Bostrom predicted. The monomaniacs will not stop with the resources of the Earth. They’ll extend their plundering to the heavens. Everything is raw material.

We have met the enemy and he is us.

Nicholas Carr touching on why it’s good business to erase “IRL”:

We want the speech of distant people to arrive in our mailbox, to issue forth from our radio and TV, to hang on the walls of a museum, to appear on the screen of our phone. Take away such freedom of movement, return us to the original communication system of mouth and ear, and you take away knowledge, culture, entertainment, pretty much the entirety of modernity.

Erasure is good for business. The more that media has erased the world, the more dependent society has become on the systems and services of media companies and the more profits those companies have earned.

Conclusion:

The more we draw on AI to shape our perception and understanding of the world, to structure our thoughts and words, to express ourselves, the more complicit we become in erasing culture, the past, others, ourselves.

Nicholas Carr on “AGI”:

what AGI really stands for [is] artificial generalizing intelligence. It encompasses the mind’s analytic ability to see common patterns in different things or phenomena and to derive general categories or rules from them. But it excludes all the aspects of intelligence that free us from the constraints of rules and patterns: imaginative thinking, metaphorical thinking, critical thinking, contemplation, aesthetic perception,

If you can’t actually make something intelligent, just change the definition of the word:

They reduce intelligence to that which their machines can do and then claim their machines are intelligent.

And this “intelligence” is not personal. It’s corporate:

the problem of automated abstracting came to be solved not with elegant theories of cognition and language but with the brute-force application of quantities of data and computer power

The personal computer is being surpassed by the cloud computer. What computers can do for humans will no longer be an individual issue but a corporate one. Whoever owns the most compute, owns the most customers.

It’s like shovels aren’t important anymore. Only backhoes and dynamite.

[we] adopted computer systems as the fundamental conduit of thought and culture…Those who control the systems control much about us. Their flaws and shortcomings are built not just into the technology but, increasingly, into society’s norms and practices. Just as the brilliant but socially maladroit Mark Zuckerberg came to set the terms for how we socialize today, so the brilliant but intellectually crippled designers of contemporary AI systems seem destined to set the terms for how we think.

Nicholas Carr:

little data—all those fleeting, discrete bits of information that swarm around us like gnats on a humid summer evening.

Our apps have recruited us all into the arcane fraternity of the logistics manager and the process-control engineer…[of] our own existence.

What we don’t see when we see the world as information are qualities of being—ambiguity, contingency, mystery, beauty—that demand perceptual and emotional depth and the full engagement of the senses and the imagination. It hardly seems a coincidence that we find ourselves uncomfortable discussing or even acknowledging such qualities today. In their open-endedness, they defy datafication.

We love UIs because they make us the star. In a UI, you are the center of the world. In the world, you are a speck, just one of many.

Little data tell us little stories in which we play starring roles. When I track a package as it hopscotches across the country from depot to depot, I know that I’m the prime mover in the process—the one who set it in motion and the one who, when I tear open the box, will bring it to a close. That little white arrowhead traveling so confidently across the map on the dashboard? That’s me. I’m going somewhere. I’m worth watching. When I monitor the advance of a song’s progress bar, I know I can stop the music anytime, purely at my whim. I’m the DJ. I’m the tastemaker. I say when one tune ends and the next begins. So lovingly personalized, so indulgent, little data put us at the center of things. They tell us that we have power, that we matter.

Thus:

when we communicate using little data, we’re speaking the language of robots.

The real threat AI poses to education isn’t that it encourages cheating. It’s that it discourages learning.

The result of writing is a proxy for the process of learning.

the pedagogical value of a writing assignment doesn’t lie in the tangible product of the work — the paper that gets handed in at the assignment’s end. It lies in the work itself: the critical reading of source materials, the synthesis of evidence and ideas, the formulation of a thesis and an argument, and the expression of thought in a coherent piece of writing. The paper is a proxy that the instructor uses to evaluate the success of the work the student has done — the work of learning. Once graded and returned to the student, the paper can be thrown away.

Generative AI enables students to produce the product without doing the work.

The output is not the product.

While the output of any given course is student assignments — papers, exams, research projects, and so on — the product of that course is student experience.

The utility of written assignments relies on two assumptions: The first is that to write about something, the student has to understand the subject and organize their thoughts. The second is that grading student writing amounts to assessing the effort and thought that went into it.

The work of learning is hard by design — unchallenged, the mind learns nothing

Armed with generative AI, a B student can produce A work while turning into a C student

We’ve been focused on how students use AI to cheat. What we should be more concerned about is how AI cheats students.

This comment from a student:

“I literally can’t even go 10 seconds without using Chat when I am doing my assignments. I hate what I have become because I know I am learning NOTHING, but I am too far behind now to get by without using it . . . my motivation is gone.”

Nicholas Carr outlines media as a system for the distribution of communication, inclusive of three parts:

1. Message creation (humans)
2. Message selection (humans, editorial)
3. Message transmission (machines)

No. 3, message transmission, has been playing out in various ways through mechanization:

stringing telegraph or telephone lines across a continent, building switching networks for the routing of phone calls, establishing broadcasting networks for the transmission of radio or television signals, creating protocols for compressing signals or instruments for amplifying them

No. 2, message selection, used to be all human. But when Facebook rolled out the News Feed circa 2006, things started to change. Mechanization began to take over the editorial process.

[Facebook News Feed] automated the selection of messages for distribution to an audience. The editorial function was no longer the exclusive purview of human beings. It had suddenly become part of the engineering problem, a matter of user-profiling systems, prediction algorithms, and other software routines. Even though the machines — networked digital computers — had no sense of meaning themselves, they were now making decisions about meaning, semantic decisions that determine the content people see or do not see.

Soon it wasn’t just Facebook. Everyone began implementing a mechanized (i.e. algorithmic) editorial process. Its effect?

Society, we discovered, was neither prepared for nor capable of addressing the automation of meaning-making.

Then OpenAI released ChatGPT and mechanization reached for the last human process, no. 1: message creation. Now meaning-making isn’t exclusively to humans.

It’s a challenge of our own making and one we’re still entirely unprepared for.

Like so many other things we’ve invented.

I like these two quotes Nicholas Carr uses from Norbert Wiener.

First, an observation:

The machines will do what we ask them to do and not what we ought to ask them to do.

Second, a story:

Some years ago, a prominent American engineer bought an expensive player-piano. It became clear after a week or two that this purchase did not correspond to any particular interest in the music played by the piano. It corresponded rather to an overwhelming interest in the piano mechanism. For this gentleman, the player-piano was not a means of producing music, but a means of giving some inventor the chance of showing how skillful he was at overcoming certain difficulties in the production of music. This is an estimable attitude in a second-year high-school student. How estimable it is in one of those on whom the whole cultural future of the country depends, I leave to the reader.

This story’s relevance to technology I also leave as an exercise for the reader.

I did an exhaustive search of every opinion column published by major U.S. newspapers since the recent election, and I found that the phrase “vibe shift” appears 4,322 times. That number is a complete fabrication — I did no such search — but I would argue that it gets the vibe right.

Vibes are a kind of weather that emerges from the immaterial flow of information in our digital world.

Out of the chaos of messages some viral wind emerges. It blows in one direction until its force dissipates and other winds begin to blow in other directions. The source of the wind remains obscure. It’s a vibe.

I know I always say it, but Nicholas Carr as incisive about tech as ever:

Handing off authority for fact checking to “the community” has practical advantages for Meta, as it did for X. The community doesn’t send invoices.

Fact checking, like content creation or open source, is unpaid labor. Democratizing it 1) lowers the cost, and 2) provides you a scape goat: the supplier.

Chalk it up to yet another misunderstanding by Silicon Valley: everything does not get better as it gets bigger and more streamlined.

The desire to “scale” fact checking, to mechanize the arbitration of truth, is just another example of the tragic misunderstanding that lies at the core of Silicon Valley’s entire, grandiose attempt to remake society in its own image: that human relations get better as they get more efficient.

Nicholas Carr, as great as ever.

The question for anything in our day is: can you make money on it?

the entertainment industry has long been obsessed with bringing the dead or merely defunct back to profit-generating life.

If you can give the consumer what they’re used to consuming, that’s “good enough”.

They may, to quote Mander, “contain no life at all,” but that doesn’t matter as long as they offer a reasonable simulation of what the consumer is used to consuming. Slop’s good enough.

Take music, for example. We don’t need albums from individuals or groups of individuals, generic vibes are good enough.

Because the songs, again, feel like the genres listeners are familiar with, they stream by unnoticed. Cue up Chill World Vibes or Lazy Sunday Morning, and let it flow.

All of this is interesting when viewed through the lens of “supply side” economics in social media:

Discussions of feed algorithms tend to focus on the demand side — the matching of a bit of content to an individual consumer through the instantaneous analysis of the triggers of that consumer’s behavior. Less attention has been given to the supply side — the sourcing of bits of content through the instantaneous analysis of the cost of the content to the supplier. In contrast to traditional media companies, which produce and distribute a fairly limited set of offerings, social-media platforms operate vast, complex cultural supply chains that have to be optimized to generate profit through a multitude of tiny informational transactions. Because they deliver billions of bits of content every moment around the clock, it’s imperative they find the cheapest possible content to feed to consumers.

It’s the real life equivalent of the production supply chain: optimize, optimize, optimize, cost over everything else.

What Spotify has been engaged in — and it’s hardly alone — is a large-scale experiment to test the fungibility of culture. How far can we go in replacing creative work (and the artists who create it) with manufactured slop? With generative AI, the scale of that experiment is going to get much, much larger. By automating content farming, platforms will be able to further drive down the cost of content — and further reduce their reliance on actual artists. More than that, they’ll be able to generate the content in real time, custom-fitted to individual demand. The supply is unlimited.

It’s a giant A/B test on all of us, to see whether we’ll be satiated with something that’s cheaper for the supplier even though we didn’t demand it.

What’s really being tested here is human taste. Will we accept a simulacrum of a work of art or craft as a satisfactory substitute for the real thing? Will we even be able to tell the difference?

At the dawn of digitization in 1995, the French philosopher Jean Baudrillard observed, “Words move quicker than meaning.”

Now with the advent of AI, words, images, and video are created quicker than their meaning.

What happens to us when all of this information is created and proliferated faster than anyone can really make sense of it?

There will be only one thing left to automate: meaning. The last task of the computer is to give our lives meaning.

Here’s Nicholas Carr:

[my three books] examine how we came to apply industrial ideals and measurements — efficiency, productivity, speed, profit — to the most essential and defining of human pursuits. [One] looks at the application of those ideals and measurements to thinking; [another], to doing; and [the third], to communicating. The way computer systems have abetted the encroachment of the industrial ethos into the most intimate facets of human life strikes me as one of the most important stories of our time.

The human brain itself — that mysterious maker of metaphors — has through the ages been portrayed as (a) a hydraulic pumping system, (b) a clock, (c) a telephone switching network, (d) a digital computer, and, now, (e) a large language model. In constructing machines, we also construct ways of seeing the world, and ourselves.

We build all these systems and we complain about them as if they’re out of control, as if they’re controlling us, but we build them.

Analogous to a code base, no?

We complain about the state of the world, but guess who constructed the world in the state that it’s in?

technology is a repository of human desire, a full critique of any machine needs also to be a critique of human desire. We’re the machine’s makers before we’re its victims.

Could someone please get me a mirror.

The human brain…has through the ages been portrayed as (a) a hydraulic pumping system, (b) a clock, (c) a telephone switching network, (d) a digital computer, and, now, (e) a large language model. In constructing machines, we also construct ways of seeing the world, and ourselves.

Our tools are mirrors to ourselves.

We build all these systems and we complain about them as if they’re out of control, as if they’re controlling us, but we build them.

[Insert analogy here to complaints about the state of the web by web developers.]

technology is a repository of human desire, a full critique of any machine needs also to be a critique of human desire. We’re the machine’s makers before we’re its victims.

A horizontal frame places a person in a landscape. It emphasizes the ground in which the figure stands. It provides context…Verticality erases the landscape, the ground, the context. The figure stands alone, monumental in its solitary confinement.

We change our tools and they change us.

An expectation of abundance breeds profligacy, a willingness to waste things. An expectation of scarcity breeds frugality, a concern with using things judiciously.

Makes me wonder: What can bring an expectation of scarcity to the way we build on the web?

Don’t worry about all the energy we’re sucking into our data centers, because our data centers are going to make energy free. Don’t worry about all the money we’re amassing, because we’re going to make everyone prosperous. Don’t worry about all the time you spend looking into the screens we’ve given you, because through those screens lies paradise.

Remember when one of the arguments for “going digital” was its eco-friendly nature? “Think of all the trees you’ll save!” Now we have AI companies that want to go nuclear for power.