In today’s post, I am inspired by the ideas of cybernetics, Heidegger, and Taleb. I am looking at what I think is the largest danger of large language models (LLMs).

LLMs are extraordinarily proficient in the domain of language, and that proficiency has quietly created a philosophical problem that most technical discussions fail to notice. They speak fluently, respond coherently, adapt style with ease, and generate text that fits seamlessly into human patterns of explanation and reflection. The danger does not lie in what they get wrong, but in how convincing they are when they appear to get things right. Fluency triggers attribution, and attribution tempts us to confuse linguistic competence with lived understanding. We humans are prone to jump to attributions. We seek purpose in anything and everything.

This confusion is not accidental. Throughout human history, language was never free-floating. It was always grounded in lives that could fail, bodies that could be injured, and situations that demanded response. To speak well was already evidence that one had survived something, endured something, or at least stood in a chain of experience anchored in the world. With LLMs, that historical coupling has been severed. What remains is language without a life.

Cybernetics offers a useful distinction here by separating the domain of coordination from the domain of perturbation. The domain of coordination is where language operates. It is the space of symbols, signs, instructions, and representations that allow systems to align behavior. The domain of perturbation is where the world asserts itself. It is the space of forces, constraints, breakdowns, and consequences that threaten the continued viability of a system. Living systems exist in both domains simultaneously. LLMs exist almost entirely in the former.

Please note that Maturana and Varela talked about languaging as a form of cognitive existence. To me, languaging is existing in both domains simultaneously. This is often missed in the discussion of AI or AGI. The domain of coordination must not be conflated with the domain of perturbation and of lived experience.

Hunger, pain, cold, loss, and social rejection are not messages waiting to be interpreted. They are disturbances that demand coping. They do not coordinate with us and they push back. A living entity must respond to these perturbations in order to remain viable, and it is precisely this necessity to cope that gives rise to meaning. Language emerges as a secondary achievement, a tool for coordinating responses to disturbances that have already been encountered. The “burn” comes before the word “hot.”

LLMs start in the inverse order and they stop where they start. They begin with language and never leave it. They operate in a closed probability space in which words refer only to other words and coherence is rewarded independently of consequence. They do not encounter resistance. They do not face breakdown. They do not have to repair themselves in response to failure in any existential sense. When they are wrong, nothing is at stake for them. No structure is threatened. No viability is endangered. There is no loss to mourn and no urgency to learn.

The existential ideas from Heidegger are indispensable here, because he refuses to separate understanding from involvement. For Heidegger, human existence is not primarily a matter of representation or cognition, but of being-in-the-world. We do not stand outside the world describing it. We are thrown into it. We do not choose the conditions of our arrival, the historical moment, the social structures, or the biological constraints we inherit. We find ourselves already entangled in demands that must be met before they can be explained.

Thrownness is not merely a description of origin. It names the condition that makes understanding possible at all. Because we are thrown, we must cope. Because we must cope, we care. Care, for Heidegger, is not an emotional add-on. It is the structure of existence itself. To be human is to have something at stake, to be concerned with how things turn out, because how they turn out matters to whether we can continue at all. Simply put, LLMs do not care.

Coping is not a flaw or a limitation. It is the source of cognition. We understand the world not by representing it accurately in advance, but by engaging with it practically and discovering, often painfully, where our expectations fail. Heidegger’s notion of ready-to-hand captures this vividly. Tools disappear into use when coping is successful. They become visible only when something breaks. A hammer reveals itself as a hammer not when it works smoothly, but when the handle snaps and the task can no longer continue.

This breakdown is not an interruption of understanding. It is its condition. Reality teaches by resisting us. Please note that I am taking poetic license here and my use of words should not be confused with attribution. Plans collapse, models fail, and language stops working. And in those moments, distinctions begin to matter. We learn what is important because something went wrong. Only cognitive entities that can be broken can learn what matters. This is simply the living condition.

LLMs cannot break in this way. They are not thrown and they do not cope. They do not have to maintain their own viability in the face of an indifferent world. They do not care, not because they are unethical or incomplete, but because care arises only where existence is at risk. This is why appeals to giving them more data or richer representations miss the point. The difference is not quantitative. It is categorical.

This also clarifies why certain philosophical slogans become dangerous when misapplied. Wittgenstein’s line, “The limits of my language mean the limits of my world,” is often invoked to suggest that language generates experience. But Wittgenstein was speaking about humans, beings already embedded in the world, already coping, already affected. Language limits what can be articulated about experience. It does not produce experience itself. To apply this claim to a system that has language without world is to collapse experience into description and mistake coordination for contact.

I have often noted the temptation to treat information as a commodity. We are prone to think that if language is sufficient for understanding, then knowledge can be accumulated without exposure, transferred without risk, and optimized without consequence. But cybernetics resists this move. Understanding is not stored in representations. It emerges in systems with histories of interaction, failure, and recovery. Meaning arises where distinctions have consequences.

This is where I bring in Taleb’s notion of “skin in the game,” which aligns naturally with both cybernetics and Heidegger. Knowledge without exposure to consequence is brittle. Assertions made without risk are cheap. LLMs produce language without skin in the game by design. The danger arises when humans, who do live in the domain of perturbation, begin to orient themselves toward that language as if it carried the weight of lived understanding.

Final Words:

The real risk is not that machines will become more human, but that humans will begin to forget what makes their own understanding possible. When linguistic fluency is privileged over coping, when clean summaries are trusted over messy lived accounts, when the map is preferred because it is easier than the territory, we drift away from the conditions that give meaning to knowledge at all.

Since LLMs do not cope, and have no need for care, we should not assign them responsibilities where care is required. Care is not a functional add-on that can be simulated through better language or richer models. It arises only where existence is exposed to risk, where failure has consequences, and where something can be lost. There is currently a great deal of discussion about the role of AI and the possibility of AI replacing humans. Much of this discussion quietly assumes that agency can be transferred wherever competence appears. This is a serious mistake!

The use of LLMs lies exclusively in the domain of coordination and language. They can generate possibilities, assist with articulation, and operate within representational spaces at a scale no human can match. But to conflate this capability with human agency, and to assign responsibilities that presuppose care, concern, or accountability, is a terrible idea. Responsibility belongs to entities that can be held answerable by the world, because the world can push back on them.

Only entities that can be broken can learn what matters. Humans are such entities. This vulnerability is the ground of responsibility, ethics, and meaning. The task, then, is not to teach machines how to live, but to remember, in the presence of their fluency, that living is something language can point to but never replace.

Stay curious and Always keep on learning…

If you liked what you have read, please consider my book “Second Order Cybernetics,” available in hard copy and e book formats. https://www.cyb3rsyn.com/products/soc-book

Note:

In referencing the work of Martin Heidegger, I want to acknowledge the deeply troubling fact of his affiliation with the Nazi party. This aspect of his life casts a long and painful shadow over his legacy. While I draw on specific philosophical ideas that I find thought-provoking or useful, this is not an endorsement of the man or his actions. Engaging with his work requires ethical vigilance, and I remain mindful of the responsibility to not separate ideas from the broader context in which they were formed.