Why "Trusting AI" is an Illusion: On the Epistemology of Human-Algorithm Relations
On human agency, relational intelligence, and the myth of the sovereign machine
As we look toward a post-AI world, contemporary discourse is paralyzed by an epistemic crisis. Confronted by intelligent machines and LLMs capable of long-horizon reasoning, humans are deeply preoccupied with fundamental questions of truth, authority, and purpose: Whom should we believe when the world is flooded with synthetic text? Where does human value live if cognition is no longer uniquely ours?
But under our discourse lives an underlying bias. We are operating under what organization and management scholars call the “Entity View” of AI: the assumption that artificial intelligence is a distinct, sovereign agent residing within an algorithm, possessing fixed properties that precede any human relationship. This is a limiting and misleading framing, challenged directly in a seminal 2026 paper by Marta Stelmaszak et al., where researchers propose an ontological shift to see AI not as a fixed entity, but as a capability that only arises through humans’ active relationships with learning algorithms.
In fact, rather than being a sovereign intelligence residing within a machine, AI is understood as an organizing capability that emerges from a system of complex relationships among human and learning algorithmic actors, all enacted in pursuit of organizational goals. This “Capability View” opens the door to a profound truth: AI never operates in isolation, meaning that “trusting AI” as if it were an independent entity is a conceptual illusion.
Reconceptualizing the Locus of Intelligence
Intelligence is not structurally “planted” inside of a machine; it is actively produced and enacted across a collective, hybrid system of relations by human choice and design. In this framework, AI is an organizing capability that does not reside inside the algorithm; rather, it materializes dynamically through the enacted relations between learning algorithms and humans, and the entanglement between human and algorithmic actors. Intelligence becomes something that is actively produced in practice.
The authors define AI’s organizing capability through three core properties that explain how a system of human-algorithmic relations operates:
1. Connectivity (The Power of Enactment)
An algorithm sitting unused in isolation cannot constitute AI. An AI capability only materializes when human-algorithmic relationships are actively enacted in practice. The paper highlights empirical cases where professionals like radiologists and police intelligence officers, bypassed or rejected algorithmic recommendations. In those moments, the human actors were not forming a system of relations with the algorithms, and so AI was not present and decisions reverted to human actors alone. Human engagement, as illustrated in these examples, is the condition that allows AI to arise.
2. Codependence (The Myth of the Sovereign Machine)
The capability view systematically deconstructs the myth of a standalone machine intelligence. The paper draws an important distinction here: “traditional” rule-based algorithms, or those that simply execute predefined human instructions faster, do not constitute AI. For AI to emerge, human actors must enter into relations with learning algorithms: those that store history, update their parameters based on outcomes, and adapt future behavior accordingly. Even these more advanced systems remain codependent on human actors. Algorithmic actors cannot set goals, and their learning is bounded by the data, structures, and objectives shaped by human actors. Equally, human actors alone, without the learning algorithms, do not give rise to AI. The organizing capability belongs to the relational network, and no single entity alone can possess it.
3. Emergence (The Living Technical Habitat)
Because these human-algorithm interactions occur continuously, the entire system is perpetually fluid. Humans adapt their behavior in response to the algorithms, and the algorithms inductively learn, and adjust internal parameters based on human input over time. Both actors are changed through their relations with each other. AI is thus an open-ended process of “doing and becoming”.
The Epistemic Takeaway: Reclaiming Human Agency
When we evaluate an algorithmic claim to “truth” or “intelligence,” treating it as an output of an autonomous machine is a category error. Because AI is fundamentally a relational capability, choosing to believe an AI output is never an act of trusting an independent machine entity. Instead, it is an act of trusting the human, institutional, and developer relationships co-constituting that system. The machine acts as a prism, refracting human choices, developer intentions, and historical data patterns.
This ontological shift provides a reassuring answer to the anxieties of a post-AI world. The paper is explicit: human actors are always a necessary condition for AI to arise, even in highly automated settings, human participation in design, training, and deployment remains constitutive. Human intentionality is structurally inseparable from the intelligence that emerges.
To preserve the conditions for a fully human life for future generations, we must look past the illusion of the autonomous machine and actively accept our responsibility as the moral anchors co-constituting the systems of relations we inhabit, because it is precisely those relations, not the algorithms, that give rise to whatever intelligence we are so eager to trust.
References
Stelmaszak, M., Joshi, M., & Constantiou, I. (2026). Artificial intelligence as an organizing capability arising from human-algorithm relations. Journal of Management Studies, 63(2).