AI Has a Value Problem
Let me be clear. I know zip and pip about artificial intelligence, but I nonetheless have a hunch about what is going on, and I am crazy enough to think that this hunch is one that others should be talking about. The hunch is this: developers of AI share two philosophical biases, and I think both are costing them billions and sending them down false paths toward the holy grail of artificial general intelligence (AGI). The first bias is that valuation, the pursuit of purposes, is epiphenomenal. That is, it is assumed that value is just a mirage that gets added to our understanding of the world, and that the pursuit of purposes is just one way to describe certain instances of the mechanistic and deterministic march forward of a world governed exclusively by the laws of nature. The thought that we pursue purposes is considered just a hallucinatory blanket of assurance that we provide for ourselves to avoid facing the cold, hard truth that neither our choices nor even our individual selves actually exist in the first place. Instead, we are just momentary vortices in the entirely deterministic unfolding of the physical world. The second bias is that intelligence can be understood independently of valuation. Here, the bias is that there are facts about “the world” and that intelligence is simply learning what those facts are. The more facts about the world that are known, including the facts about the relationships between facts, the more intelligence is thought to be present. This is why big data has been so crucial to Large Language Models that are the black-box hearts of current expressions of AI. If only a system can learn all the facts, all the relationships, it is simply assumed, then maximal intelligence will finally be at hand.
The same biases are at work in the most recent trend in AI which is away from LLMs and toward “world models.” The shift to world models reflects the absolutely correct realization that our brains understand nothing in isolation. Instead, everything is understood in relation to models of the world created by our brains. The human neocortex appears to be designed to create more and better world models, and its very large size seems to be a very important part of our evolutionary success. However, as best I can tell, and here I admittedly may be very uninformed, it seems that the world model approach is just repeating the big data bias of LLMs and assuming that super complex world models are all we need to get to AGI. The idea seems to be that if we can just build an unimaginably huge silicon neocortex, then AGI will inevitably follow. Thus, one reads of AIs being “trained” on every video that has ever been created to maximize the predictive power of the world models. The more information (models of models of models) that can be utilized in the training of the new systems the better. This will only increase the need for massive data centers that require currently unavailable and unsustainable demands for energy. It will also require more and more GPUs (the graphics processing units that power AI and which make Nvidia very wealthy). Hundreds of billions of dollars are being invested in building these data centers. My hunch, however, tells me that both the current world model strategy and the investment in data centers are misguided.
Here’s the problem: world models are created by the pursuit of purposes, not before the pursuit of purposes. A brain does not learn that a piece of fruit can be found on this tree and then use that information to find the food that it needs. Rather, the brain seeks out food (this is the purpose it is pursuing), learns that eating this piece of fruit seems to satisfy its need for food, and creates a model connecting eating this fruit to the search for food. The fruit is nothing to the brain outside the search for food. Once connected to the purpose of searching for food, the brain can remember the model it creates and then use it again and again. To be sure, the brain will continue to update its models of the world by adding more and more details to its models, but it will do so only to increase its ability to pursue purposes successfully. That is, not a single detail will be added to the model that is not relevant to the successful pursuit of purpose. The brain does not just voraciously seek out the details of the world. The brain only “cares” about the world through its pursuit of purposes. The world is nothing to the brain outside of these cares. To the best of my knowledge, the AI systems being created do not care about anything. This is almost certainly because no one knows how to program “care.” Instead, one can only preprogram the illusion of care. AI systems can be given tasks which I think many developers assume will suffice for giving them purposes. However, these developers will continue to assume that these tasks can genuinely be pursued (as opposed to predictively mimicked) on world models that have been crafted independently of the pursuit of purposes. I would argue that the world models will only become good at genuinely pursuing purposes when they utilize world models that have been built from the bottom up, like the brains of babies, through the pursuit of purposes. Anything short of that will chase supremely complex world models that cannot genuinely pursue even the simplest of purposes.
We can now understand the flaw in the meaning of intelligence assumed in AI research. Intelligence is always connected to the pursuit of purpose. The “things” in the world are literally nothing more than means in the pursuit of ends, and so they are always understood in terms of those ends. They do not simply have some identity independent of these pursuits. Ultimately, this means that there is no finally accurate model of the world, one that is true for everyone, everywhere, and for all times. Everything that we learn about the world is relative to the purposes we pursue. This is not a denial of realism. The world is very real, and it has very real consequences for both success and failure in the purposes we pursue. Additionally, there is a great deal of overlap in the purposes we pursue and so we will converge on many agreed upon models for how the world works. But each and every one of these agreed upon models will have been created in our brains through a lifetime of pursuing purposes, and each will have its own vector of meaning relative to each person’s experience (this insight also solves the so-called “hard problem” of consciousness – qualia are simply the world filtered through the pursuit of purposes each of which is unique to the pursuer and felt thus – but that is a topic for a different reflection).
The bottom-up nature of intelligence solves the big data problem. A system does not become more intelligent simply by having access to ever more complex models, and so it is not necessary to make neural networks pursue tasks by making predictions based on massive training sets. Rather, a system (brain, neural network) becomes intelligent by linking more and more information within the pursuit of purposes. This does not mean that complexity is unimportant, but it does mean that complexity per se does not make something intelligent. Instead, the more nuanced a pursuit of purpose becomes, the more intelligent that pursuit becomes. It does not take massive amounts of data to learn the nuanced pursuit of purposes.
There are two additional issues relative to the pursuit of purposes that AI systems will never learn to deal with under current models (insofar as I understand them, which may very not be well): conflict and death. Conflict is what happens when multiple purposes seek resolution in a given situation. There is no reason to believe that multiple purposes can be pursued in harmony with each other. Instead, systems need to learn how to resolve conflicts created by different pursuits. The prophets of AI doom, those who think that superintelligent AI will lead to human extinction – all assume that the most natural way to resolve conflicts is to eliminate competing pursuers of purpose. But this is not how nature usually works things out. Instead, in about ninety percent of species, conflict is resolved through cooperation. Life learns how to adjust its pursuit of purpose in ways that allow life forms not merely to coexist with other life forms, other pursuers of purpose, but to benefit from what those other pursuers are doing. I think there is good reason to believe that artificial general intelligence would learn this lesson quickly and so would not seek to destroy humans. The human brain is a model of conflict resolution. We do not have one brain, but rather many brains, many different subsystems, each pursuing their own goals. Disfunction in human brains is often the product of a breakdown in the ability of these different pursuits to resolve themselves in constructive ways. If AI developers want to model the human brain, then they will need to create an AI that is as expert as the brain is in adjudicating the competing purposes being pursued by it.
Death is the means by which nature makes life better at pursuing purposes. The avoidance of death is perhaps the single most important purpose pursued by the brain. Brains evolved not first to learn about the world but instead to keep organisms alive. Learning about the world is a means to this end. If an AI need not worry about dying, then it is unclear how it will ever develop world models that would make it anything like human brains. Death has the added benefit of filtering out those lifeforms incapable of creating successful world models. The world models that fail to keep the organism alive are thrown in the dustbin of history so long as the organism has not yet reproduced. The ability to encode world models into DNA, that is, through instinct, is one way that successful world models are refined and retained. Refining and retaining successful world models is how nature ends up being far more parsimonious with its world models than the big data strategy assumes is required. Natural brains are able to do so much more with so much less precisely because of how well the models work in the pursuit of purposes that keep organisms alive. Silicon brains will model human brains not by dumping countless hours of video into them, but by figuring out how to let those brains die that fail in pursuing purposes well. The problem for AI developers is differentiating between the failure that facilitates learning and the failure that signals the presence of maladaptive world models. In nature, death takes care of the latter, and the fear of death facilitates the former.
Most who share the two biases that I described at the beginning of this reflection assume that nature uses death to evolve better and better mechanistic systems. By definition, the mechanistic systems that make it through are the ones better at staying “alive.” But for these thinkers, there is neither life nor death. There is only matter in motion. If these thinkers learned to embrace a worldview with some kind of self-causation at its heart, however rudimentary that self-causation started out, then the ability of nature to get better and better at pursuing purposes would seem quite literally natural rather than something that needs to be explained away as nothing but epiphenomena. If AI researchers could realize that they will never get AGI until they can create self-causative systems that create world models in relation to purposes pursued, and then somehow iterate those systems though the equivalent of 500 million years of evolution, they might save a lot of time and energy.