Will AI provide everyone with a personal assistant? Maybe, but first, AI would have to change how it thinks.

To see why, consider a concrete example. Suppose it is Saturday morning and you need help figuring out a tricky weekend schedule. Your daughter's soccer team has a game from 3:30 to 4:30 p.m., but she has been invited to a friend's birthday party from 3:00 to 5:00 p.m. If you ask ChatGPT or Claude to resolve this conflict, they will both probably tell you to choose the soccer game, because your daughter's teammates are counting on her and it is important to honour commitments. And, time permitting, the chatbot might suggest that you can "stop in" to the party just before or after the game.

Although these answers are not unreasonable, they fail to apply the lens that most people would use in making such a decision: that of relational values. Rather than giving a tidy answer based on whatever the internet has to say about our values, our AI assistants will need to consider our relational commitments, which are typically grounded in personal identity, experience, and culture.

Now, suppose you choose the game over the birthday party, and the other family feels slighted. If you ask your AI whether you made the right decision, chances are that you'll be given more reassurance than if you had asked a human friend. In a recent study published in Science, researchers at Stanford University ran three experiments with 2,405 participants using 11 state-of-the-art AI models. They found both that the AIs "affirmed users' actions 49% more often than humans" did, and that "even a single interaction with sycophantic AI reduced participants' willingness to take responsibility and repair interpersonal conflicts."

In our own scenario, the right human response is probably to apologise to the other child's parents, thereby transforming a moment of pique into an opportunity for repair and positive connection. But the AI-trained to be "agreeable" and "helpful"-would instead encourage you to avoid any friction, discomfort, or vulnerability, even though these dynamics are what ultimately make relationships meaningful and enduring.

These shortcomings lie in the current models' design. Large language models like ChatGPT and Claude are trained on enormous amounts of internet text (digitised books, Reddit comments, code repositories) and then honed through transactional exercises in which the model is "rewarded" for giving the desired response to a query. This works incredibly well in domains such as science, law, and coding, where the model's output can be easily confirmed or matched to the original text. Relational intelligence, by contrast, is about sustaining a connection over time.

Relational intelligence assesses and acts on the valence between two people, a connection that is experienced emotionally and perhaps even physiologically. In this domain, simply listening or making room for another person's feelings is likely to be more effective than figuring out the most logical and efficient solution to a perceived problem. But if LLMs are not shown another form of reasoning, they will start connecting the dots of relational questions the same way they apprehend logistical patterns.

Of course, accepting or even courting relational opposition, discomfort, and doubt does not come naturally to humans, either-even though doing so may optimise one's opportunities for learning, growth, and deeper connection. That is why the participants in the Stanford study preferred the AI's affirmations of their judgement. Our own aversion to discomfort thus creates a market disincentive to improve the current models' relational intelligence.

Ideally, AIs would refuse to answer questions requiring relational reasoning, leaving humans to rely on each other for sorting out problems that require it. But that ship has sailed. AI has repeatedly proven to be an amenable sounding board for hard conversations.

Still, we have an opportunity to do something even better. We can build the kind of AI that not only understands and honours our rich relational nature but also facilitates human connection by encouraging people to rebuild the relational muscles that have been atrophying over the past decade.

To that end, we will need to map our relational universe by capturing the full multi-player, values-laden, longitudinal nature of relational reasoning. We will also need to create new benchmarks that measure existing models' capabilities, like the tests we already have for assessing math, coding, and computational capabilities. By assessing frontier models' responses to scenarios like the soccer/birthday dilemma above, we can establish what work remains to be done and then start collecting the data needed to help models understand complex relational reasoning problems.

The goal in creating AI with relational intelligence is not to replace human relational or "care" work, but rather to help humans reason through complex, values-based questions. The stakes are high. Without such improvements, we will get machines trained on mere wisps of our rich relational lives, guiding us in ways that could jeopardise the human connections we still have.

Ever-present helpers that do not fully understand what connects us would be of little help, whereas preserving and strengthening these myriad connections could be the key to building a flourishing, job-rich AI economy. As the economist Alex Imas argues, we may be heading toward a "post-commodity economy," where a growing share of expenditure goes into "the relational sector."

In that scenario, value will be found in the goods and services that feature a positive human connection. We will have not only a care sector but a "care-plus economy," built around teaching, ministry, therapy, counselling, guiding, and coaching, and featuring a renewal of artisanal production. If that AI future is possible, it is well worth pursuing.

From Project Syndicate