On Becoming Human: A Discussion of "Klara and the Sun"

In 1950, Alan Turing suggested that machines would exhibit behavior indistinguishable from that of humans by the end of the twentieth century. When that possibility first loomed, programmers began to wonder if a computer might replace a living therapist. Eliza was a computer program designed in the mid-1960s whose script DOCTOR engaged in reflective listening, a technique still widely used by therapists as well as crisis line counselors. Woebot, a program launched in 2017, engages in a form of CBT, asking people to externalize thoughts and feelings and then to revise or talk back to negative statements in order to ameliorate anxiety and depression. Some people using these programs claim to have felt a human presence, or at least to have been helped, by these AI therapists. But then again, humans have a tendency to anthropomorphize.

Weekly therapy session

Source: Wendy Jones/personal photo

Asking whether an AI can replace a therapist is tantamount to asking what it means to be human because only humans can offer what Irving Yalom calls “the gift of psychotherapy,” a relationship that facilitates healing; as Judith Herman has noted, healing takes place in the context of relationships. A genuine relationship comprises the ability to respond flexibly and spontaneously, and just as important, to resonate emotionally with a wide range of feelings. Social signals are often as important as words in human (indeed mammalian) interactions, and no machine can yet replicate the innovation or moment-to-moment nuanced responsiveness of a real person. These psychological features largely define what it means to be human.

An AI with "human" qualities

The definition of “human” is one of the issues that has preoccupied the novelist Kazuo Ishiguro, both in his earlier novel Never Let Me Go, about clones, and in his latest novel Klara and the Sun, about artificial intelligence. Klara is an AF, an artificial friend, who runs on solar power. She displays exceptional qualities from the start. The manager in the shop where she is displayed observes, “Oh Klara ... You never miss a thing.” But this is the least of Klara’s extraordinary powers.

As Klara narrates her story—and first-person narration is crucial to establishing Klara’s capabilities—we see that she genuinely loves Josie, the 14-year-old girl who owns her, in a way that no machine can love. She supports and helps Josie through difficult times, and she achingly yearns for Josie’s recovery from a potentially fatal illness. Klara feels negative emotions such as fear and anxiety as well as positive emotions. She is intuitive about humans and their motives, and on a visceral level, she understands that Josie’s mother’s love is self-centered and narcissistic.

Artificial Neural Network with Chip

Source: Wikimedia Commons/mikemacmarketing / original posted on flickr Liam Huang

Klara's other psychological capabilities go far beyond what her program dictates. She counsels Josie and her friend Rick in ways that help them maintain their friendship. She invents a religion, worshipping the sun and bargaining with her god for a cure for Josie. She hallucinates briefly, having visions with meaningful content, as many people who experience psychotic episodes tend to do. She acts independently, taking the initiative to pursue original goals. Her thinking is emergent, greater than the sum of her parts. Brains consist of neural networks; computers also consist of networks, although these are electronic (at least to date). Theoretically, if electronic networks with the density and complexity of brains could be manufactured, computers could think and feel as humans do.

The question Ishiguro asks, a question with ethical and spiritual as well as psychological dimensions, is: If a being is made rather than born, does that disqualify them from being considered human? In the imaginary and largely dystopian societies depicted in Klara and the Sun and Never Let Me Go—societies alarmingly like our own—the answer appears to be yes. Yet the manufactured beings in both these novels narrate stories that demonstrate their humanity both through their expression of thoughts and feelings and in the events they recount. Moreover, in Klara and the Sun, Ishiguro challenges the distinction between the made and the born by the presence of genetic engineering as an option for families to ensure the future financial and professional success of their children; this storyline comments ironically on Klara’s status. If human intelligence is routinely “manufactured,” how can we draw a line between the artificial and the real?

Literature abounds with examples of AI gone haywire and dangerous: I Have No Mouth and I Must Scream (Harlan Ellison), 2001: A Space Odyssey (Arthur C. Clarke), The Wastelands (Stephen King), Sea of Rust: A Novel (C. Robert Cargill), and many others. Klara and the Sun is a brilliant addition to a counter-tradition, far less prevalent, that depicts AI as benevolent, as in Galatea 2.2 (Richard Powers). Indeed, in the heartless world that Klara inhabits, with its economically based designation of winners and losers, humanity belongs to outsiders, those who refuse to participate or who lack resources. I use humanity here to mean the quality of being benevolent, but it also means the human race, collectively. Ishiguro plays on these multiple meanings in a novel whose protagonist has more humanity than most of the people around her, who is indeed more human(e) than the humans who created her.