Can AI be trusted?

Recently, there have been numerous articles, interviews and other types of media reports about Google engineer Blake Lemoine, because Lemoine claimed that the artificial intelligence (AI) large language model LaMDA has "perception ability" in the process of communicating with users, even "like a seven or eight-year-old child."

Figure｜Google engineer Blake Lemoine (Source: The Washington Post)

After reading a dozen different perspectives on this topic, I have to say that the media hype around current AI technology has become (a little) disappointing. There is a lot of discussion about why deep neural networks are not "sentient" or "conscious". This is an improvement compared to a few years ago, when the news media was still creating sensational stories about AI systems inventing their own language, taking over all human jobs, and accelerating towards artificial general intelligence (AGI).

But the fact that people are discussing “perception” and “consciousness” reiterates an important point: Even though today’s big language models are becoming increasingly convincing, they are still being pointed out by scientists on various occasions as having fundamental flaws of one kind or another. The topic of “AI fooling humans” has been discussed since the ELIZA chatbot in the 1960s, but today’s big language models are really on another level. If you don’t know how language models work, Lemoine’s conversations with LaMDA can seem bizarre — even if they are carefully selected and edited.

However, “perception” and “awareness” are not the best topics to discuss about big language models and current AI technologies; a more important discussion should be about human compatibility and trust, especially as these technologies are increasingly integrated into everyday human applications.

Big language models don’t know “human language”

There has been a lot of discussion over the past week about how neural networks and large language models work. In this post I will give a zoomed-out view of the current situation, starting with human language.

For humans, language is a means of communicating the complex, multidimensional activity that goes on in our brains. For example, when two brothers are talking and one of them says "Mom," the word is associated with many activities in different parts of the brain, including memories of Mom's voice, face, feelings, and different experiences from the distant past to the most recent. But in fact, the way they are represented in their brains may be very different, depending on their respective experiences. However, the word "Mom" provides a concise, representative approximation that helps them agree on the same concept.

When you use the word "mom" in a conversation with a stranger, the difference between experience and memory becomes even greater. But you still reach a consensus based on a shared concept in your mind.

Think of language as an algorithm that helps communicate the vast amounts of information in our brain to another person. The evolution of language is tied to our experiences in the world, from physical interactions in our environment to social interactions with other people.

Language is built on our shared experiences in the world. Children know about gravity, dimensions, the physical consistency of objects, and concepts in humans and society such as pain, grief, fear, family, and friendship before they even say their first word. Without these experiences, language would have no meaning. That is why language often ignores common sense and information shared by the interlocutors. On the other hand, the degree of shared experiences and memories will determine the depth of your communication with another person.

(Source: Pixabay)

In contrast, large language models have no physical and social experience. They are simply trained with billions of words and learn to respond to prompts by predicting the next sequence of words. This approach has achieved great results in the past few years, especially after the introduction of the transformer architecture.

So how do transformers make convincing predictions? They first convert text into “tokens” and “embeddings,” mathematical representations of words in a multidimensional space. The embeddings are then processed to add other dimensions, such as the relationships between words in a sequence of text and their role in sentences and paragraphs. With enough examples, these embeddings can create a good approximation of how words should appear in a sequence. The transformer architecture is particularly popular because it is scalable: its accuracy improves as it becomes larger and takes in more data, and they can mostly be trained using unsupervised learning.

But the fundamental difference remains. Neural networks process language by converting it into embeddings. Whereas for humans, language is an embedding of thoughts, feelings, memories, physical experiences, and many other things about the brain that we have yet to discover.

Therefore, despite the great progress made by transformers, large language models, deep neural networks, etc., they are still far from human language.

Can AI be trusted?

There is a lot of discussion in the industry right now about whether we should attribute attributes like sentience, consciousness, and personality to AI systems. The problem with these discussions is that they focus on fuzzy concepts that mean different things to different people.

For example, a functionalist might argue that neural networks and large language models are conscious because they exhibit more or less the same kinds of behavior we expect to see from humans, even though they are built on different foundations. But others might disagree, arguing that an organic entity is necessary for consciousness and that neural networks can never be conscious.

However, a more practical question is how compatible are current neural networks with human thinking, and whether they can be trusted by humans in key application scenarios? This is an important issue because large language models are mostly used commercially by companies.

For example, given enough training, a chimpanzee might be able to learn to drive a car. But would you let it drive if a pedestrian was about to cross the road? You wouldn't, because you know that no matter how smart a chimpanzee is, they don't think the same way as humans do and are not capable of tasks involving human safety.

Likewise, a parrot is capable of learning a few phrases, but would you let it be your customer service agent? Probably not.

(Source: Pixabay)

Even when it comes to humans, cognitive impairments can disqualify some populations from performing jobs and tasks that require interpersonal skills or involve human safety. In many cases, these people are able to read, write, and speak fluently and maintain consistency and logic in long conversations. We do not question their sentience, consciousness, or personhood. But we know that their decisions on certain matters may be inconsistent and unpredictable because of their disorder.

What matters is whether you can trust them to think and make decisions like normal people. In many cases, we trust people in important positions because their sensory systems, common sense knowledge, feelings, goals, and rewards are largely consistent with ours, even if they don’t speak our language.

So, back to recent events, what do we know about LaMDA? First, it perceives the world differently than we do. Its "knowledge" of language is not based on the same experiences as ours. Its common-sense knowledge is on a shaky foundation, because there's no guarantee that a large body of text will cover everything we've overlooked in language.

Given this incompatibility, how much can you trust LaMDA and other big language models, no matter how good they are at generating text output? A friendly, entertaining chatbot program that doesn't steer the conversation into some sensitive topics is probably not a bad idea. Search engines are also a good application area for big language models (Google has been using BERT in search in recent years). But can you give them sensitive tasks like open-ended customer service chatbots or bank advisors, even if they have been trained or fine-tuned on a large number of relevant conversation records?

In my opinion, we need application-specific benchmarks to test the consistency of large language models and their compatibility with human common sense in different domains. When it comes to real applications, there should always be well-defined boundaries to determine where to terminate the conversation of a large language model and hand it over to a human operator.

Be careful with AI

In effect, human intelligence is used to find the right problems to ask, and AI is used to solve them in the most efficient way.

We have seen time and again that computers can find shortcuts to solving complex problems without requiring human cognitive abilities, and have been successful at checkers, chess, Go, programming competitions, protein folding, and other well-defined problems.

Natural language is different in some ways from all the other problems AI has solved, but it’s also similar.

On the one hand, transformers and large language models have demonstrated that they do not need to explore the world and understand its basic rules before acquiring the language to interact with others on a common knowledge basis, as humans do. They can directly produce impressive results. On the other hand, they lack the experience that humans gain in the process of learning language.

Although large language models work well for well-defined language-related problems, their compatibility with human language processing is very limited, so we should be careful about trusting them.

References:

https://bdtechtalks.com/2022/06/20/lamda-large-language-models-sentient-ai/

https://www.reddit.com/r/artificial/comments/vgl1so/sentience_is_the_wrong_discussion_to_have_on_ai/

https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/