Fact-checking and AI hallucinations

We traced the sources of AI anomalies, provided practical tips on how to avoid them, and explained how fact-checking can ensure the reliability of AI results. Read on.

Fact-checking and AI hallucinations - table of contents

In the world of artificial intelligence, the lines between fiction and reality sometimes blur. While innovative AI systems are accelerating progress in almost every field, they also come with challenges, such as hallucinations – a phenomenon where AI generates inaccurate or false information. To fully harness the potential of this technology, we need to understand hallucinations and fact-checking them.

What are AI hallucinations?

AI hallucinations are false or misleading results generated by AI models. This phenomenon has its roots at the heart of machine learning – a process in which algorithms use huge data sets, or training data, to recognize patterns and generate responses according to observed patterns.

Even the most advanced AI models are not error-free. One of the causes of hallucinations is the imperfection of the training data. If the data set is insufficient, incomplete, or biased, the system learns incorrect correlations and patterns, which leads to the production of false content.

For example, imagine an AI model for facial recognition that has been trained primarily on photos of Caucasian people. In such a case, the algorithm may have trouble correctly identifying people of other ethnic groups because it has not been properly “trained” in this regard.

Another cause of hallucinations is overfitting, which occurs when the algorithm adapts too closely to the training data set. As a result, it loses the ability to generalize and correctly recognize new, previously unknown patterns. Such a model performs well on training data but fails in real, dynamic conditions.

Finally, hallucinations can result from faulty assumptions or inadequate model architecture. If the AI designers base their solution on faulty premises or use the wrong algorithmic structure, the system will generate false content in an attempt to “match” these faulty assumptions with real data.

Source: DALL·E 3, prompt: Marta M. Kania (https://www.linkedin.com/in/martamatyldakania/)

Examples of hallucinations

The impact of AI hallucinations goes far beyond the realm of theory. Increasingly, we are encountering real, sometimes surprising, manifestations of them. Here are some examples of this phenomenon:

In May 2023, a lawyer used ChatGPT to prepare a lawsuit that included fictitious citations of court decisions and non-existent legal precedents. This led to serious consequences – the lawyer was fined, as he claimed that he knew nothing about ChatGPT’s ability to generate false information,
it happens that ChatGPT creates false information about real people. In April 2023, the model fabricated a story about the alleged harassment of students by a law professor. In another case, it falsely accused an Australian mayor of taking bribes, when, in fact, he was a whistleblower exposing such practices.

These are not isolated cases – generative AI models often invent historical “facts,” for example, providing false records of crossing the English Channel. What’s more, they can create completely different false information on the same subject each time.

However, AI hallucinations are not just a problem of faulty data. They can also take bizarre, disturbing forms, as in the case of Bing, which declared that it was in love with journalist Kevin Roose. This shows that the effects of these anomalies can go beyond simple factual errors.

Finally, hallucinations can be deliberately induced by special attacks on AI systems, known as adversarial attacks. For example, slightly altering a photo of a cat made the image recognition system interpret it as …. “guacamole.” This type of manipulation can have serious consequences in systems where accurate image recognition is crucial, like in autonomous vehicles.

How to prevent hallucinations?

Despite the scale of the challenge posed by AI hallucinations, there are effective ways to combat the phenomenon. The key is a comprehensive approach that combines:

high-quality training data,
relevant prompts, i.e., commands for AI,
directly providing knowledge and examples for AI to use,
continuous supervision by humans and the AI itself to improve AI systems.

Prompts

One of the key tools in the fight against hallucinations are properly structured prompts, or commands and instructions given to the AI model. Often, minor changes to the prompt format are enough to greatly improve the accuracy and reliability of the generated responses.

An excellent example of this is Anthropic’s Claude 2.1. While using a long context gave 27% accuracy without a relevant command, adding the sentence “Here is the most relevant sentence from the context: ” to the prompt, increased the effectiveness to 98%.

Such a change forced the model to focus on the most relevant parts of the text, rather than generating responses based on isolated sentences that were taken out of context. This highlights the importance of properly formulated commands in improving the accuracy of AI systems.

Creating detailed, specific prompts that leave the AI as little room for interpretation as possible also helps reduce the risk of hallucinations and makes fact-checking easier. The clearer and more specific the prompt, the lower the chance of hallucination.

Examples

Besides efficient prompts, there are many other methods to reduce the risk of AI hallucinations. Here are some of the key strategies:

using high-quality, diverse training data that reliably represents the real world and possible scenarios. The richer and more complete the data, the lower the risk of AI generating false information,
using data templates as a guide for AI responses – defining acceptable formats, scopes, and output structures, which increases the consistency and accuracy of generated content,
limiting sources of data to only reliable, verified materials from trusted entities. This eliminates the risk that the model will “learn” information from uncertain or false sources.

Continuous testing and refinement of AI systems, based on analyzing their actual performance and accuracy, allows for ongoing correction of any shortcomings and enables the model to learn from mistakes.

Context

Properly defining the context in which AI systems operate also plays an important role in preventing hallucinations. The purpose for which the model will be used, as well as the limitations and responsibilities of the model, should be clearly defined.

Such an approach makes it possible to set a clear framework for AI to operate within, reducing the risk of it “coming up with” unwanted information. Additional safeguards can be provided by using filtering tools and setting probability thresholds for acceptable results.

Applying these measures helps establish safe paths for AI to follow, increasing the accuracy and reliability of the content it generates for specific tasks and domains.

Source: Ideogram, prompt: Marta M. Kania (https://www.linkedin.com/in/martamatyldakania/)

Fact-checking. How to verify the results of working with AI?

Regardless of what precautions are taken, a certain amount of hallucination by AI systems is unfortunately unavoidable. Therefore, a key element that guarantees the reliability of the obtained results is fact-checking – the process of verifying facts and data generated by AI.

Reviewing AI results for accuracy and consistency with reality should be considered one of the primary safeguards against the spread of false information. Human verification helps identify and correct any hallucinations and inaccuracies that the algorithms could not detect on their own.

In practice, fact-checking should be a cyclical process, in which AI-generated content is regularly examined for errors or questionable statements. Once these are identified, it is necessary not only to correct the AI-generated statement itself, but also to update, supplement, or edit the AI model’s training data to prevent similar problems from recurring in the future.

Importantly, the verification process should not be limited to simply rejecting or approving questionable passages, but should actively involve human experts with in-depth knowledge in the field. Only they can properly assess the context, relevance, and accuracy of AI-generated statements and decide on possible corrections.

Human fact-checking thus provides a necessary and difficult-to-overestimate “safeguard” for the reliability of AI content. Until machine learning algorithms reach perfection, this tedious but crucial process must remain an integral part of working with AI solutions in any industry.

How to benefit from AI hallucinations?

While AI hallucinations are generally an undesirable phenomenon that should be minimized, they can find surprisingly interesting and valuable applications in some unique areas. Ingeniously exploiting the creative potential of hallucinations offers new and often completely unexpected perspectives.

Art and design are areas where AI hallucinations can open up entirely new creative directions. By taking advantage of the models’ tendency to generate surreal, abstract images, artists and designers can experiment with new forms of expression, blurring the lines between art and reality. They can also create unique, dreamlike worlds – previously inaccessible to human perception.

In the field of data visualization and analysis, in turn, the phenomenon of hallucination offers the opportunity to discover alternative perspectives and unexpected correlations in complex sets of information. For example, AI’s ability to spot unpredictable correlations can help improve the way financial institutions make investment decisions or manage risk.

Finally, the world of computer games and virtual entertainment can also benefit from the creative aberrations of AI. The creators of these solutions can use hallucinations to generate entirely new, captivating virtual worlds. By infusing them with an element of surprise and unpredictability, they can provide players with an incomparable, immersive experience.

Of course, any use of this “creative” side of AI hallucinations must be carefully controlled and subject to strict human supervision. Otherwise, the tendency to create fiction instead of facts can lead to dangerous or socially undesirable situations. The key, therefore, is to skillfully weigh the benefits and risks of the phenomenon, and to use it responsibly only within a safe, structured framework.

Fact-checking and AI hallucinations – summary

The emergence of the phenomenon of hallucinations in AI systems is an inevitable side effect of the revolution we are witnessing in this field. The distortions and false information generated by AI models are the flip side of their immense creativity and ability to assimilate colossal amounts of data.

For now, the only way to verify the validity of AI-generated content is through human verification. While there are several methods for reducing hallucinations, from prompting techniques to complex methods such as Truth Forest, none of them can yet provide satisfactory response accuracy that would eliminate the need for fact-checking.

If you like our content, join our busy bees community on Facebook, Twitter, LinkedIn, Instagram, YouTube, Pinterest, TikTok.

Author: Robert Whitney
JavaScript expert and instructor who coaches IT departments. His main goal is to up-level team productivity by teaching others how to effectively cooperate while coding.
View all posts