AI hallucinations: what they are, why they happen and how to prevent them

Your AI can invent data, cite non-existent laws and fabricate statistics with complete confidence. Hallucinations are the most underestimated reputational risk of generative AI. We explain why they happen and the 7 techniques to reduce them dramatically.

In March 2023, a New York lawyer filed a court submission with six legal cases cited as precedents. The problem: all six cases were invented. ChatGPT had generated them with the names of real judges, real courts and plausible case numbers. The lawyer was sanctioned by the court.

This is an AI hallucination: information generated by a language model that is false but appears authentic. It is not an isolated error or a bug. It is a structural feature of how LLMs (Large Language Models) work. And it is the most underestimated reputational and operational risk for companies adopting generative AI.

The lawyer’s case is not an anecdote. New examples appear every week: airline chatbots inventing compensation policies, health assistants suggesting incorrect medication doses and research tools citing academic papers that do not exist. In every case, the pattern is the same: AI generates false information with the same fluency and confidence as true information.

In this article we explain why AI hallucinates, the types of hallucinations that can affect your business and the 7 proven techniques to reduce them to operationally acceptable levels.

Why AI hallucinates (and why it cannot stop completely)

An LLM does not “know” things. It predicts the next most likely word in a sequence. It has learned statistical patterns from billions of texts, but it has no model of the world, cannot distinguish truth from fiction and cannot verify its own claims.

The main causes of hallucinations:

• Training gaps: If the model has no information about a specific topic, it does not say “I don’t know”. It generates a plausible answer based on similar patterns. If you ask about your returns policy and it does not know it, it will invent one that “sounds” reasonable.
• Overconfidence: LLMs are optimised to generate fluent, confident text. They have no internal mechanism for doubt. When they are unsure, they do not express it: they generate the most likely answer with the same confidence as when they are sure.
• Sycophancy: Models tend to tell users what they want to hear. If you phrase the question in a biased way (“isn’t X better than Y?”), the model will tend to confirm your premise instead of questioning it.
• Context confusion: In long conversations, the model may mix information from different parts of the conversation or confuse context data with general knowledge.

AI does not lie. It does not know that it is saying something false. It simply cannot distinguish truth from fiction.

Types of hallucinations and their risk to your company

• Invented data: Statistics, percentages, sales figures and market data that appear real but are fabricated. Risk: including false data in reports, sales proposals or public communications.
• Non-existent sources: Citations of articles, laws, judgments or studies that do not exist. Risk: citing non-existent legislation in legal or compliance documents.
• False attribution: Attributing statements to real people who never made them. Risk: defamation or credibility problems if published.
• Outdated information presented as current: The model has a knowledge cutoff and may present data from years ago as if it were current. Risk: decisions based on obsolete information.
• Product confabulation: In an ecommerce chatbot, inventing product features, prices or availability. Risk: customer complaints, loss of trust and legal problems.

Risk varies according to the use case. In an internal support chatbot, a hallucination about the holiday procedure is annoying but correctable. In an ecommerce chatbot that invents prices or availability, it is a legal problem. In an AI system that generates financial reports with false data, it can be devastating. The seriousness of a hallucination depends on context, which is why the mitigation strategy must be proportionate to the risk.

The sectors most vulnerable to hallucinations are those handling regulated information: healthcare (false diagnoses), legal (non-existent case law), finance (invented market data) and ecommerce (false product features). In these sectors, an AI system without hallucination controls is not just reckless: it is potentially illegal.

Fact: Studies place the hallucination rate of the most advanced LLMs between 3% and 15% for general queries. In specialised domains without RAG, the rate can exceed 25%. With properly implemented RAG, it falls to 2–5%.

The 7 techniques to reduce hallucinations

1. RAG (Retrieval-Augmented Generation). The most effective technique. Instead of relying on the model’s general knowledge, RAG searches for information in your document base and provides it as context. It reduces hallucinations from 15–25% to 2–5%. (See Article 19 for a detailed explanation of RAG.)
2. Grounding with explicit instructions. Include instructions in the system prompt such as: “respond ONLY with information from the context provided. If the information is not in the context, reply: I do not have enough information to answer that question”. Simple but effective.
3. Low temperature. The temperature parameter controls the model’s creativity. A high temperature (0.8–1.0) generates more varied responses but with a greater risk of hallucination. For business use, a temperature of 0.1–0.3 dramatically reduces invention.
4. Cross-checking. For critical data, have AI verify its own answers: generate the answer, then ask it to identify which claims it can support with the context and which it cannot. Remove or flag anything it cannot support.
5. Source citation. Configure the system to cite the source of each claim. If it cannot cite a source, that is a signal that it is generating unsupported information. This is especially useful when combined with RAG.
6. Human oversight (HITL). For high-impact outputs (customer communications, legal documents and financial reports), require human review before publication. The human does not need to review everything: only outputs leaving the system for external use.
7. Continuous evaluation. Monitor the hallucination rate through periodic evaluations: random samples of responses, manual accuracy checks and a faithfulness metric if you use RAG. If the rate rises, investigate and correct it before it becomes an incident.

Hallucination traffic light: when to act

• Green (GO): Hallucination rate < 3% in periodic evaluations. The system is reliable for general use with standard oversight.
• Amber (FIX): Rate between 3% and 10%. Review the knowledge base (if you use RAG), adjust the system prompt and lower the temperature. Increase human oversight for critical outputs.
• Red (KILL): Rate > 10% or a serious hallucination detected (false data in a public communication). Stop the system for external use, investigate the cause and do not reactivate it until corrected.

Recommended evaluation frequency: weekly during the first month of deployment, fortnightly during months 2–3 and monthly thereafter if the system remains green.

Hallucinations and regulatory compliance

Hallucinations are not only a quality problem. They have direct regulatory implications:

• AI Act: The regulation requires AI systems to be robust and accurate. A system that regularly generates false information does not meet reliability requirements. For high-risk systems, the hallucination rate must be documented in the technical documentation.
• GDPR: If your chatbot provides false information about how you process customers’ personal data, you are breaching the transparency principle.
• Consumer protection: If your ecommerce chatbot says that a product has a feature it does not have, or that it is available when it is not, you may face claims under consumer-protection law.

Legal advice: Include a visible disclaimer in any interface where AI interacts with customers: “This assistant uses artificial intelligence. The information may contain errors. For confirmation, contact our team.” It does not remove responsibility, but it demonstrates diligence.

Hallucinations are not eliminated; they are managed

Pretending that your AI will never hallucinate is naïve. The question is not whether it will hallucinate, but how often, in which contexts and with what consequences. Combining RAG, grounding, low temperature, source citation and human oversight can reduce the rate to operationally acceptable levels.

The important thing is to measure, monitor and act. If you treat hallucinations as a quality KPI (with traffic lights, thresholds and periodic reviews), they go from being an uncontrolled risk to a managed risk.

If you want to assess your AI system’s hallucination rate, implement reliable RAG or configure a continuous evaluation programme, Impulsa3 carries out AI-system reliability audits.

impulsa3.com · Digital Transformation and AI for SMEs and ecommerce · servicios@impulsa3.com

Control of these risks should be integrated with an AI incident plan and AI agents.