Grounding
Grounding
What is grounding?
- Answers based on real evidence
- Reduces hallucination
- Purpose of RAG
Once grounding is in place, the question an organization asks of AI changes. It stops asking “Is this model smart?” and starts asking “Which document and which clause did this answer come from?” That is why the real payoff of grounding shows up first not in accuracy, but in how fast wrong answers get caught. When sources are shown, a manager can verify an answer in tens of seconds instead of minutes, and when it is wrong, what needs fixing is the document, not the model.
Grounding is not a single technology but a bundle of several approaches. They are used together: RAG, which retrieves and attaches documents; attaching files directly when asking a question; and pulling in frequently changing values such as inventory or accounts receivable through internal system APIs. Fine-tuning, often confused with it, is different in nature. Fine-tuning shapes tone and format; grounding creates sources. Areas that change often, such as figures and regulations, should be solved by lookup, not by training.
Success or failure actually depends on the condition of the source material, not the model. If revised and retired versions of the same policy sit in one folder, retrieval pulls in both. Tag each document with its effective date and version, have answers show the document name, clause, and effective date, and set a refusal rule that ends with 'not confirmed' instead of making something up when no source is found. More than half of the implementation effort goes not into model settings but into cleaning up the document repository.
The most common misconception is that 'we added RAG, so hallucination is solved.' If retrieval picks up the wrong document, the AI faithfully follows it and gives a wrong answer, complete with a source. When a source is attached, employees actually trust the answer more. If a travel expense policy abolished three years ago is still in the index, it gets quoted as written. If an HR folder is loaded wholesale without access controls, the salary table can be queried from every employee's chatbot. The moment you connect sources, you also take on the accountability for managing them.
John KimInstructor John Kim, author of "B2B Marketing Design" and CEO of Nanum Consulting
Sungil KimGlobal pitching expert from KAIST and Samsung Electronics who coached the winning teams at the UN Tourism and Select USA IR pitch competitionsThe idea of tying AI output to external facts and evidence. As RAG has spread, it has become a core principle for reducing hallucinations.
Consider a hypothetical case. Company A, an auto-parts manufacturer with 1,200 employees, launched an internal chatbot because it was receiving over 900 administrative and HR inquiries a month. Within three months of launch, complaints piled up: “I trusted the chatbot, and my request was rejected.” The information strategy team collected 32 incorrect-answer reports over three weeks and dug into the causes.
- When Manager Kim of the information strategy team categorized the 32 wrong answers, 24 were based on an old travel expense policy that was retired in 2021. The revised and retired versions had been left side by side on the shared drive.
- Together with Manager Park in HR, they went through 726 policy documents over two days, kept only the 214 current versions, and tagged every file with 'effective date, version, owning department.' The rest were removed from the search scope entirely.
- They also changed the answer rules. If no supporting passage is found, the bot must not guess; it may only reply, 'This is not confirmed in current policies. Please contact HR at ext. 3120.' The refusal wording is fixed.
- Every answer was set to end with the document name, clause, and effective date, and the team lead added one line to the launch notice: “Please report any answer without a source as a wrong answer.”
- Salary tables and performance review documents were split into a separate index and opened only to accounts at team-lead level and above. Access checks were handled not by the chatbot but by the company’s authentication system.
- Eight weeks later, wrong-answer reports fell from 32 to 5 a month, and simple HR inquiries dropped from 900 to 510 a month. The share of answers showing their sources reached 97%.