Chatbot
Chatbot
What is a chatbot?
- A program that answers conversationally and helps with tasks
- Evolved from rule-based to LLM-based
- Connect your data with RAG
The first thing a chatbot changes is not response speed but the structure of the support desk. Until now, staff handled questions arriving by phone and email in the order received. Now a machine takes the repeat questions first, and people handle only exceptions and cases that need judgment. The side effect is even bigger: what customers ask, and in what words, builds up in logs and becomes data for improvement. On phone support, that information vanished the moment the call ended.
There are three broad types: rule-based bots that follow buttons and scenarios, retrieval bots that find the closest answer in a prepared FAQ, and generative bots where an LLM writes the sentences itself. Add RAG, which grounds answers in internal documents, and it becomes 'tailored to your company.' But if you attach RAG before your documents are organized, you only produce wrong answers faster. If your source documents number only in the hundreds, polishing 100 FAQ entries by hand is faster and more accurate. The boundary is worth knowing too: if it stops at answering in words, it is a chatbot, but if it directly touches systems, such as checking inventory or changing a reservation, it is in agent territory.
Measure results by numbers, not by whether answers 'sound natural.' The metrics are the automated resolution rate (conversations that ended without a human), the handoff rate to agents, time to first response, and the repeat-contact rate (customers who call again after the chatbot answered). An automated resolution rate of 30–40% right after launch is a healthy start, and it takes weekly log review and answer fixes to reach around 60%. If you set a 90% target from day one, the bot ends up tuned to act as if it knows questions it does not.
The most common point of failure in the field is the expectation that installing a chatbot lets you cut support staff. In practice, an operator must read the logs and refine answers for the first 3 to 6 months, and without that person, accuracy stays frozen at its launch-week level. Leaving refund policies, delivery dates or unit prices to a generative model is also risky. Customers take whatever appears on screen as the company's promise, so a single wrong line about a cancellation fee can turn straight into a dispute. If you do not build a path to hand off to a person when the bot does not understand from the start, you only get more customers who repeat the same question three times and leave.
John KimInstructor John Kim, author of B2B Marketing Design and CEO of Nanum Consulting
Seolyeong ParkGenerative AI and AX training specialist with 17 years of hands-on experience and 200+ lectures, helping non-technical staff put AI to work right awayIt traces back to ELIZA, an early chatbot from 1966. After the rule-based era, it leapt to LLM-based conversational AI.
Let’s look at a hypothetical case: Company A, an industrial parts manufacturer with 210 employees and annual revenue of around KRW 48 billion (≈ US$35,912,000). Its 4-person customer support team handles 3,200 inquiries a month, and nearly half are repeat questions like 'When will it ship?' and 'Is this model compatible?' In late February, the team lead raised the idea of a chatbot, saying 'Answering phones is pushing complaint handling back three days at a time.'
- Lee, a team lead in customer support, downloaded three months of support logs (9,600 tickets) and sorted them by type. Delivery status, compatible models and tax invoice reissues alone made up 58% of all inquiries.
- They pulled 1,200 product spec sheets, previously split between the sales and quality assurance teams, into one place, and tagged 340 models discontinued before 2019 as 'discontinued' so they were excluded from answer sources entirely.
- In week two after launch, the chatbot started calculating unit prices on its own, so the team blocked it: it now does not answer on price or delivery commitments and hands off with 'Let me connect you with your sales representative right away.'
- For after-hours inquiries, it only reads out the expected ship date pulled from the ERP, and if it fails to catch the intent twice in a row, it hands off to the human queue immediately. This is the principle Assistant Manager Kim insisted on: 'Don’t trap customers inside the chatbot.'
- Four months later, phone inquiries fell from 1,900 to 1,050 a month, the share resolved without a human handoff rose to 62%, and the average handling time per complaint dropped from 2.7 days to 1.4 days.