Fine-Tuning
Fine-tuning
What is fine-tuning?
- Further training a trained model for a specific task
- Tailored to your tone and field
- Specialize with a small amount of data
With fine-tuning, the instructions you used to attach to every prompt move inside the model. Rules like 'apologize in the first line, three paragraphs in the body, contact person at the end,' which you sent along with ten examples on every call, get built in once during training. What changes is not what the model knows but how it behaves when it answers. Shorter prompts mean fewer tokens per call and lower response latency, and the output variation that came from each person tweaking the prompt slightly goes away too.
There are three main approaches. Full fine-tuning, which updates all of the model's weights, demands heavy hardware and cost. LoRA and adapter methods (PEFT), which add only a subset of parameters, can run on a single GPU, so most practical work in Korea uses them. Using a commercial API vendor's tuning feature only requires uploading a data file, but your training data leaves your environment. The amount of data needed shrinks as the task narrows: document classification or a fixed output format shows results with a few hundred 'input-answer' pairs, while transferring tone and writing style usually takes about 1,000 to 3,000. A good rule of thumb: facts that change often (prices, policies, inventory) belong in RAG, while formats, tone and judgment criteria that rarely change belong in fine-tuning.
The most common point of failure in practice is the expectation that 'if we train on our documents, the model will memorize them.' If you tune on an entire manual PDF, the model doesn’t memorize the content; it imitates the document’s tone, and it makes up numbers and clauses that sound plausible. When a policy changes, RAG only needs one document swapped, but fine-tuning means fixing the data and retraining, and the moment you move to a newer base model, your tuning results are thrown out with it. It is also easy to overlook that awkward phrasing from past staff, or customer names that were never masked, will be reproduced as is if they are in the training data. If you don’t set aside about 200 evaluation questions in advance, you end up concluding with a gut feeling that 'it seems better than before.'
John KimInstructor John Kim, author of B2B Marketing Design and CEO of Nanum Consulting
Ara LimExpert in global leadership, AI, and business communicationThe idea is to further train a pre-trained model on a specific task or dataset. It grew out of transfer learning research and came into wide practical use as large language models spread.
Consider a hypothetical case: the customer center of Company A, a furniture manufacturer and distributor with 240 employees and annual revenue of about KRW 90 billion (≈ US$67,335,000). Eleven agents handle an average of 400 inquiry emails a day. Even after they started using generative AI to draft replies, agents kept rewriting nearly every sentence. Here is how the company evaluated fine-tuning over three months.
- Kim, the CX team lead, pulled 100 draft replies and checked them: 62 needed edits before they could go out. Agents kept raising the same feedback: 'The apology sounds too American, and it comes across as presumptuous to our customers.'
- For three weeks they experimented with attaching 12 model answers to the prompt, but tokens per call quadrupled and responses slowed to an average of 4.2 seconds, so they concluded it was impractical for peak hours.
- From roughly 30,000 support emails over the past two years, we selected 1,800 strong replies that drew no follow-up question. Two operations staff spent two weeks masking customer names, order numbers, and phone numbers, then organized them into question-and-answer pairs.
- Park, who leads AI adoption, ran three rounds of LoRA retraining on a small open-source model, and the total training cost came to under KRW 1 million (≈ US$750). Content that changes every quarter, such as shipping policies and gift rules, was left out of the model and kept in RAG.
- Before launch they scored tone fit and policy accuracy separately on 200 evaluation questions. The first trained version passed on tone, but nine cases invented policy figures, so they moved all of those question types to RAG.
- Two months after rollout, the draft revision rate fell from 62% to 21%, and the average time to write one email dropped from 11 minutes to 4. Shorter prompts also cut monthly API costs by 38%.