Small Language Model · sLM
Small Language Model
What is a small language model?
- Small, lightweight language models
- Low cost, fast, on-device
- Fine-tuned for a specific use
What an sLM really changes is not the model benchmark table but the approval chain for adopting AI. Giant models charge per call and send data outside the company, so legal and security reviews come first. A model of a few GB, by contrast, can run on an internal server or a work laptop, so a team can pilot it first without a large contract. The biggest difference people notice in practice is that 'try it, then decide' becomes possible.
A sense of scale makes the decision easier. Models in the 1B–10B parameter range are generally called sLMs. At FP16, a 7B model takes about 14GB of memory, but with INT4 quantization it drops to around 4GB and runs on a laptop or a recent smartphone. What matters is not size but division of labor. Finding and reading internal documents and organizing them into a set format is well within reach of an sLM plus RAG, while work that needs the model’s 'brain', such as multi-step root-cause analysis or long-form planning, still belongs to giant models.
Tasks where an sLM works well share a few traits: the input format is consistent, the range of correct answers is narrow, and the task repeats hundreds of times a day. Customer inquiry classification, call-recording summaries, code autocompletion, and equipment error-code guidance all fit. For work where the questions differ every time, a small model quickly reaches its limits. The test for adoption is simple: do you get more than 1,000 questions of the same type per month, and can you verify the answers against your own documents?
The most common misconception is 'it’s small, so it’s cheap.' The model itself may be cheap, but the people who build and maintain the fine-tuning data cost just as much. Creating thousands of QA pairs ties up business staff for several weeks, and whenever products or internal policies change, you have to rerun training. Even riskier is swapping models without an evaluation set. An sLM is poor at admitting what it doesn’t know, so it delivers wrong answers in plausible-sounding sentences. Unless you first build a scorecard of 200–300 fixed questions, you will find out about a drop in quality from customer complaints.
John KimInstructor John Kim, author of "B2B Marketing Design" and CEO of Nanum Consulting
Sungil KimGlobal pitching expert from KAIST and Samsung Electronics who coached the winning teams at the UN Tourism and Select USA IR pitch competitionsIt emerged as an alternative to the cost, speed, and privacy limits of giant models, and it is gaining attention as compression techniques (distillation, quantization) improve.
Consider a hypothetical case. Company A makes auto parts, with 380 employees and annual revenue of KRW 95 billion (≈ US$71,075,900). Its factory network is cut off from the outside internet, so when a floor worker hit an equipment error, the only options were to dig through a paper manual or call the production engineering team. Every time a machine stopped, an average of 14 minutes was lost.
- Manager Kim of the production engineering team collected 1,840 questions from the shop floor over three weeks. Of these, 78% were repetitive queries, such as interpreting error codes and part replacement intervals.
- The head of information security put their foot down: "I won’t sign off on process drawings leaving the company." So the team changed course and put a 7B model, quantized to INT4, on a single factory server.
- They organized 2,300 pages of equipment manuals and six years of maintenance records into 4,000 QA pairs and fine-tuned for two weeks, but first-pass accuracy came in at only 61%.
- Twelve shop-floor foremen used it for two weeks and flagged 340 wrong answers. Feeding these back into retraining raised accuracy from 61% to 88%.
- Reasoning questions such as "analyze the root cause of this defect" kept producing nonsense. So the team split the system into two tiers: only this type, 40–50 requests a month, goes to an external LLM that has passed a security review.
- Six months later, wait times for equipment inquiries had fallen from 14 minutes to 3. Processing everything through a cloud API would have cost an estimated KRW 19 million (≈ US$14,200) per month; the system instead runs at KRW 2.1 million (≈ US$1,570) per month, including server depreciation.
- When fine-tuning for a specific use
- When cost, speed, and privacy matter
- When it must run on-device
- Diverse, complex general-purpose tasks
- When top performance is the priority
- When the use case changes often