Parameters
Parameters
What are parameters?
- Internal values a model adjusts through training
- 'X billion parameters' = scale
- Not to be confused with hyperparameters
Knowing the parameter count lets you work out, in numbers, whether your company can run a given model. Each weight typically takes 2 bytes (FP16), so a 7 billion (7B) parameter model needs about 14GB for the weights alone, and a 70 billion (70B) model about 140GB of memory. That decides whether it fits on a single GPU (24–80GB) or needs several, and the difference shows up directly in the server quote and in response speed. Parameters are not marketing copy; they are the starting point for infrastructure design.
In practice, you need to tell three things apart. First, total versus active parameters. In the MoE architectures common today, even if the total runs to hundreds of billions, only a few billion are active when processing a single token, so the model runs far cheaper than its size suggests. Second, quantization. Dropping from 16-bit to 4-bit cuts memory to roughly a quarter, at the cost of some precision. Third, the line between parameters and hyperparameters. Learning rate and batch size are set by people before training and stay fixed afterward, whereas parameters keep changing during training and are frozen at deployment. Retraining the whole model is also rare in fine-tuning; the usual approach, such as LoRA, touches less than 1% of the total.
The expectation that breaks most often is 'more parameters means it will know our business'. Even a 400-billion-parameter model has never been trained on your company’s work rules or price list. If you miss this and just attach a big model, it will confidently invent plausible-sounding policies, and staff will paste them straight into emails. Many people also confuse context window length with parameter count. One more: if you push quantization too hard to save cost, everyday conversation stays fine but amount calculations or code quietly go wrong. That is why you should always mix numeric problems into your validation questions.
John KimInstructor John Kim, author of "B2B Marketing Design" and CEO of Nanum Consulting
Sungil KimGlobal pitching expert from KAIST and Samsung Electronics who coached the winning teams at the UN Tourism and Select USA IR pitch competitionsA term for a neural network’s weights and biases. Scaling up models (adding parameters) has improved performance, which made parameter count a key metric of the large language model era.
Let’s look at a hypothetical case. Company A, an auto-parts manufacturer with 430 employees, decided to launch a chatbot for internal policy and technical-document Q&A within 6 weeks. Drawings and unit prices were mixed into the documents, so external APIs were off the table and on-premises was a given, yet for nearly two months no one could settle the question of “how many billion parameters do we need?”
- Manager Kim of the Information Strategy Team received an executive instruction to "install the biggest one available" and priced out a 70B model. Meeting the 140GB VRAM requirement took 4 GPUs, and the server cost alone came to KRW 62 million (≈ US$46,400).
- An outside consultant pulled 300 real questions from the business teams. 82% were document lookups like "What is the rule on carrying over annual leave?", and only 12 required multi-step reasoning.
- When they added in-house document retrieval (RAG) to an 8B model and ran the same 300 questions, accuracy hit 91%, while the 70B model without documents reached only 63%. Manager Kim said, "So it was a data problem, not a size problem."
- Cutting memory from 16GB to 6GB with 4-bit quantization sped things up, but 3 numeric errors appeared in answers that summed unit prices, so only accounting and quote queries were routed to a model that kept full precision.
- As a result, server investment fell from KRW 62 million (≈ US$46,400) to KRW 14.5 million (≈ US$10,800), average response time dropped from 4.1 to 1.3 seconds, and monthly running costs went from KRW 3.1 million (≈ US$2,320) to KRW 880,000 (≈ US$660). Four months later, active users had grown from 90 to 310.