Open-Source LLMs · Open Weights
Open-Source Large Language Model / Open Weights
What is an open-source LLM?
- An LLM whose weights are published so it can run on your own infrastructure
- Data sovereignty, cost control, and customization drive adoption
- The standard approach is a hybrid that runs only sensitive tasks privately
Holding the weights file means shifting AI from a service subscription to an owned asset. With a closed API, when the provider upgrades a model or retires an old version, prompts that worked yesterday produce different output. A downloaded model lets you pin the exact version, so validation results stay valid. In finance and healthcare, which require revalidation documents every time a model changes, reproducible output decides whether adoption is even possible. And because open models can run the same functions in an air-gapped environment with no external internet, they offer something closed models cannot.
The single word 'open' hides at least three different levels. Open weights (only the weights are released), fully open (training code and data composition are published too), and restricted open (usable only under certain conditions) are all called 'open models.' Apache 2.0 and MIT-family licenses have simple commercial terms, but models under community licenses carry clauses such as monthly active user caps, naming rules for derivative models, and bans on using outputs to train other companies’ models. If you promote a PoC to production without legal review and find a violation later, you have to replace the model entirely and redo fine-tuning from scratch.
Choosing a size is really choosing a GPU budget. An 8B model fits on a single 24GB VRAM card with 4-bit quantization, while a 70B model needs around 140GB at fp16, which means pooling several high-end GPUs. So real-world setups tend toward not 'one big model' but splitting narrow tasks such as summarization, classification, and extraction across smaller models. The break-even point depends not on the per-token price but on GPU utilization, so for workloads of a few dozen requests a day, self-hosting is actually more expensive.
The most common misconception is 'it’s open source, so it’s free.' GPU rental, power, monitoring, and the labor of overnight incident response are paid every month, so there is a range where low utilization makes it more expensive than an API. The second is 'we downloaded it, so it’s safe.' If fine-tuning datasets and inference logs pile up in plain text, you recreate the same leak path inside the company. The third is trusting leaderboard scores at face value: a model near the top of English benchmarks often drops key sentences and mistranslates internal terms on your own Korean documents. Until you have tested it on 100 samples of your own data, do not treat published numbers as a conclusion.
John KimInstructor John Kim, author of "B2B Marketing Design" and CEO of Nanum Consulting
Sungil KimGlobal pitching expert from KAIST and Samsung Electronics who coached the winning teams at the UN Tourism and Select USA IR pitch competitionsIn 2023, Meta released LLaMA for research and soon followed with the commercially usable Llama 2, and the open-weights ecosystem grew explosively. Since then, Mistral, Qwen, DeepSeek, and others have narrowed the performance gap and kept widening the range of 'tasks where an open model is enough.'
Consider a hypothetical case. Company A, a Korean property and casualty insurer with 900 employees, had run a pipeline for six months on a closed API that automatically summarizes call-center recordings and classifies complaint types. Monthly token costs climbed to KRW 23 (≈ US$1) million, and an internal audit raised two findings that the setup sends customer names and contract numbers outside the company, which kicked off a four-month migration project.
- When Manager Park of the Digital Innovation Team put three months of token statements in front of the CFO, the reply was one line: 'For this money, wouldn’t it be better to rent GPUs?' The review of a switch was assigned as a task on the spot.
- Two AI engineers tested four open 8B and 32B models against 200 real call-center recordings, and the 32B model reached 94% of the existing API’s summarization accuracy.
- Legal review found that the top candidate’s license had a '700 million monthly active users' clause and naming rules for derivative models, so the team moved to an Apache 2.0-family model with simpler terms.
- Kim, a manager on the infrastructure team, rented two L40S GPUs in the company’s own VPC instead of buying servers and deployed with vLLM. In the first two weeks, throughput per second was half the target, so the team reworked the batch size and quantization settings three times.
- Manager Park ran two paths: only call-recording summaries containing customer information went to the private model, while press-release drafts and meeting-minute cleanup stayed on the existing closed API.
- Four months after the switch, monthly operating costs fell from KRW 23 million (≈ US$17,200) to KRW 9.8 million (≈ US$7,330), including GPU rental and labor, and external transmission of customer data dropped to zero, resolving both audit findings.