Token · Context Window
Token / Context Window
What is a token?
- Token = the smallest processing unit
- Context window = the most it can see at once
- The basis for cost and limits
Once you understand tokens and the context window, “how much to include” becomes the first thing to decide, before “what to include.” For the same question, pasting in ten times as much material generally means roughly ten times the response time and cost, and once you cross a certain limit, the beginning is silently cut off. In practice, the habit of estimating volume in advance shapes both answer quality and monthly usage fees.
Here is a rule of thumb. In Korean, one character is roughly 1–2 tokens, so a single A4 page runs 1,000–1,500 tokens and a 100-page contract easily exceeds 100,000. Pricing charges input tokens and output tokens separately, and output is usually the more expensive of the two. The context window is the combined total of input and output, so the more material you feed in, the less room is left for the answer. There is also a cost that never shows up on a price sheet: in a long conversation, the entire earlier exchange is re-sent on every turn.
There are two main ways to manage this. One splits the document into clauses or paragraphs and feeds in only the pieces you need (chunking, RAG). The other periodically summarizes the earlier conversation and feeds the summary back in. The first suits document review; the second suits long consultations and meeting-minute work. Both share one goal: not to fill the window, but to strip out irrelevant text. File-upload features simply automate this process; they don’t remove the limit.
The most common misconception is “the window is big enough, so I can just paste everything in.” You can, but models often lose the middle of a long input, and readers who don’t notice the omission copy a plausible-looking summary into their report. Another point: material placed in the context is not training. It disappears when the window closes, which is why people ask, “I told it last week, so why doesn’t it know?” Design your workflow on the premise that a short request with only the essentials is often more accurate than one that dumps in everything.
John KimInstructor John Kim, author of B2B Marketing Design and CEO of Nanum Consulting
Ara LimExpert in global leadership, AI, and business communicationTokens and context windows come from how transformer-based language models work. They refer to the units of input a model processes at one time and the maximum length it can handle.
Consider a hypothetical case. Company A, an auto-parts manufacturer with 180 employees, has two legal staff who review 260 supplier contracts a year. In March it piloted AI-assisted review. When an 80-page supply agreement was pasted in whole, the AI summarized only the damages clause near the end and dropped the unit-price adjustment clause at the front entirely, and this happened eleven times in three months.
- When Manager Kim in the legal team measured one contract in tokens, it came to about 60,000. Feeding in all 22 contracts a month whole would cost three times the pilot budget.
- When the purchasing manager asked, 'Do we really need to read all 80 pages?', the team narrowed the review to the 7 clauses that had actually caused disputes, including unit-price adjustment, late-delivery penalties, defect warranty, and termination.
- After Assistant Manager Lee in the IT team changed the template to split each contract into 120 clause-level chunks and paste in only the top 5 relevant to the question, each request dropped from 60,000 tokens to 4,000.
- Once Manager Kim saw that earlier turns are re-sent with every message as a conversation grows, she circulated a team rule — 'start a new window after each contract' — and the habit of running 20-turn conversations disappeared.
- Three months later, the average review time per contract fell from 95 to 32 minutes, and monthly AI usage fees dropped from KRW 410,000 (≈ US$305) to under KRW 100,000 (≈ US$75). Missed clauses near the start of the document fell from 11 to 0.