Chain of Thought · CoT
Chain of Thought
What is chain of thought?
- Get the AI to think 'step by step through the solution'
- Improves complex reasoning ↑
- Triggered by the prompt
CoT changes more than the accuracy rate. When an answer that used to arrive as a single block is split into several reviewable lines, the reviewer can say not 'this answer is wrong' but 'the exchange rate in step three is wrong.' Instead of discarding the whole output and asking again, you fix only the step that went astray, and once review standards apply to the process rather than the result, practical quality control becomes possible.
There are two main ways to use it. The zero-shot method adds a single line like 'Solve this step by step' at the end of the instruction. The few-shot method shows two or three worked examples up front. If your company has a fixed calculation order or a set sequence for applying rules, the latter is far more stable. You can also add self-consistency: run the same problem three or four times and take the majority answer. However, the latest models already specialized for reasoning work through steps internally, so adding 'think slowly' gains little and can multiply the response tokens several times over, raising cost and latency.
Nor does it help every task equally. The difference is clear when you must combine overlapping conditions or apply rules in sequence, but for tasks with no intermediate steps, such as translation or polishing a sentence, it only adds length. Whether the problem can be broken into steps is the test for when to use it. Also, asking for steps is not the same as fixing the output format as a table. A table only asks for a tidy summary, while specifying the order of thinking is what CoT does.
The most common misconception is believing that the working shown on screen is the AI’s real reasoning. A model may write a plausible process first and then justify a wrong answer to fit it. The smoother the process looks, the more the reviewer lets their guard down, which is the greater danger: a quote with a single wrong number passes approval because it looks logical. That is why a step to check intermediate figures against the price list and ledger must always follow, and why it is safer not to attach the working as-is to documents going to customers.
John KimInstructor John Kim, author of B2B Marketing Design and CEO of Nanum Consulting
Ara LimExpert in global leadership, AI, and business communicationIntroduced in 2022 by Google researchers (Jason Wei et al.) in the paper “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” It showed that having a model generate intermediate reasoning steps greatly improves the reasoning performance of large models.
Consider a hypothetical case. Company A is an industrial parts distributor with 180 employees and annual revenue in the KRW 90 billion (≈ US$67,335,000) range. Its sales support team stacks volume-tier discounts, exchange rates, and ocean freight into every quote. When they entered the conditions into the in-house AI chatbot, answers came back in one second, but a sample audit of 40 quotes last month found the final amount wrong in 9. As re-sends piled up, the team lead set up a three-week improvement project.
- Ms. Kim, a manager on the sales support team, added one line to the end of her existing prompt, 'Apply the discount, then convert the currency, then add freight; write the amount at each step and give the total last,' and ran it again.
- When the AI listed the unit price, tiered discount rate, and applied exchange rate line by line, the reviewer stopped skimming only the total and started checking from the third line, where the exchange-rate reference date appeared.
- The cause surfaced within three days. Seven of the 9 wrong quotes had the same error: the discount was subtracted twice. The team lead hard-coded the line 'Apply the discount only once' into the prompt.
- For simple repeat orders where the price list is unchanged, the team decided to drop CoT entirely. In the first two weeks, response tokens per request rose 3–4x, and the monthly API bill jumped by more than 40%.
- Three weeks later, the same process was used to review another 40 quotes. Errors fell from 9 to 2, quote re-sends dropped from 11 to 3 per week, and review time per quote fell from an average of 6 minutes to about 2.