Turing Test
Turing Test
What is the Turing Test?
- Indistinguishable means 'intelligent'
- The classic benchmark for defining AI
- Its limits are debated
What the Turing Test actually changed was not machine performance but the form of the question. Answering “Can machines think?” would require opening up the inside of a mind, but Turing turned it into a problem of judging by behavior observable from the outside. Thanks to this shift, artificial intelligence moved from philosophical debate to a measurable engineering task, and today’s practice of ranking models with benchmarks and evaluation sets starts from here.
The original design is more specific than most people remember. Turing proposed that a machine pass if more than 30% of judges mistake it for a human in a five-minute conversation, and that threshold became the scoring standard for the Loebner Prize competition, which ran from 1991 to 2019. But this test does not measure the upper limit of capability. Deliberately getting mental arithmetic wrong and mixing in typos actually helps a machine pass. The point John Searle attacked in his 1980 Chinese Room argument was likewise the gap between imitation and understanding.
The reason “It finally passed!” headlines keep recurring lies in the conditions. One program that made news in 2014 set up the persona of a “13-year-old Ukrainian boy” in advance, so awkward grammar and non-sequiturs could be put down to age and nationality. Results can flip entirely depending on the number of judges, the length of the conversation, and topic restrictions, so check the experimental conditions before the pass-or-fail result. This is also why modern evaluation has shifted its focus from imitating humans to task-specific accuracy and reproducibility.
The biggest mismatch in practice comes when the concept is used as a product-marketing line. Make 'human-like' the goal, and the model is pulled toward inventing plausible answers instead of saying it doesn’t know, so you end up with an evaluation scorecard that mistakes hallucination for quality. On top of that, a growing number of regulations, such as the EU AI Act, require disclosing that the other party is an AI, so a design that hides its identity is itself a legal risk. In adoption reviews, change the question from 'Does it feel human?' to how many misinformation cases occur and how long handling takes.
John KimAuthor of “B2B Marketing Design” and CEO of Nanum Consulting
Hyerang ParkContent storytelling and AI-driven brand content specialistAlan Turing proposed it as the 'imitation game' in his 1950 paper “Computing Machinery and Intelligence.”
Consider a hypothetical case. Company A, a home appliance distributor with 740 employees and KRW 280 billion (≈ US$209,486,800) in annual revenue, launched a customer service chatbot with the goal of 'customers can’t tell it’s not a person'. Three months after launch, the handoff rate to human agents was unchanged, but repeat complaints had doubled.
- In an improvement meeting, Team Lead Park of the CX team played 12 call recordings. One customer complained for 20 minutes, saying, “Put me back through to the agent I just spoke with. I spent 30 minutes explaining this.”
- When Quality Manager Kim pulled the logs, the chatbot had replied “I’ll check that right away” in 380 of 4,100 conversations, yet the order lookup never actually ran.
- When an outside training session pointed out the original conditions of the Turing test, Team Lead Park said: 'What we measured wasn’t whether it was accurate, but whether it avoided getting caught. We set the wrong goal.'
- They rebuilt the KPIs, giving less weight to “conversation naturalness satisfaction” and more to first-contact resolution rate and misinformation count. They also rewrote the opening greeting to “I’m an AI agent. For complex requests, I’ll connect you to a human agent.”
- Two months later, misinformation cases fell from 380 to 46, repeat complaints dropped by more than half, and first-contact resolution rose from 51% to 68%. Manager Kim summed it up: 'Once we said it was an AI, customers’ questions got shorter and clearer.'