Speech Recognition · Speech Synthesis
Speech Recognition / Synthesis (STT/TTS)
What is speech recognition?
- Speech recognition (STT) + speech synthesis (TTS)
- Speech to text, text to speech
- The ears and voice of AI assistants, captions, and call centers
Voice data has been piling up for years, but no one could search it. Even with tens of thousands of call recordings on a server, a QA manager could listen to only two or three calls per agent each month, and once a meeting was recorded, no one ever opened the file again. With STT in place, all of that audio becomes searchable text. You can pull up only the calls where the word 'refund' came up, or jump straight to the part of a meeting where a particular product name was mentioned. The real payoff of speech AI isn't the transcription itself, but that voice becomes part of your database.
STT comes in two modes: batch, which processes a whole recording file at once, and streaming, which transcribes while the person is still speaking. Batch is enough for meeting minutes and call analysis, but for real-time interpretation or agent screen assist, a response delay over 0.3 seconds makes it hard to use on the floor. Accuracy is usually measured by word error rate (WER). Even though general conversation has come down to around 5%, industry jargon, product names, and personal names are only caught if you register them separately in a custom vocabulary. Speaker diarization, which separates who spoke, is a separate feature from STT and must be turned on separately, and extracting summaries or sentiment classification from the text is the job of the LLM layered on top. Understanding that STT is only the ears, not the brain, is what keeps quotes and expectations aligned.
Where projects most often go wrong is the number '95% accuracy.' The remaining 5% of errors isn't spread evenly across particles and conjunctions; it clusters on amounts, quantities, and proper nouns. When '150,000 won' becomes '500,000 won', or one character in a drug name turns it into a different drug, the meeting minutes and the call history both become liabilities. A quiet meeting room during a demo and a real call full of noise, dialects, and overlapping voices are completely different conditions. The moment recordings become text, the scope of personal data processing widens, so advance notice to both customers and employees is required. And cloning a voice with TTS without written consent from the voice actor or employee immediately raises likeness-rights and impersonation issues.
John KimInstructor John Kim, author of "B2B Marketing Design" and CEO of Nanum Consulting
Sungil KimGlobal pitching expert from KAIST and Samsung Electronics who coached the winning teams at the UN Tourism and Select USA IR pitch competitionsThis field has been studied for decades. Applying deep learning in the 2010s dramatically improved accuracy, and recent large models have greatly advanced naturalness and multilingual handling.
Let's walk through a hypothetical case. Company A distributes medical devices and health supplements, with 240 employees and 35 inbound call agents. It handles 12,000 calls a month, but QA reviewed only 3 calls per agent, 105 in total. Each time a return dispute came up, staff spent half a day tracking down and listening to recordings, until the company launched a three-month speech AI rollout.
- When the CS team lead ran 300 recordings through a pilot STT, the company's own product name 'Leprinol' came out as 'Eprinol.' "If the numbers and product names are wrong, we can't use it for QA," someone said in the first meeting.
- The quality control team added 420 product names, insurance terms, and competitor names to a custom vocabulary list. Two weeks later, the product-name error rate had dropped from 18% to 4%.
- The operations team lead told the 35 agents in advance, "Calls are converted to text automatically, but this is for improving service, not for individual evaluation," and added a recording-and-analysis notice at the start of the IVR.
- They turned on speaker diarization to separate agent and customer speech, then automatically tallied the silent gaps between the moment a customer asked, "So can I get a refund or not?" and the answer. This surfaced 210 calls with delayed responses.
- TTS was used only for delivery tracking and business-hours announcements; complaint and refund calls stayed with people. The head of CS drew the line, saying, "Putting a machine voice on an angry customer only makes them angrier."
- Three months later, QA listening went from 105 calls a month to all 12,000 calls, and post-call wrap-up time dropped from an average of 4 min 20 sec to 1 min 50 sec per call.