Loss function
Loss Function
What is a loss function?
- Turns the gap between prediction and answer into a number
- Training is the process of reducing this value
- Acts as the “grading rubric”
The loss function you choose turns the same data and the same model architecture into an entirely different product. The model has no idea what the business wants; it moves only in the direction that makes the loss smaller. So designing a loss function is less a coding task than translating a business goal into numbers. Which errors you count heavily and which you let slide determines exactly which mistakes the model will tolerate and which it will avoid.
In regression, the choice splits between MSE, which squares the error, and MAE, which uses the absolute value. A single outlier with an error of 10 balloons to 100 under MSE, so for demand data with frequent spikes, MAE or Huber is safer. In classification, cross-entropy is the default, and when classes are heavily skewed, you add class weights or Focal Loss. Here you need to distinguish loss from evaluation metrics. Loss is the differentiable value the model sees during training, while accuracy, F1, and recall are the values people look at to judge performance. If the two point in different directions, the training succeeds but the project fails.
The most common failure in practice is a loss curve that slopes down nicely while business metrics don’t budge. On a line with a 1% defect rate, simply labeling everything normal still produces a low enough cross-entropy. A drop in loss does not mean inspection performance has improved. Another mistake is leaving the asymmetry of costs out of the loss. Missing one defect that reaches a client costs orders of magnitude more than re-checking a perfectly good product, yet the default loss function counts both as one error. If you leave the loss function as is and only tweak the threshold to get by, the same problem returns every time you retrain.
Sungil KimKAIST and Samsung Electronics alumnus and global pitching expert who coached the winning teams at the UN Tourism and Select USA IR pitch competitions
Ara LimExpert in global leadership, AI, and business communicationIt comes from the error-measurement concepts of statistics and optimization theory, and became the standard objective function for training neural networks.
Consider a hypothetical case. Company A, an auto parts maker with 240 employees and annual revenue of around KRW 90 (≈ US$1) billion, introduced AI into its visual inspection process. The defect rate was 1.2% of total volume, and after three months of training, a reported accuracy of 98.9% reached the management meeting. Yet customer complaints over the same period stayed at 14 a month.
- When Manager Park of the Quality Assurance team opened the inspection logs himself, he found the model had caught only 11 of the 86 actual defects. “Isn’t 98.9% accuracy just the score you get by labeling everything normal?” someone asked in the meeting.
- The data team traced the cause to using plain cross-entropy on imbalanced data, so they gave the defect class a weight of 20, added Focal Loss, and retrained.
- When the head of production stated flatly, “Missing one defect costs KRW 30 million (≈ US$22,400) in recall handling; one false alarm costs KRW 4,000 (≈ US$3) in re-inspection labor,” the team built a miss penalty 30 times the false-alarm penalty into the loss.
- In validation, accuracy fell from 98.9% to 97.4%, but the defect detection rate rose from 12.8% to 79%. Manager Park changed the reporting metrics from accuracy to two: detection rate and false-alarm rate.
- Three months later, defects escaping to the client dropped from 14 a month to 2. The false-reject rate rose from 3.1% to 7.6%, but the added re-inspection labor cost only about KRW 600,000 (≈ US$450) a month, so they decided to keep the setting.