Rating Errors · Leniency, Central Tendency, Recency Effect
Rater Errors in Performance Appraisal
What are rating errors?
- Rating errors are skewed judgment, not a lack of conscience
- Three groups: distribution, time and comparison, and people
- Reduce it through procedures such as distribution checks and observation logs
If you treat rating errors as a matter of the rater’s conscience or effort, the remedy ends with an appeal to “be fair.” But rating errors are judgment biases that show up in a consistent direction even among conscientious raters. Memory favors recent events, people avoid conflict, and standards shift depending on who is being compared. That is why the practical starting point is that errors are reduced by procedure, not willpower.
It helps to sort rating errors into three groups. The first is distribution errors, where scores cluster on one side. These include leniency (high scores for most people), central tendency (everyone rated as average), and strictness (low scores across the board). The second is time and comparison errors: recency effect (reflecting only the last two or three months before the review), primacy effect (a first impression that lasts to the end), and contrast effect (scores that swing depending on the person rated just before). The third is errors about the person: the halo effect, where a single strength lifts the whole score, and similarity bias, where raters favor people like themselves.
You can diagnose this with numbers. Pull the rating distribution for each rater: if the share of top ratings is clearly above the organizational average, suspect leniency; if the standard deviation is unusually small, suspect central tendency. If one person's scores are identical across all criteria, that signals a halo effect. The remedies differ by error, too. Quarterly observation logs work for recency effect, rating-level anchor statements and calibration meetings work for distribution errors, and rating everyone on one criterion at a time instead of one person at a time works for contrast effect.
Where things go wrong in practice is running error training once and stopping there. Knowing the names of the errors is different from reducing them, and without record templates and distribution checks, the next review produces the same distribution. Be equally careful about using forced distribution as the fix. The distribution looks even, but a team that truly performed well gets marked down without justification, which only swaps one bias for another kind of unfairness.
Anna ChoExpert in organizational culture, coaching, emotion management, and CS consulting
John KimInstructor John Kim, CEO of Nanum Consulting and author of B2B Marketing Design| Category | Typical errors | Warning signs | How to reduce them |
|---|---|---|---|
| Distribution errors | Leniency, central tendency, strictness | Skewed ratings, small standard deviation | Rating-level anchor statements, calibration meeting |
| Time and comparison errors | Recency effect, primacy effect, contrast effect | Evidence clustered in one period | Quarterly observation logs, criterion-level rating |
| Errors about the person | Halo effect, similarity bias | Identical scores across all criteria | Independent criterion-level rating, multiple raters |
The problem of rater scores drifting systematically from actual performance has long been studied in industrial and organizational psychology research on performance appraisal, and labels like leniency, central tendency, and the halo effect were codified within that body of work. Today they are a standard topic in rater training.
Consider a hypothetical case. Company A, a retailer with 350 employees, has 28 team leaders conducting first-round reviews. Last year, 71% of all employees scored 4 or higher on a 5-point scale, and all 9 members of one team received the same score. Critics noted that the ratings gave no real basis for differentiating pay.
- An HR manager pulls the rating distributions of the 28 team leaders. Nine leaders give scores of 4 or higher more than 85% of the time, and six have a score standard deviation of 0.3 or lower.
- In rater training, each person gets a copy of their own distribution chart with names hidden. One team leader says, “I didn’t realize I had given everyone the exact same score.”
- The logistics team lead finds that the notes backing the ratings only cover events since October. HR adds a field to the review system for two lines of observation notes per team member each quarter.
- The review screen changes from entering scores person by person to criterion by criterion. A team leader first compares all team members on “goal achievement” and then moves on to the next criterion.
- Attach two anchor sentences to each rating level. A 4 reads “exceeded goals and helped colleagues with their work,” so team leaders recall concrete examples when they assign a score.
- In the next review, the share of scores of 4 or higher drops from 71% to 46%, and teams where everyone gets the same score disappear. Managers with a standard deviation of 0.3 or lower fall from 6 to 1.