Recommendation System
Recommendation System
What is a recommendation system?
- Analyzes tastes and behavior to suggest what you’ll like
- The engine behind Netflix and YouTube
- The flagship of personalization
What changes when recommendations arrive is not a few spots on the screen but the fact that merchandising authority passes from people to a model. Merchandisers and planners used to decide the main banner and top placement order in meetings; now the first screen is assembled differently for each user. So the conflict that erupts early in adoption is not technical but a turf battle. The marketing team, which must push promotional items, and the model, which is trying to lift click-through, collide over the same slots, and you end up documenting an agreement on “this many fixed slots, this many algorithm slots”.
There are two main approaches. Collaborative filtering pulls in what people with similar behavior to yours have viewed. It gets stronger as data accumulates, but it is helpless with new users and new items. Content-based filtering, by contrast, reads item descriptions, tags, and categories. It works from day one, but keeps returning more of the same. So in practice most teams blend the two, and split the process into two stages: candidate generation and ranking. Only by first narrowing tens of thousands of items to a few hundred, then ranking just those few hundred with care, can you keep response times within tens of milliseconds.
Success is judged on two levels. During development you watch accuracy metrics such as Recall@K and NDCG, but the final verdict comes from conversion and repeat visits in an A/B test. Offline scores often rise while actual revenue stays flat. The line between this and a rule-based “frequently viewed together” list is also clear. A rule gives the same list to everyone, while a recommendation system produces a different list for each person, even on the same page. That means the two differ in operating cost and management effort from day one.
The most common failure in practice is crediting all “revenue via recommendations” to the recommender. Showing an item once more to someone who was going to buy anyway lifts the metric, but total sales stay put. If you report results without a control group that has recommendations switched off, the budget grows while the effect is never proven. It is also common for out-of-stock items or courses a user has already completed to keep showing up, and once users have been burned, they start ignoring the recommendation area altogether. Lock down diversity rules and exclusion conditions before launch.
John KimInstructor John Kim, author of “B2B Marketing Design” and CEO of Nanum Consulting
Junyoung NohConsumer and generational trend analyst & AI content marketing expertIt began with 1990s collaborative filtering research, advanced through milestones such as the 2006 Netflix Prize, and has become more sophisticated with deep learning.
Let’s look at a hypothetical case. Company A, a Korean parts manufacturer, has 3,200 employees and 1,840 courses on its internal learning platform. Yet only 19% of monthly active users went on to complete a course, and the top 30 most popular courses accounted for 71% of all enrollments. The HRD team lead said, “We have plenty of courses, but no one can find their own.”
- When Kim on the HRD team pulled search, enrollment, and drop-off logs for the past 12 months, 43 of the top 100 search terms ended in “0 results.”
- For phase one, we started with rules rather than a model. Using three axes (job function, level, and completion history), we showed the 10 courses most taken by senior colleagues in the same role on the home page, and always excluded courses the learner had already completed.
- Three months later, after collaborative filtering was added, the recommendation panel was empty for 400 new hires. Park, the data lead, split out a separate cold-start phase, saying, “Let’s run the first two weeks on job profiles.”
- We split exposure 50/50 and ran an 8-week A/B test. Click-through rose 14 points, but completion stayed flat, so we switched the success metric from “clicks” to “completion within 4 weeks” and retrained the ranking.
- Six months later, monthly course completion had risen from 19% to 31%, and the share captured by the top 30 courses had fallen from 71% to 52%. Of the 610 courses that had never been opened, 280 got their first learner.