recommender-evaluation

Measure whether a recommender or ranking system is any good, offline and online. Use whenever someone asks how to evaluate or compare recommendation models; how to split interaction data (random, temporal, leave-one-out) without leaking the future; whether to use RMSE or ranking metrics; how to compute precision@k, recall@k, hit rate, MAP, MRR or NDCG; how to check catalogue coverage, diversity, novelty, popularity bias or long-tail exposure; whether a new recommender beats the current one or a best-seller baseline; how to run a shadow deployment or A/B test for recommendations; or how many users that experiment needs to detect a lift. Also use for judging search-result ranking quality. Not for building the recommender, and not for setting a classifier's probability threshold.