Statistical biases in Information Retrieval metrics for recommender systems

Armstrong, T. G., Moffat, A., Webber, W., & Zobel, J. (2009a). Has ad-hoc retrieval improved since 1994? In Proceedings of the 32nd ACM conference on Research and development in Information Retrieval, SIGIR’09. ACM, pp. 692–693.

Armstrong, T. G., Moffat, A., Webber, W., & Zobel, J. (2009b). Improvements that don’t add up: Ad hoc retrieval results since 1998. In Proceedings of the 18th ACM conference on Information and knowledge management, CIKM’09. ACM, pp. 601–610.

Baeza-Yates, R., & Ribeiro-Neto, B. (2011). Modern information retrieval: The concepts and technology behind search (2nd ed.) (ACM Press Books). Addison-Wesley Professional.

Barbieri, N., Costa, G., Manco, G., & Ortale, R. (2011). Modeling item selection and relevance for accurate recommendations: a bayesian approach. In Proceedings of the 5th ACM conference on recommender systems, RecSys’11. ACM, pp. 21–28.

Basu, C., Hirsh, H., & Cohen, W. W. (1998). Recommendation as classification: Using social and content-based information in recommendation. In Proceedings of AAAI/IAAI’98, pp. 714–720.

Bellogín, A., Castells, P., & Cantador, I. (2011). Precision-oriented evaluation of recommender systems: an algorithmic comparison. In Proceedings of the 5th ACM conference on recommender systems, RecSys’11. ACM, pp. 333–336.

Breese, J. S., Heckerman, D., & Kadie, C. (1998). Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the 14th annual conference on uncertainty in artificial intelligence, UAI’98, pp. 43–52.

Buckley, C., Dimmick, D., Soboroff, I., & Voorhees, E. M. (2007). Bias and the limits of pooling for large collections. Information Retrieval, 10(6), 491–508.

Buckley, C., & Voorhees, E. M. (2004). Retrieval evaluation with incomplete information. In Proceedings of the 27th ACM conference on research and development in information retrieval, SIGIR’04. ACM, pp. 25–32.

Cañamares, R., & Castells, P. (2014). Exploring social network effects on popularity biases in recommender systems. In Proceedings of the 6th workshop on recommender systems and the social web, RSWeb’14, at the 8th ACM conference on recommender systems, RecSys’14.

Cañamares, R., & Castells, P. (2017). A probabilistic reformulation of memory-based collaborative filtering—Implications on popularity biases. In Proceedings of the 40th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR’17. ACM.

Celma, O. (2010). Music recommendation and discovery: The long tail, long fail, and long play in the digital music space (1st ed.). Berlin: Springer.

Celma, O., & Cano, P. (2008). From hits to niches? Or how popular artists can bias music recommendation and discovery. In NETFLIX’08: Proceedings of the 2nd KDD workshop on large-scale recommender systems and the netflix prize competition. ACM, pp. 1–8.

Celma, O., & Herrera, P. (2008). A new approach to evaluating novel recommendations. In Proceedings of the 2nd ACM conference on recommender systems, RecSys’08. ACM, pp. 179–186.

Chen, L., & Pan, W. (2013). Cofiset: Collaborative filtering via learning pairwise preferences over item-sets. In Proceedings of the 13th SIAM international conference on data mining, pp. 180–188.

Cremonesi, P., Koren, Y., & Turrin, R. (2010). Performance of recommender algorithms on top-n recommendation tasks. In Proceedings of the 4th ACM conference on recommender systems, RecSys’10. ACM, pp. 39–46.

Fleder, D. M. and Hosanagar, K. (2007). Recommender systems and their impact on sales diversity. In Proceedings 8th ACM conference on electronic commerce (EC’07), pp. 192–199.

Harper, F. M., & Konstan, J. A. (2016). The movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems, 5(4), 19.

Herlocker, J. L., Konstan, J. A., Terveen, L. G., & Riedl, J. T. (2004). Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1), 5–53.

Hofmann, T. (2004). Latent semantic models for collaborative filtering. ACM Transactions on Information Systems, 22(1), 89–115.

Jambor, T., & Wang, J. (2010a). Goal-driven collaborative filtering—A directional error based approach. In C. Gurrin, Y. He, G. Kazai, U. Kruschwitz, S. Little, T. Roelleke, S. Rüger, & K. Rijsbergen (Eds.), Advances in information retrieval (vol. 5993, chapter 36, pp. 407–419). Springer.

Jambor, T., & Wang, J. (2010b). Optimizing multiple objectives in collaborative filtering. In Proceedings of the 4th ACM conference on recommender systems, RecSys’10. ACM, pp. 55–62.

Jannach, D., Lerche, L., Kamehkhosh, I., & Jugovac, M. (2015). What recommenders recommend: an analysis of recommendation biases and possible countermeasures. User Modeling and User-Adapted Interaction, 25(5), 427–491.

Kluver, D., & Konstan, J. A. (2014). Evaluating recommender behavior for new users. In Proceedings of the 8th ACM conference on recommender systems, RecSys’14. ACM, pp. 121–128.

Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30–37.

Levy, M., & Bosteels, K. (2010). Music recommendation and the long tail. In 1st workshop on music recommendation and discovery, WOMRAD’10, at the 4th ACM conference on recommender systems, RecSys’10.

Pradel, B., Usunier, N., & Gallinari, P. (2012). Ranking with non-random missing ratings: Influence of popularity and positivity on evaluation metrics. In Proceedings of the 6th ACM conference on recommender systems, RecSys’12. ACM, pp. 147–154.

Shani, G., Chickering, D. M., & Meek, C. (2008). Mining recommendations from the web. In Proceedings of the 2nd ACM conference on recommender systems, RecSys’08. ACM, pp. 35–42.

Shani, G., & Gunawardana, A. (2011). Evaluating recommendation systems. In F. Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds.), Recommender systems handbook (chapter 8, pp. 257–297). Springer.

Shi, Y., Karatzoglou, A., Baltrunas, L., Larson, M., Oliver, N., & Hanjalic, A. (2012). Climf: Learning to maximize reciprocal rank with collaborative less-is-more filtering. In Proceedings of the 6th ACM conference on recommender systems, RecSys’12. ACM, pp. 139–146.

Shi, Y., Serdyukov, P., Hanjalic, A., & Larson, M. (2011). Personalized landmark recommendation based on geotags from photo sharing sites. In Proceedings of the 5th international conference on weblogs and social media.

Steck, H. (2011). Item popularity and recommendation accuracy. In Proceedings of the 5th ACM conference on recommender systems, RecSys’11. ACM, pp. 125–132.

Steck, H., & Xin, Y. (2010). A generalized probabilistic framework and its variants for training top-k recommender system. In Proceedings of the workshop on the practical use of recommender systems, algorithms and technologies, PRSAT’10, pp. 35–42.

van Rijsbergen, C. J. (1989). Towards an information logic. SIGIR Forum, 23(SI), 77–86.

Vargas, S., & Castells, P. (2011). Rank and relevance in novelty and diversity metrics for recommender systems. In Proceedings of the 5th ACM conference on recommender systems, RecSys’11. ACM, pp. 109–116.

Voorhees, E. M. (2001). The philosophy of information retrieval evaluation. In Evaluation of cross-language information retrieval systems, 2nd workshop of the cross-language evaluation forum, CLEF’01, revised papers, pp. 355–370.

Voorhees, E. M., & Harman, D. K. (Eds.). (2005). TREC: Experiment and evaluation in information retrieval. Cambridge: MIT Press.

Yilmaz, E., & Aslam, J. A. (2006). Estimating average precision with incomplete and imperfect judgments. In Proceedings of the 15th ACM conference on information and knowledge management, CIKM’06. ACM, pp. 102–111.