Stability of Recommendation Algorithms

ACM Transactions on Information Systems - Tập 30 Số 4 - Trang 1-31 - 2012
Gediminas Adomavičius1, Jingjing Zhang2
1University of Minnesota
2Indiana University,

Tóm tắt

The article explores stability as a new measure of recommender systems performance. Stability is defined to measure the extent to which a recommendation algorithm provides predictions that are consistent with each other. Specifically, for a stable algorithm, adding some of the algorithm’s own predictions to the algorithm’s training data (for example, if these predictions were confirmed as accurate by users) would not invalidate or change the other predictions. While stability is an interesting theoretical property that can provide additional understanding about recommendation algorithms, we believe stability to be a desired practical property for recommender systems designers as well, because unstable recommendations can potentially decrease users’ trust in recommender systems and, as a result, reduce users’ acceptance of recommendations. In this article, we also provide an extensive empirical evaluation of stability for six popular recommendation algorithms on four real-world datasets. Our results suggest that stability performance of individual recommendation algorithms is consistent across a variety of datasets and settings. In particular, we find that model-based recommendation algorithms consistently demonstrate higher stability than neighborhood-based collaborative filtering techniques. In addition, we perform a comprehensive empirical analysis of many important factors (e.g., the sparsity of original rating data, normalization of input data, the number of new incoming ratings, the distribution of incoming ratings, the distribution of evaluation data, etc.) and report the impact they have on recommendation stability.

Từ khóa


Tài liệu tham khảo

10.1109/TKDE.2005.99

10.1145/1864708.1864722

10.1145/245108.245124

Bell R. M., Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 7--14

10.1145/1345448.1345465

10.1109/ICDM.2007.90

Bennett J., Proceedings of the KDD-Cup and Workshop. www.netflixprize.com.

Billsus D., Proceedings of the 15th International Conference on Machine Learning (ICML’98)

Breese J. S., Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence.

Burden R. L. and Faires D. 2004. Numerical Analysis. Thomson Brooks/Cole. Burden R. L. and Faires D. 2004. Numerical Analysis . Thomson Brooks/Cole.

10.1002/(SICI)1520-6793(199912)16:8<677::AID-MAR4>3.0.CO;2-T

10.1145/1454008.1454054

Duda R. O. Hart P. E. and Stork D. G. 2000. Pattern Classification. Wiley. Duda R. O. Hart P. E. and Stork D. G. 2000. Pattern Classification . Wiley.

Funk S. 2006. Netflix update: Try this at home. http://sifter.org/~simon/journal/20061211.html. Funk S. 2006. Netflix update: Try this at home. http://sifter.org/~simon/journal/20061211.html.

Garfinkel R. Gopal R. Pathak B. Venkatesan R. and Yin F. 2006. Empirical analysis of the business value of recommender systems. http://ssrn.com/abstract=958770. Garfinkel R. Gopal R. Pathak B. Venkatesan R. and Yin F. 2006. Empirical analysis of the business value of recommender systems. http://ssrn.com/abstract=958770.

10.1207/153276603768344870

10.1023/A:1011419012209

Grouplens. 2006. Movielens Data Sets. http://www.grouplens.org/. Grouplens. 2006. Movielens Data Sets. http://www.grouplens.org/.

Hastie T. Tibshirani R. and Friedman J. 2009. The Elements of Statistical Learning: Data Mining Inference and Prediction. Springer. Hastie T. Tibshirani R. and Friedman J. 2009. The Elements of Statistical Learning: Data Mining Inference and Prediction. Springer.

10.1145/312624.312682

10.1145/963770.963772

Higham N. J., Accuracy and Stability of Numerical Algorithms, 10.1137/1.9780898718027

10.5555/2017317.2017325

10.1145/1644873.1644874

10.1109/MC.2009.263

10.1145/245108.245126

10.1145/988672.988726

Legwinski T. 2010. Recommendation explanations increase sales & customer confidence. http://blog.strands.com/2010/03/17/recommendations-increase-sales-customer-confidence/. Legwinski T. 2010. Recommendation explanations increase sales & customer confidence. http://blog.strands.com/2010/03/17/recommendations-increase-sales-customer-confidence/.

Lyapunov A. M., The General Problem of the Stability of Motion

10.1145/1297231.1297235

Massa P., Proceedings of the International Conference on Trust Management. Springer, 221--235

Mobasher B., Proceedings of the 21st Conference on Artificial Intelligence (AAAI’06)

10.1007/11891321_6

10.1145/1278366.1278372

10.1145/1040830.1040870

10.1142/S0218213006003053

10.1023/B:AIRE.0000036256.39422.25

10.1145/192844.192905

Rigby C. 2011. Internet Retailing webinars. Review: Boosting email marketing revenue with personalised recommendations by Silverpop Baynote and MusicRoom.com. http://www.internetretailing.net/2011/10/internet-retailing-webinars-review-boosting-email-marketing-revenue-with-personalised-recommendations-by-silverpop-baynote-and-musicroom-com/. Rigby C. 2011. Internet Retailing webinars. Review: Boosting email marketing revenue with personalised recommendations by Silverpop Baynote and MusicRoom.com. http://www.internetretailing.net/2011/10/internet-retailing-webinars-review-boosting-email-marketing-revenue-with-personalised-recommendations-by-silverpop-baynote-and-musicroom-com/.

10.1145/289444.289509

10.1145/371920.372071

Shani G. and Gunawardana A. 2011. Evaluating recommender systems. In Recommender Systems Handbook: A Complete Guide for Research Scientists and Practitioners P. Kantor F. Ricci L. Rokach and B. Shapira Eds. Springer. Shani G. and Gunawardana A. 2011. Evaluating recommender systems. In Recommender Systems Handbook: A Complete Guide for Research Scientists and Practitioners P. Kantor F. Ricci L. Rokach and B. Shapira Eds. Springer.

Simitses G. and Hodges D. 2005. Fundamentals of Structural Stability. Butterworth-Heinemann Burlington MA. Simitses G. and Hodges D. 2005. Fundamentals of Structural Stability . Butterworth-Heinemann Burlington MA.

10.1080/09528139408953794

10.1007/BF00993473

10.1348/014466604X17092

10.17705/1jais.00065

Thông tin
Thông tin xuất bản

ACM Transactions on Information Systems

Tập 30 Số 4

1-31

Thông tin tác giả