Martijn Willemsen The recommender LAB of Martijn Willemsen researches how decisions can be supported by recommender systems, and includes domains such as movies, music, health related decisions and energy-saving measures. The LAB focuses on how insights from decision psychology can improve recommender algorithms, how to best evaluate recommender systems and on novel recommendation methods that help users with developing preferences and goals.

Movie recommendations

Classically, many research on recommender systems is done on movie recommendations, due to the richness of this domain and the availability of good data sets. Interest for this domain was strengthened by the Netflix price (2006-2009), that rewarded a one million dollar price to those that would improve 10% in accuracy over CineMatch, the Netflix Algorithm. The price brought many new algorithmic approaches, such as the use of Matrix Factorization but was also critized as it focuses too much on predictive accuracy improvements and offline evaluation. Our own work in our lab has always been focused more on evaluation and AB testing with real users, employing our user centric framework to measure user perceptions and evaluations of different algorithmic approaches and interface designs. Below we highlight a few topics and relevant papers.

Choice Difficulty and Diversification

As recommender algorithms tend to provide many similar and often quite accurate recommendations, they might result in choice overload, as we have shown in Bollen et al. (2010). Based on this initial research, we developed a latent feature diversification algorithm that increases the diversity of recommendations based on insights from psychology, while controlling for the overall attractiveness of the items. Our work (Willemsen et al., 2016) has shown that more diversity increases the perceived quality of the recommendations and reduces choice difficulty, leading to increased satisfaction over standard top-N, even though users actually selected items that had lower predicted ranks.

Relevant research papers:

  • Willemsen, M.C., Graus, M.P, & Knijnenburg, B.P. (2016). Understanding the role of latent feature diversification on choice difficulty and satisfaction. User Modeling and User-Adapted Interaction (UMUAI), vol 26 (4), 347-389 doi:10.1007/s11257-016-9178-6
  • Bollen, D., Knijnenburg, B. P., Willemsen, M. C., & Graus, M. (2010). Understanding choice overload in recommender systems. In Proceedings of the fourth ACM conference on Recommender systems - RecSys ’10 (pp. 63–70). Barcelona, Spain. free access via ACM

Improving user experience in recommender systems from Martijn Willemsen

Understanding and improving ratings

Many recommender systems use ratings (typically on a 5-star scale) as a proxy for user liking or preference. Ratings have shown to be quite noisy and unstable, and our own work has focused on the role of our memory on ratings (Bollen et al. 2012). Ratings for movies change as the time between viewing the movie and providing the rating increases. Work with grouplens (Nguyen et al. 2013) has investigated how we can support users during the rating task to counter such memory effects.

Relevant research papers:

  • Nguyen, T. T., Kluver, D., Wang, T.-Y., Hui, P.-M., Ekstrand, M. D., Willemsen, M. C., & Riedl, J. (2013). Rating Support Interfaces to Improve User Experience and Recommender Accuracy. In Proceedings of the 7th ACM Conference on Recommender Systems (pp. 149–156). New York, NY, USA: ACM free access via ACM
  • Bollen, D., Graus, M., & Willemsen, M. C. (2012). Remembering the stars?: effect of time on preference retrieval from memory. In Proceedings of the sixth ACM conference on Recommender systems (pp. 217–220). New York, NY, USA: ACM free access via ACM

Choice-based preference elicitation

Ratings are made on an absolute scale, providing an overall assessment of an item, which is a difficult task for a user. Moreover preferences are inherently relative, we typically say, I like A over B (see also Jameson et al. 2015 for a discussion on this). Therefore, we developed choice-based preference elicitation techniques that measure user profiles via a set of choices rather than by asking a number of ratings.

Relevant research papers:

  • Graus, M.P. & Willemsen, M.C. (2016). Can Trailers Help to Alleviate Popularity Bias in Choice-Based Preference Elicitation? Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at the ACM RecSys 2016 conference, September 15-19, 2016 link to paper
  • Jameson, A., Willemsen, M. C., Felfernig, A., Gemmis, M. de, Lops, P., Semeraro, G., & Chen, L. (2015). Human Decision Making and Recommender Systems. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (pp. 611–648). Springer US. link to Springer
  • Graus, M.P. & Willemsen, M.C. (2015). Improving the user experience during cold start through choice-based preference elicitation. In Proceedings of the 9th ACM conference on Recommender systems (pp. 273-276), New York, NY, USA: ACM free access via ACM