9.1 Introduction

9.1.1 What is a Recommender System?

A recommender (recommendation) system is a method to predict the rating a user would give to an item.

For example:

  • playlist generators at Spotify, YouTube or Netflix,
  • content recommendations on Facebook, Instagram, Twitter or Apple News,
  • product recommendations at Amazon or Alibaba.

(Ricci et al. 2011) list the following functions of recommender systems:

  • increase the number of items sold,
  • sell more diverse items,
  • increase users’ satisfaction,
  • increase users’ fidelity,
  • better understand what users want.

9.1.2 The Netflix Prize

In 2006 Netflix (back then a DVD rental company) released one of the most famous benchmark sets for recommender systems, which helped boost the research on algorithms in this field.

See https://www.kaggle.com/netflix-inc/netflix-prize-data; data archived at https://web.archive.org/web/20090925184737/http://archive.ics.uci.edu/ml/datasets/Netflix+Prize and https://archive.org/details/nf_prize_dataset.tar

The dataset consists of:

  • 480,189 users
  • 17,770 movies
  • 100,480,507 ratings in the training sample:

    • MovieID
    • CustomerID
    • Rating from 1 to 5
    • Title
    • YearOfRelease from 1890 to 2005
    • Date of rating in the range 1998-11-01 to 2005-12-31

The quiz set consists of 1,408,342 ratings and it was used by the competitors to assess the quality of their algorithms and compute leaderboard scores.

Root mean squared error (RMSE) of predicted vs. true rankings was chosen as a performance metric.

The test set of 1,408,789 ratings was used to determine the winner.

On 21 Sept. 2009, the grand prize of US$1,000,000 was given to the BellKor’s Pragmatic Chaos team which improved over the Netflix’s Cinematch algorithm by 10.06%, achieving the winning RMSE of 0.8567 on the test subset.

9.1.3 Main Approaches

Current recommender systems are quite complex and use a fusion of various approaches, also those based on external knowledge bases.

However, we may distinguish at least two core approaches, see (Ricci et al. 2011) for more:

  • Collaborative Filtering

    It is based on the assumption that if two people interact with the same product, they’re likely to have other interests in common as well.

    John and Mary both like bananas and apples and dislike spinach. John likes sushi. Mary hasn’t tried sushi yet. It seems they might have similar tastes, so we recommend that Mary should give sushi a try.

  • Content-based Filtering

    It builds a user’s profile that represent information of what kind of products does she/he like.

    We have discovered that John likes fruits but dislikes vegetables. An orange is a fruit (an item similar to those he liked in the past) with which John is yet to interact. John should give it a try.

Jim Bennett, vice president of recommendations systems at Netflix on the idea behind the original Cinematch algorithm (see https://www.technologyreview.com/s/406637/the-1-million-netflix-challenge/):

First, you collect 100 million user ratings for about 18,000 movies. Take any two movies and find the people who have rated both of them. Then look to see if the people who rate one of the movies highly rate the other one highly, if they liked one and not the other, or if they didn’t like either movie. Based on their ratings, Cinematch sees whether there’s a correlation between those people. Now, do this for all possible pairs of 65,000 movies.

See also: https://web.archive.org/web/20070821194257/http://www.netflixprize.com/faq

Is it an example of collaborative or context-based filtering?

9.1.4 Formalism

Let \(\mathcal{U}=\{ U_1,\dots,U_m \}\) denote the set of \(m\) users.

Let \(\mathcal{I}=\{ I_1,\dots,I_n \}\) denote the set of \(n\) items.

Let \(R\in\mathbb{R}^{m\times n}\) be a user-item matrix such that: \[ r_{u,i}=\left\{ \begin{array}{ll} r & \text{if the $u$-th user ranked the $i$-th item as $r>0$}\\ 0 & \text{if the $u$-th user hasn't interacted with the $i$-th item yet}\\ \end{array} \right. \]

Note that here 0 is used to denote a missing value (NA)

In particular, we can assume:

  • \(r_{u,i}\in\{0,1,\dots,5\}\) (ratings on the scale 1–5 or no interaction)
  • \(r_{u,i}\in\{0,1\}\) (“Like” or no interaction)

The aim of an recommender system is to predict the rating \(\hat{r}_{u,i}\) that the \(u\)-th user would give to the \(i\)-th item provided that currently \(r_{u,i}=0\).