10.X Case Study: Recommender Systems

1Learning Objectives¶

By the end of this page, you should know:

• the definition of a recommender system based off of collaborative filtering
• how to express collaborative filtering as a matrix completion problem
• one approach to solve the matrix completion via the singular value decompositon

2Types of Recommender System¶

Recommender systems have become a vital part of our digital lives, guiding us towards products, services, and content we might like. Generally speaking, recommender systems are algorithms designed to suggest relevant items to users. These systems are used in various domains such as e-commerce, streaming services, and social media. They enhance user experience by filtering vast amounts of information to deliver personalized content. There are two main categories of recommender systms:

• Content-Based Filtering: Content-based filtering recommends items to users by analyzing the features of items they have previously liked, matching similar items based on those attributes.
• Collaborative Filtering: Collaborative filtering recommends items to users by identifying patterns and similarities in user behavior and preferences, leveraging the collective data from multiple users to make personalized suggestions.

3Movie Recommendations¶

The tools that we have learned in this class are used in the design of both types of recommender system. In this case study, we will focus on collaborative filtering, where the singular value decomposition plays a critical roll. To start off with, let’s download a dataset consisting of movie ratings from various users.

We loaded a dataframe that the ratings info from a collection of users.

Our dataset now consists only of the movie titles, the user id, and the user rating. Below, we’ll look at the number of unique movies, the number of users, the range of the ratings, and the total number of entries in our dataset.

Since there are 100000 datapoints, and 943 users, it means that we have reviews for about 80 movies per user on average. This means that on average there are over 1500 movies that haven’t been watched by any given user. Our goal is to help each user to select which of these movies to watch next!

Our first step in doing so will be to arrange the dataset into a matrix, and the users are the rows, and the movies are the columns. For every movie that has been reviewed by a user, we will fill the corresponding entry in the matrix with the ratings given by the user.

We first construct our matrix, called “ratings_matrix”. All entries will be initialized to zero. Then we will iterate through the dataset, and for every (movie, user) pair for which we have a rating, we will add that rating to our matrix.

As the minimum rating was 1.0, all 0.0 entries are correspond to (user, movie) pairs where no rating was available in our dataset. Our first step to recommend movies to users will be to impute, or fill in, the missing entries.

Without additional assumptions about the data, the imputation of this data can be quite random. Therefore, the standard assumption imposed for collaborative recommender systems is the rankings matrix is of rank $k < \min\{\# \mathrm{Users}, \# \mathrm{Movies}\}$.

This assumption suggests that if we apply an singular value decomposition to the ratings matrix, there will be relatively few large singular values. Therefore, using the patterns just in the data that is present in the ratings matrix, we may be able to guess values for entries that are missing. Generally, this problem is referred to as low-rank matrix completion. In this case study, we will take a rather simple approach consisting of three steps:

• Fill in the missing entries with some crude guess
• Take the singular value decomposition, and truncate all but $k$ singular values.
• Construct an estimate for the ratings matrix using the truncated singular vectors and values.

There are many ways to fill in the missing entries. One option would be to assign the missing entries to have the average rating over all of the entries that are present. Another option would be to assume that users give fairly similar ratings to all movies, and assign missing entries in each row with the average of the existing entries in each row. Alternatively, we could assume that good movies are appreciated by all users, and assign missing entries in each column with the average of the existing entries in the column. We will stick with the last option for the remainder of this case study, but you are encouraged to try the other possibilities and see how the results change if you are curious!

Let’s see what this matrix looks like below.

We now see that there are no longer zero entries in the ratings matrix. All missing values have been imputed with a crude guess. Let’s now move on to the second step: taking the SVD of the ratings matrix.

Let’s see how the singular values decay by plotting them on a log scale.

The entries quickly decay in importance after about the first hundred. Let’s try keeping 100 singular values for the reconstruction.

The result looks quite similar to the crudely imputed matrix from earlier, but with some small deviations. Let’s use the result to recommend movies to a user. To start, we’ll examine the hisotry of ratings by user zero.

Now let’s look what our recommender system suggests. To recover the recommendations, we first make a python dictionary mapping the movie index back into the movie name.

The recommendations we make to the user will now be the entries in the user’s row of the reconstruction matrix with the highest score. Let’s print out 20 recommendations for user zero below. Along with each recommendation, we’ll check whether the user has already watched that movie.

We have successfully recommended movies to the user! Some of them have already been watched, some of them are new. Unfortunately, it is impossible to know the effectiveness of our recommendation system unless the user watches some of the recommended movies, and tells us whether they enjoyed the movie or not.

Even within this simple SVD based approach for developing a recommender system, there are many options that we can choose as engineers to tune the performance of the system. The two most critical are 1) the way in which the crude initial imputation is performed, and 2) the rank of the ratings matrix that we should assume.