HW1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Матричные факторизации"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В данной работе вам предстоит познакомиться с практической стороной матричных разложений.\n",
    "Работа поделена на 4 задания:\n",
    "1. Вам необходимо реализовать SVD разложения используя SGD на explicit данных\n",
    "2. Вам необходимо реализовать матричное разложения используя ALS на implicit данных\n",
    "3. Вам необходимо реализовать матричное разложения используя BPR(pair-wise loss) на implicit данных\n",
    "4. Вам необходимо реализовать матричное разложения используя WARP(list-wise loss) на implicit данных\n",
    "\n",
    "Мягкий дедлайн 28 Сентября (пишутся замечания, выставляется оценка, есть возможность исправить до жесткого дедлайна)\n",
    "\n",
    "Жесткий дедлайн 5 Октября (Итоговая проверка)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/lib/python3.7/site-packages/lightfm/_lightfm_fast.py:9: UserWarning: LightFM was compiled without OpenMP support. Only a single thread will be used.\n",
      "  warnings.warn('LightFM was compiled without OpenMP support. '\n"
     ]
    }
   ],
   "source": [
    "import implicit\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import scipy.sparse as sp\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from tqdm import tqdm\n",
    "from lightfm.datasets import fetch_movielens\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "from collections import namedtuple"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В данной работе мы будем работать с explicit датасетом movieLens, в котором представленны пары user_id movie_id и rating выставленный пользователем фильму\n",
    "\n",
    "Скачать датасет можно по ссылке https://grouplens.org/datasets/movielens/1m/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings = pd.read_csv('RecSysHSE/ml-1m/ratings.dat', delimiter='::', header=None, \n",
    "        names=['user_id', 'movie_id', 'rating', 'timestamp'], \n",
    "        usecols=['user_id', 'movie_id', 'rating'], engine='python')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "movie_info = pd.read_csv('RecSysHSE/ml-1m/movies.dat', delimiter='::', header=None, \n",
    "        names=['movie_id', 'name', 'category'], engine='python')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Explicit данные"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>movie_id</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1193</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>661</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>914</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3408</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>2355</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>1197</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1</td>\n",
       "      <td>1287</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>2804</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>594</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>919</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  movie_id  rating\n",
       "0        1      1193       5\n",
       "1        1       661       3\n",
       "2        1       914       3\n",
       "3        1      3408       4\n",
       "4        1      2355       5\n",
       "5        1      1197       3\n",
       "6        1      1287       5\n",
       "7        1      2804       5\n",
       "8        1       594       4\n",
       "9        1       919       4"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Для того, чтобы преобразовать текущий датасет в Implicit, давайте считать что позитивная оценка это оценка >=4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "implicit_ratings = ratings.loc[(ratings['rating'] >= 4)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>movie_id</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1193</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3408</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>2355</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1</td>\n",
       "      <td>1287</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>2804</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>594</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>919</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1</td>\n",
       "      <td>595</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1</td>\n",
       "      <td>938</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1</td>\n",
       "      <td>2398</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    user_id  movie_id  rating\n",
       "0         1      1193       5\n",
       "3         1      3408       4\n",
       "4         1      2355       5\n",
       "6         1      1287       5\n",
       "7         1      2804       5\n",
       "8         1       594       4\n",
       "9         1       919       4\n",
       "10        1       595       5\n",
       "11        1       938       4\n",
       "12        1      2398       4"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "implicit_ratings.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Удобнее работать с sparse матричками, давайте преобразуем DataFrame в CSR матрицы"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def to_csr(data, rows, cols):\n",
    "    user_item = sp.coo_matrix((data, (rows, cols)))\n",
    "    user_item_t_csr = user_item.T.tocsr()\n",
    "    user_item_csr = user_item.tocsr()\n",
    "    return user_item_csr, user_item_t_csr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Implicit data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "users_imp = implicit_ratings[\"user_id\"]\n",
    "movies_imp = implicit_ratings[\"movie_id\"]\n",
    "user_item_imp, user_item_imp_t = to_csr(np.ones_like(users_imp), users_imp, movies_imp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explicit data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "users_exp = ratings[\"user_id\"]\n",
    "movies_exp = ratings[\"movie_id\"]\n",
    "user_item_exp, user_item_exp_t = to_csr(ratings[\"rating\"] , users_exp, movies_exp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В качестве примера воспользуемся ALS разложением из библиотеки implicit\n",
    "\n",
    "Зададим размерность латентного пространства равным 64, это же определяет размер user/item эмбедингов"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Intel MKL BLAS detected. Its highly recommend to set the environment variable 'export MKL_NUM_THREADS=1' to disable its internal multithreading\n"
     ]
    }
   ],
   "source": [
    "model = implicit.als.AlternatingLeastSquares(factors=64, iterations=100, calculate_training_loss=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В качестве loss здесь всеми любимый RMSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "352b6219a7e945789047891cab23c0bf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "model.fit(user_item_imp_t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Построим похожие фильмы по 1 movie_id = Истории игрушек"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movie_id</th>\n",
       "      <th>name</th>\n",
       "      <th>category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Animation|Children's|Comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children's|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movie_id                                name                      category\n",
       "0         1                    Toy Story (1995)   Animation|Children's|Comedy\n",
       "1         2                      Jumanji (1995)  Adventure|Children's|Fantasy\n",
       "2         3             Grumpier Old Men (1995)                Comedy|Romance\n",
       "3         4            Waiting to Exhale (1995)                  Comedy|Drama\n",
       "4         5  Father of the Bride Part II (1995)                        Comedy"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_info.head(5) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_similars = lambda item_id, model : [movie_info[movie_info[\"movie_id\"] == x[0]][\"name\"].to_string() \n",
    "                                        for x in model.similar_items(item_id)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как мы видим, симилары действительно оказались симиларами.\n",
    "\n",
    "Качество симиларов часто является хорошим способом проверить качество алгоритмов.\n",
    "\n",
    "P.S. Если хочется поглубже разобраться в том как разные алгоритмы формируют разные латентные пространства, рекомендую загружать полученные вектора в tensorBoard и смотреть на сформированное пространство"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['0    Toy Story (1995)',\n",
       " '3045    Toy Story 2 (1999)',\n",
       " \"2286    Bug's Life, A (1998)\",\n",
       " '33    Babe (1995)',\n",
       " '584    Aladdin (1992)',\n",
       " '2315    Babe: Pig in the City (1998)',\n",
       " '360    Lion King, The (1994)',\n",
       " '1526    Hercules (1997)',\n",
       " '1838    Mulan (1998)',\n",
       " '2618    Tarzan (1999)']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_similars(1, model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Давайте теперь построим рекомендации для юзеров\n",
    "\n",
    "Как мы видим юзеру нравится фантастика, значит и в рекомендациях ожидаем увидеть фантастику"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_user_history = lambda user_id, implicit_ratings : [movie_info[movie_info[\"movie_id\"] == x][\"name\"].to_string() \n",
    "                                            for x in implicit_ratings[implicit_ratings[\"user_id\"] == user_id][\"movie_id\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['3399    Hustler, The (1961)',\n",
       " '2882    Fistful of Dollars, A (1964)',\n",
       " '1196    Alien (1979)',\n",
       " '1023    Die Hard (1988)',\n",
       " '257    Star Wars: Episode IV - A New Hope (1977)',\n",
       " '1959    Saving Private Ryan (1998)',\n",
       " '476    Jurassic Park (1993)',\n",
       " '1180    Raiders of the Lost Ark (1981)',\n",
       " '1885    Rocky (1976)',\n",
       " '1081    E.T. the Extra-Terrestrial (1982)',\n",
       " '3349    Thelma & Louise (1991)',\n",
       " '3633    Mad Max (1979)',\n",
       " '2297    King Kong (1933)',\n",
       " '1366    Jaws (1975)',\n",
       " '1183    Good, The Bad and The Ugly, The (1966)',\n",
       " '2623    Run Lola Run (Lola rennt) (1998)',\n",
       " '2878    Goldfinger (1964)',\n",
       " '1220    Terminator, The (1984)']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_user_history(4, implicit_ratings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Получилось! \n",
    "\n",
    "Мы действительно порекомендовали пользователю фантастику и боевики, более того встречаются продолжения тех фильмов, которые он высоко оценил"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_recommendations = lambda user_id, model : [movie_info[movie_info[\"movie_id\"] == x[0]][\"name\"].to_string() \n",
    "                                               for x in model.recommend(user_id, user_item_imp)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['585    Terminator 2: Judgment Day (1991)',\n",
       " '1271    Indiana Jones and the Last Crusade (1989)',\n",
       " '1178    Star Wars: Episode V - The Empire Strikes Back...',\n",
       " '2502    Matrix, The (1999)',\n",
       " '1182    Aliens (1986)',\n",
       " '1284    Butch Cassidy and the Sundance Kid (1969)',\n",
       " '3402    Close Encounters of the Third Kind (1977)',\n",
       " '1884    French Connection, The (1971)',\n",
       " '2880    Dr. No (1962)',\n",
       " '847    Godfather, The (1972)']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_recommendations(4, model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Теперь ваша очередь реализовать самые популярные алгоритмы матричных разложений\n",
    "\n",
    "Что будет оцениваться:\n",
    "1. Корректность алгоритма\n",
    "2. Качество получившихся симиларов\n",
    "3. Качество итоговых рекомендаций для юзера"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Необходимые функции"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rmse(pred, real):\n",
    "    nz = real.nonzero()\n",
    "    pred = pred[nz].flatten()\n",
    "    real = real[nz].flatten()\n",
    "    pred = np.expand_dims(pred, axis=0)\n",
    "    return mean_squared_error(real, pred, squared=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_initial_matrix(latent_size, return_bias=False):\n",
    "    n_users, n_movies = user_item_imp.shape\n",
    "    std = 1 / np.sqrt(latent_size)\n",
    "    user = np.random.normal(scale=std, size=(n_users, latent_size))\n",
    "    item = np.random.normal(scale=std, size=(n_movies, latent_size))\n",
    "    \n",
    "    if return_bias:\n",
    "        b_u = np.random.normal(scale=std, size=(n_users))\n",
    "        b_i = np.random.normal(scale=std,size=(n_movies))\n",
    "        b_glob = np.random.normal(scale=std)\n",
    "        return user, item, b_u, b_i, b_glob\n",
    "        \n",
    "    return user, item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_prediction(user, item):\n",
    "    return user @ item.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def title_by_id(id):\n",
    "    return movie_info[movie_info['movie_id'] == id]['name'].to_string()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_similar_movies(movie_id, pred, k=10):\n",
    "    print(f'For movie {title_by_id(movie_id)} similars are:')\n",
    "    print()\n",
    "    \n",
    "    score = cosine_similarity(np.expand_dims(pred[movie_id], axis=0), pred)[0]\n",
    "    ranked_similars = np.argsort(score)[::-1]\n",
    "    top_k_ids = [s for s in ranked_similars if s != movie_id and s in movie_info['movie_id'].values][:k]\n",
    "    top_k_titles = [title_by_id(r) for r in top_k_ids]\n",
    "    print(*top_k_titles, sep='\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict_for_user(user_id, ratings, ratings_pred):\n",
    "    real_movie_id = ratings[user_id].indices   \n",
    "    print('--- User\\'s choice ---')\n",
    "    real_titles = [title_by_id(r) for r in real_movie_id[:10]]\n",
    "    print(*real_titles, sep='\\n')\n",
    "    print()\n",
    "    \n",
    "    print('--- Our recommendations ---')\n",
    "    user_pred = ratings_pred[user_id]\n",
    "    ranked_movie_id = np.argsort(user_pred)[::-1]\n",
    "    not_rated_movie_id = [i for i in ranked_movie_id if i not in real_movie_id and i in movie_info['movie_id'].values]\n",
    "    pred_titles = [title_by_id(r) for r in not_rated_movie_id[:10]]\n",
    "    print(*pred_titles, sep='\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 1. Не использую готовые решения, реализовать SVD разложение используя SGD на explicit данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def SVD(rating, user, item, b_u, b_i, b_glob, lambd, lr, epochs):\n",
    "    non_zero_idx = list(zip(*rating.nonzero()))\n",
    "\n",
    "    for k in tqdm(range(epochs)):\n",
    "        np.random.shuffle(non_zero_idx)\n",
    "        \n",
    "        for idx in non_zero_idx:\n",
    "            i, j = idx\n",
    "            error = get_prediction(user[i], item[j]) + b_u[i] + b_i[j] + b_glob - rating[i, j] \n",
    "            user[i] -= lr * (error * item[j] + lambd * user[i])\n",
    "            item[j] -= lr * (error * user[i] + lambd * item[j])\n",
    "            b_u[i] -= lr * (error + lambd * b_u[i])\n",
    "            b_i[j] -= lr * (error + lambd * b_i[j])\n",
    "            b_glob -= lr * error\n",
    "        \n",
    "        if k % 5 == 0:\n",
    "            loss = rmse(get_prediction(user, item), rating)\n",
    "            print(f'{k}/{epochs} epoch: loss = {loss}')\n",
    "\n",
    "    return user, item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  2%|▎         | 1/40 [01:05<42:41, 65.67s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0/40 epoch: loss = 3.581099606630683\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 15%|█▌        | 6/40 [06:24<35:30, 62.66s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5/40 epoch: loss = 3.5307313927647246\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 28%|██▊       | 11/40 [11:46<31:22, 64.91s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10/40 epoch: loss = 3.483030313267346\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 40%|████      | 16/40 [18:04<28:41, 71.75s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15/40 epoch: loss = 3.458682062431463\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 52%|█████▎    | 21/40 [26:14<30:50, 97.41s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20/40 epoch: loss = 3.4449466258256813\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 65%|██████▌   | 26/40 [34:14<22:47, 97.68s/it] "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25/40 epoch: loss = 3.4359517683528584\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 78%|███████▊  | 31/40 [39:43<10:51, 72.40s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "30/40 epoch: loss = 3.43051750471871\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 90%|█████████ | 36/40 [44:45<04:07, 61.87s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "35/40 epoch: loss = 3.427188101485979\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 40/40 [49:34<00:00, 74.36s/it]\n"
     ]
    }
   ],
   "source": [
    "W, H, b_u, b_i, b_glob = get_initial_matrix(latent_size=32, return_bias=True)\n",
    "W, H = SVD(user_item_exp, W, H, b_u, b_i, b_glob, lambd=5e-3, lr=1e-2, epochs=40)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### похожие фильмы"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For movie 0    Toy Story (1995) similars are:\n",
      "\n",
      "3045    Toy Story 2 (1999)\n",
      "2286    Bug's Life, A (1998)\n",
      "584    Aladdin (1992)\n",
      "591    Beauty and the Beast (1991)\n",
      "2012    Little Mermaid, The (1989)\n",
      "1838    Mulan (1998)\n",
      "360    Lion King, The (1994)\n",
      "2011    Lady and the Tramp (1955)\n",
      "33    Babe (1995)\n",
      "3682    Chicken Run (2000)\n"
     ]
    }
   ],
   "source": [
    "movie_id = 1\n",
    "get_similar_movies(movie_id, H)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### предсказания для пользователя"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- User's choice ---\n",
      "257    Star Wars: Episode IV - A New Hope (1977)\n",
      "476    Jurassic Park (1993)\n",
      "1023    Die Hard (1988)\n",
      "1081    E.T. the Extra-Terrestrial (1982)\n",
      "1180    Raiders of the Lost Ark (1981)\n",
      "1183    Good, The Bad and The Ugly, The (1966)\n",
      "1196    Alien (1979)\n",
      "1220    Terminator, The (1984)\n",
      "1366    Jaws (1975)\n",
      "1885    Rocky (1976)\n",
      "\n",
      "--- Our recommendations ---\n",
      "2102    Next Stop, Wonderland (1998)\n",
      "2249    Happiness (1998)\n",
      "3673    Battleship Potemkin, The (Bronenosets Potyomki...\n",
      "2506    Dreamlife of Angels, The (La Vie r�v�e des ang...\n",
      "502    Orlando (1993)\n",
      "1553    In the Company of Men (1997)\n",
      "3847    Hellraiser (1987)\n",
      "1745    Hana-bi (1997)\n",
      "377    When a Man Loves a Woman (1994)\n",
      "1507    Schizopolis (1996)\n"
     ]
    }
   ],
   "source": [
    "user_id = 4\n",
    "predict_for_user(user_id, user_item_imp, get_prediction(W, H))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 2. Не использую готовые решения, реализовать матричное разложение используя ALS на implicit данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ALS(rating, user, item, epochs, lambd):\n",
    "    for i in tqdm(range(epochs)):\n",
    "        user = als_step(item, rating, lambd)\n",
    "        item = als_step(user, rating.T, lambd)\n",
    "    \n",
    "        if i % 1000 == 0:\n",
    "            loss = rmse(get_prediction(user, item), rating)\n",
    "            print(f'{i}/{epochs} epoch: loss = {loss}')\n",
    "\n",
    "    return user, item\n",
    "    \n",
    "def als_step(fixed, rating, lambd):\n",
    "    X = fixed.T @ fixed + lambd * np.eye(fixed.shape[1])\n",
    "    X = np.linalg.inv(X)\n",
    "    X = fixed @ X.T\n",
    "    return rating @ X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 5/3000 [00:00<07:17,  6.85it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0/3000 epoch: loss = 0.7568758437446995\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 34%|███▎      | 1007/3000 [00:31<01:14, 26.61it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000/3000 epoch: loss = 0.627284696786587\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 67%|██████▋   | 2007/3000 [01:01<00:36, 26.96it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2000/3000 epoch: loss = 0.6272372994997109\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 3000/3000 [01:32<00:00, 32.55it/s]\n"
     ]
    }
   ],
   "source": [
    "U, V = get_initial_matrix(latent_size=32)\n",
    "U, V = ALS(user_item_imp, U, V, epochs=3000, lambd=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### похожие фильмы"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For movie 0    Toy Story (1995) similars are:\n",
      "\n",
      "3045    Toy Story 2 (1999)\n",
      "33    Babe (1995)\n",
      "584    Aladdin (1992)\n",
      "2286    Bug's Life, A (1998)\n",
      "1245    Groundhog Day (1993)\n",
      "2252    Pleasantville (1998)\n",
      "360    Lion King, The (1994)\n",
      "591    Beauty and the Beast (1991)\n",
      "2315    Babe: Pig in the City (1998)\n",
      "2225    Antz (1998)\n"
     ]
    }
   ],
   "source": [
    "movie_id = 1\n",
    "get_similar_movies(movie_id, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### предсказания для пользователя"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- User's choice ---\n",
      "257    Star Wars: Episode IV - A New Hope (1977)\n",
      "476    Jurassic Park (1993)\n",
      "1023    Die Hard (1988)\n",
      "1081    E.T. the Extra-Terrestrial (1982)\n",
      "1180    Raiders of the Lost Ark (1981)\n",
      "1183    Good, The Bad and The Ugly, The (1966)\n",
      "1196    Alien (1979)\n",
      "1220    Terminator, The (1984)\n",
      "1366    Jaws (1975)\n",
      "1885    Rocky (1976)\n",
      "\n",
      "--- Our recommendations ---\n",
      "1178    Star Wars: Episode V - The Empire Strikes Back...\n",
      "585    Terminator 2: Judgment Day (1991)\n",
      "1271    Indiana Jones and the Last Crusade (1989)\n",
      "1182    Aliens (1986)\n",
      "847    Godfather, The (1972)\n",
      "2502    Matrix, The (1999)\n",
      "1192    Star Wars: Episode VI - Return of the Jedi (1983)\n",
      "1284    Butch Cassidy and the Sundance Kid (1969)\n",
      "453    Fugitive, The (1993)\n",
      "1203    Godfather: Part II, The (1974)\n"
     ]
    }
   ],
   "source": [
    "user_id = 4\n",
    "predict_for_user(user_id, user_item_imp, get_prediction(U, V))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 3. Не использую готовые решения, реализовать матричное разложение BPR на implicit данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_ids = np.unique(ratings['user_id'].values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def pos_neg_dataset(csr_ratings):\n",
    "    movie_ids = movie_info['movie_id'].values\n",
    "    User = namedtuple('User', ['pos', 'neg'])\n",
    "    Ds = dict()\n",
    "    \n",
    "    for u in user_ids:\n",
    "        user_rating = csr_ratings[u]\n",
    "        rated = list(user_rating.indices)\n",
    "        not_rated = list(np.setdiff1d(movie_ids, rated))\n",
    "        user = User(rated, not_rated)\n",
    "        Ds[u] = user\n",
    "    return Ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "Ds = pos_neg_dataset(user_item_imp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "def BPR(D, user, item, iters, lr, lambd):  \n",
    "    for k in tqdm(range(iters)):\n",
    "        np.random.shuffle(user_ids)\n",
    "        \n",
    "        for u in user_ids:\n",
    "            np.random.shuffle(D[u].pos)\n",
    "            \n",
    "            for i in D[u].pos:\n",
    "                for _ in range(5):\n",
    "                    j = np.random.choice(D[u].neg)\n",
    "\n",
    "                    x_uij_hat = get_prediction(user[u], item[i]) - get_prediction(user[u], item[j])\n",
    "                    user[u] += lr * (bpr_step(x_uij_hat, item[i] - item[j]) + lambd * user[u])\n",
    "                    item[i] += lr * (bpr_step(x_uij_hat, user[u]) + lambd * item[i])\n",
    "                    item[j] += lr * (bpr_step(x_uij_hat, -user[u]) + lambd * item[j]) \n",
    "    return user, item\n",
    "\n",
    "def bpr_step(x_uij_hat, partial_d):\n",
    "    exp_x = np.exp(-x_uij_hat)\n",
    "    return exp_x / (1 + exp_x) * partial_d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 5/5 [1:30:36<00:00, 1087.29s/it]\n"
     ]
    }
   ],
   "source": [
    "U, V = get_initial_matrix(latent_size=100)\n",
    "U, V = BPR(Ds, U, V, iters=5, lr=3e-2, lambd=1e-5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### похожие фильмы"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For movie 0    Toy Story (1995) similars are:\n",
      "\n",
      "584    Aladdin (1992)\n",
      "3045    Toy Story 2 (1999)\n",
      "2286    Bug's Life, A (1998)\n",
      "591    Beauty and the Beast (1991)\n",
      "1854    There's Something About Mary (1998)\n",
      "360    Lion King, The (1994)\n",
      "2252    Pleasantville (1998)\n",
      "33    Babe (1995)\n",
      "352    Forrest Gump (1994)\n",
      "1245    Groundhog Day (1993)\n"
     ]
    }
   ],
   "source": [
    "movie_id = 1\n",
    "get_similar_movies(movie_id, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### предсказания для пользователя"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- User's choice ---\n",
      "257    Star Wars: Episode IV - A New Hope (1977)\n",
      "476    Jurassic Park (1993)\n",
      "1023    Die Hard (1988)\n",
      "1081    E.T. the Extra-Terrestrial (1982)\n",
      "1180    Raiders of the Lost Ark (1981)\n",
      "1183    Good, The Bad and The Ugly, The (1966)\n",
      "1196    Alien (1979)\n",
      "1220    Terminator, The (1984)\n",
      "1366    Jaws (1975)\n",
      "1885    Rocky (1976)\n",
      "\n",
      "--- Our recommendations ---\n",
      "1178    Star Wars: Episode V - The Empire Strikes Back...\n",
      "585    Terminator 2: Judgment Day (1991)\n",
      "2502    Matrix, The (1999)\n",
      "537    Blade Runner (1982)\n",
      "1182    Aliens (1986)\n",
      "1192    Star Wars: Episode VI - Return of the Jedi (1983)\n",
      "1353    Star Trek: The Wrath of Khan (1982)\n",
      "1284    Butch Cassidy and the Sundance Kid (1969)\n",
      "847    Godfather, The (1972)\n",
      "1204    Full Metal Jacket (1987)\n"
     ]
    }
   ],
   "source": [
    "user_id = 4\n",
    "predict_for_user(user_id, user_item_imp, get_prediction(U, V))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Задание 4. Не использую готовые решения, реализовать матричное разложение WARP на implicit данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "def WARP(D, user, item, iters, lr):\n",
    "    for k in tqdm(range(iters)):\n",
    "        np.random.shuffle(user_ids)\n",
    "        \n",
    "        for u in user_ids:\n",
    "            np.random.shuffle(D[u].pos)\n",
    "            \n",
    "            for i in D[u].pos:\n",
    "                negatives = D[u].neg\n",
    "                np.random.shuffle(negatives)\n",
    "\n",
    "                pred = get_prediction(user[u], item)\n",
    "                pos_score = pred[i]\n",
    "\n",
    "                for n_neg, j in enumerate(negatives):\n",
    "                    neg_score = pred[j]\n",
    "\n",
    "                    if neg_score > pos_score - 1:\n",
    "                        n_neg += 1\n",
    "                        rank = len(negatives) / n_neg\n",
    "                        user[u] -= lr * np.log(rank) * (-item[i] + item[j])\n",
    "                        item[i] -= lr * np.log(rank) * (-user[u])\n",
    "                        item[j] -= lr * np.log(rank) * (user[u])\n",
    "                        break\n",
    "    return user, item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 20/20 [1:28:46<00:00, 266.34s/it]\n"
     ]
    }
   ],
   "source": [
    "U, V = get_initial_matrix(latent_size=64)\n",
    "U, V = WARP(Ds, U, V, iters=20, lr=1e-3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### похожие фильмы"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For movie 0    Toy Story (1995) similars are:\n",
      "\n",
      "3045    Toy Story 2 (1999)\n",
      "584    Aladdin (1992)\n",
      "2286    Bug's Life, A (1998)\n",
      "1029    That Thing You Do! (1996)\n",
      "2692    Iron Giant, The (1999)\n",
      "2012    Little Mermaid, The (1989)\n",
      "1526    Hercules (1997)\n",
      "2315    Babe: Pig in the City (1998)\n",
      "1838    Mulan (1998)\n",
      "510    Ref, The (1994)\n"
     ]
    }
   ],
   "source": [
    "movie_id = 1\n",
    "get_similar_movies(movie_id, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### предсказания для пользователя"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- User's choice ---\n",
      "257    Star Wars: Episode IV - A New Hope (1977)\n",
      "476    Jurassic Park (1993)\n",
      "1023    Die Hard (1988)\n",
      "1081    E.T. the Extra-Terrestrial (1982)\n",
      "1180    Raiders of the Lost Ark (1981)\n",
      "1183    Good, The Bad and The Ugly, The (1966)\n",
      "1196    Alien (1979)\n",
      "1220    Terminator, The (1984)\n",
      "1366    Jaws (1975)\n",
      "1885    Rocky (1976)\n",
      "\n",
      "--- Our recommendations ---\n",
      "3458    Predator (1987)\n",
      "108    Braveheart (1995)\n",
      "1178    Star Wars: Episode V - The Empire Strikes Back...\n",
      "2502    Matrix, The (1999)\n",
      "1539    Men in Black (1997)\n",
      "847    Godfather, The (1972)\n",
      "1271    Indiana Jones and the Last Crusade (1989)\n",
      "957    African Queen, The (1951)\n",
      "2460    Planet of the Apes (1968)\n",
      "585    Terminator 2: Judgment Day (1991)\n"
     ]
    }
   ],
   "source": [
    "user_id = 4\n",
    "predict_for_user(user_id, user_item_imp, get_prediction(U, V))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}