Recommendation Systems

What You Should Understand

• 能比較 collaborative filtering、content-based、hybrid 的適用場景
• 理解 matrix factorization 的數學和 implicit feedback 的差異
• 知道 cold start 問題的具體解法
• 能設計 offline + online evaluation 策略
• 理解 feedback loop、position bias、popularity bias 等 production 問題

Overview

The Three Paradigms

Collaborative Filtering

User-Based CF

Find users similar to you, recommend items they liked:

1. Compute similarity between users（based on their rating patterns）
2. Find top-$k$ most similar users to the target user
3. Recommend items those similar users rated highly but the target user hasn't seen

Item-Based CF

Find items similar to items the user already liked:

1. Compute similarity between items（based on who rated them similarly）
2. For each item the user liked, find similar items
3. Recommend the most similar items the user hasn't seen

User-Based vs Item-Based

Item-Based CF 在實務中更常見 — item similarity 更穩定（電影的特性不會變，但使用者的偏好會），計算量更小（items 通常比 users 少），且 similarity matrix 可以 pre-compute。

Similarity Metrics

Cosine Similarity: $\cos(\mathbf{u}, \mathbf{v})$ — normalized dot product, range [-1, 1]。不受 rating scale 影響。

Pearson Correlation: Mean-centered cosine — 減去每個 user 的 mean rating → 消除 rating bias（有些人都給高分，有些人嚴格）。

Jaccard Similarity: $J(A,B) =$ intersection / union — 只看有沒有互動，不看 rating value。適合 binary implicit feedback。

Content-Based Filtering

Recommend items similar to what the user has liked, based on item features:

Item Features

User Profile

Aggregate features of items the user has interacted with:

或用 weighted average（rating 高的 item 權重大）。

Content-Based vs CF

Matrix Factorization

The Idea

Decompose the sparse user-item rating matrix $\mathbf{R}$ ($m \times n$) into two low-rank matrices:

where $\mathbf{P}$ ($m \times k$) = user latent factors, $\mathbf{Q}$ ($n \times k$) = item latent factors, $k \ll \min(m, n)$.

Predicted rating:

每個 latent factor 代表一個 hidden dimension（例如 factor 1 = 「動作程度」, factor 2 = 「深度」）— 但 factors 通常不 directly interpretable。

SVD-based Methods

Vanilla SVD: $\mathbf{R} = \mathbf{U} \boldsymbol{\Sigma} \mathbf{V}^\top$

問題：$\mathbf{R}$ 有大量 missing values → standard SVD 不能直接用在 incomplete matrix。

Regularized SVD (Funk SVD): Only minimize loss on observed ratings:

用 SGD 或 ALS（Alternating Least Squares）optimize。

Adding Biases

Users and items have inherent biases（某些 user 都打高分、某些 item 本身就好）：

• $\mu$: global average rating
• $b_u$: user bias（相對於平均）
• $b_i$: item bias

ALS (Alternating Least Squares)

ALS algorithm:

1. Fix $\mathbf{Q}$, solve for $\mathbf{P}$（closed-form linear regression per user）
2. Fix $\mathbf{P}$, solve for $\mathbf{Q}$（closed-form per item）
3. Repeat until convergence

ALS 的優勢：每步都是 closed-form → 可以高度 parallelize（Spark MLlib 用 ALS）。SGD 是 sequential，不好 parallelize。

Implicit Feedback

大部分推薦場景沒有 explicit ratings — 只有 implicit signals：clicks, views, purchases, time spent.

Key differences from explicit feedback:

Weighted ALS for implicit feedback (Hu et al., 2008):

• $p_{ui} = 1$ if user interacted with item, 0 otherwise（binary preference）
• $c_{ui} = 1 + \alpha \cdot f(r_{ui})$（confidence — 互動越多 confidence 越高）

from implicit.als import AlternatingLeastSquares

# CSR matrix of user-item interactions (implicit)
model = AlternatingLeastSquares(factors=50, regularization=0.1, iterations=20)
model.fit(user_item_matrix)

# Recommend for user 42
ids, scores = model.recommend(42, user_item_matrix[42], N=10)

Deep Learning for Recommendations

Two-Tower Model

Modern large-scale recommendation systems 常用 two-tower architecture:

User tower: Encode user features → user embedding $\mathbf{u}$

Item tower: Encode item features → item embedding $\mathbf{v}$

兩個 tower 分開訓練 → item embeddings 可以 pre-compute 和 index → 用 ANN（Approximate Nearest Neighbor）做 millisecond-level retrieval。

Multi-Stage Architecture

Production 推薦系統通常是 multi-stage pipeline：

Cold Start Problem

Types

Solutions in Practice

For new users:

• Show popular / trending items（non-personalized → safe baseline）
• Onboarding：ask for preferences or import from other platforms
• 用 demographic features 做 content-based（age, location → similar users' preferences）
• Contextual bandits：explore user preferences while serving recommendations

For new items:

• Inject new items into small % of recommendations（exploration）
• Use item features + content-based similarity to existing items
• Editorial / curated placement
• Time-based boost（new items get temporary ranking lift）

Evaluation

Offline Metrics

Offline vs Online Gap

Offline metrics 好不代表 online metrics 好：

Online metrics（A/B test）：CTR, conversion rate, revenue per session, time spent, user retention.

Production Challenges

Popularity Bias

Model 傾向推薦已經熱門的 items（因為 training data 中熱門 items 有更多 interactions）→ 越推越熱門 → 長尾 items 永遠沒機會。

Solutions: Inverse propensity scoring, exploration budget, diversity penalty in re-ranking.

Position Bias

Users click items partly because they're ranked higher, not because they're more relevant.

Training data 中高位置的 items 得到更多 clicks → model 學到「高位置 = good」→ 自我強化。

Solutions: Inverse propensity weighting on position, random shuffle experiments to collect unbiased data.

Feedback Loop

Model predictions → affect what users see → affect user behavior → affect training data → affect model → ...

這個循環讓 model 的 mistakes 自我強化。

Solutions: Regular exploration（隨機推薦一小比例 items）、counterfactual evaluation、定期用 unbiased data retrain。

Real-World Use Cases

Case 1: 電商推薦（E-commerce）

Case 2: 影音串流（Video/Music）

Case 3: 社群 Feed（Social Media）

Hands-on: Recommendation in Python

Item-Based CF with Cosine Similarity

from sklearn.metrics.pairwise import cosine_similarity
import numpy as np

# user_item_matrix: rows=users, cols=items, values=ratings
item_sim = cosine_similarity(user_item_matrix.T)  # items × items

def recommend(user_id, user_item_matrix, item_sim, top_k=10):
    user_ratings = user_item_matrix[user_id]
    scores = item_sim.dot(user_ratings)  # weighted sum of similarities
    # Zero out already-rated items
    scores[user_ratings > 0] = 0
    return np.argsort(scores)[::-1][:top_k]

Matrix Factorization with ALS

from implicit.als import AlternatingLeastSquares
from scipy.sparse import csr_matrix

# Implicit feedback: user_item_matrix (sparse, values = interaction counts)
sparse_matrix = csr_matrix(user_item_matrix)

model = AlternatingLeastSquares(
    factors=64,
    regularization=0.05,
    iterations=30,
)
model.fit(sparse_matrix)

# Recommend for user 0
recommended_ids, scores = model.recommend(0, sparse_matrix[0], N=10)

# Similar items to item 42
similar_ids, scores = model.similar_items(42, N=5)

Two-Tower with PyTorch

import torch
import torch.nn as nn

class TwoTowerModel(nn.Module):
    def __init__(self, n_users, n_items, embed_dim=64):
        super().__init__()
        self.user_tower = nn.Sequential(
            nn.Embedding(n_users, embed_dim),
            nn.Linear(embed_dim, embed_dim),
            nn.ReLU(),
            nn.Linear(embed_dim, embed_dim),
        )
        self.item_tower = nn.Sequential(
            nn.Embedding(n_items, embed_dim),
            nn.Linear(embed_dim, embed_dim),
            nn.ReLU(),
            nn.Linear(embed_dim, embed_dim),
        )

    def forward(self, user_ids, item_ids):
        user_emb = self.user_tower(user_ids)    # [B, embed_dim]
        item_emb = self.item_tower(item_ids)    # [B, embed_dim]
        return (user_emb * item_emb).sum(dim=1) # dot product → score

Interview Signals

What interviewers listen for:

• 你能描述 multi-stage pipeline（candidate gen → ranking → re-ranking）
• 你知道 CF 和 content-based 的 tradeoffs，以及各自的 cold start 問題
• 你能解釋 implicit vs explicit feedback 的差異和處理方式
• 你知道 offline NDCG ≠ online CTR，以及 feedback loop / position bias 等 production 問題
• 你能結合 business context 設計 evaluation 策略

Practice

Flashcards

Quiz