政安晨：【Keras机器学习示例演绎】（五十七）—— 基于Transformer的推荐系统

目录

介绍

数据集

设置

准备数据

将电影评分数据转换为序列

定义元数据

创建用于训练和评估的 tf.data.Dataset

创建模型输入

输入特征编码

创建 BST 模型

开展培训和评估实验

---

政安晨的个人主页：政安晨

欢迎 👍点赞✍评论⭐收藏

希望政安晨的博客能够对您有所裨益，如有不足之处，欢迎在评论区提出指正！

本文目标：在 Movielens 上使用行为序列转换器（BST）模型预测评级率。

介绍

本示例使用 Movielens 数据集演示了陈启伟等人的行为序列转换器（BST）模型。 BST 模型利用用户观看电影和给电影评分的顺序行为，以及用户资料和电影特征，来预测用户对目标电影的评分。

更确切地说，BST 模型旨在通过接受以下输入来预测目标电影的评分：

1. 用户观看过的电影的固定长度序列。
2. 用户观看过的电影评分的固定长度序列。
3. 输入序列中每部电影和目标电影的类型集。
4. 输入序列中每部电影和目标电影的类型集。
5. 要预测评分的 target_movie_id。

该示例以下列方式修改了原始 BST 模型：

1. 我们在处理输入序列中的每部电影和目标电影的嵌入过程中都加入了电影特征（流派），而不是将其视为转换层之外的 "其他特征"。

2. 我们利用输入序列中电影的评分以及它们在序列中的位置来更新它们，然后再将它们输入自我关注层。

（请注意，本示例应在 TensorFlow 2.4 或更高版本中运行。）

数据集

我们使用的是 Movielens 数据集的 1M 版本。 该数据集包含 6000 名用户对 4000 部电影的约 100 万个评分，以及一些用户特征和电影类型。 此外，数据集还提供了每个用户对电影评分的时间戳，这样就可以按照 BST 模型的预期，为每个用户创建电影评分序列。

设置

import osos.environ["KERAS_BACKEND"] = "tensorflow"import math
from zipfile import ZipFile
from urllib.request import urlretrieveimport keras
import numpy as np
import pandas as pd
import tensorflow as tf
from keras import layers
from keras.layers import StringLookup

准备数据

下载并准备数据框

首先，让我们下载 movielens 数据。

下载的文件夹将包含三个数据文件：users.dat、movies.dat 和 ratings.dat。

urlretrieve("http://files.grouplens.org/datasets/movielens/ml-1m.zip", "movielens.zip")
ZipFile("movielens.zip", "r").extractall()

然后，我们用正确的列名将数据加载到 pandas DataFrames 中。

users = pd.read_csv("ml-1m/users.dat",sep="::",names=["user_id", "sex", "age_group", "occupation", "zip_code"],encoding="ISO-8859-1",engine="python",
)ratings = pd.read_csv("ml-1m/ratings.dat",sep="::",names=["user_id", "movie_id", "rating", "unix_timestamp"],encoding="ISO-8859-1",engine="python",
)movies = pd.read_csv("ml-1m/movies.dat",sep="::",names=["movie_id", "title", "genres"],encoding="ISO-8859-1",engine="python",
)

在此，我们进行一些简单的数据处理，以固定列的数据类型。

users["user_id"] = users["user_id"].apply(lambda x: f"user_{x}")
users["age_group"] = users["age_group"].apply(lambda x: f"group_{x}")
users["occupation"] = users["occupation"].apply(lambda x: f"occupation_{x}")movies["movie_id"] = movies["movie_id"].apply(lambda x: f"movie_{x}")ratings["movie_id"] = ratings["movie_id"].apply(lambda x: f"movie_{x}")
ratings["user_id"] = ratings["user_id"].apply(lambda x: f"user_{x}")
ratings["rating"] = ratings["rating"].apply(lambda x: float(x))

每部电影都有多种类型。 我们将它们分成电影 DataFrame 中的不同列。

genres = ["Action", "Adventure", "Animation", "Children's", "Comedy", "Crime"]
genres += ["Documentary", "Drama", "Fantasy", "Film-Noir", "Horror", "Musical"]
genres += ["Mystery", "Romance", "Sci-Fi", "Thriller", "War", "Western"]for genre in genres:movies[genre] = movies["genres"].apply(lambda values: int(genre in values.split("|")))

将电影评分数据转换为序列

首先，我们使用 unix_timestamp 对评分数据进行排序，然后按用户 ID 对电影 ID 值和评分值进行分组。

ratings_group = ratings.sort_values(by=["unix_timestamp"]).groupby("user_id")ratings_data = pd.DataFrame(data={"user_id": list(ratings_group.groups.keys()),"movie_ids": list(ratings_group.movie_id.apply(list)),"ratings": list(ratings_group.rating.apply(list)),"timestamps": list(ratings_group.unix_timestamp.apply(list)),}
)

现在，让我们把 movie_ids 列表拆分成一组固定长度的序列。 我们对评分也做同样的处理。 设置 sequence_length 变量可改变模型输入序列的长度。 您还可以改变 step_size 来控制为每个用户生成的序列数量。

sequence_length = 4
step_size = 2def create_sequences(values, window_size, step_size):sequences = []start_index = 0while True:end_index = start_index + window_sizeseq = values[start_index:end_index]if len(seq) < window_size:seq = values[-window_size:]if len(seq) == window_size:sequences.append(seq)breaksequences.append(seq)start_index += step_sizereturn sequencesratings_data.movie_ids = ratings_data.movie_ids.apply(lambda ids: create_sequences(ids, sequence_length, step_size)
)ratings_data.ratings = ratings_data.ratings.apply(lambda ids: create_sequences(ids, sequence_length, step_size)
)del ratings_data["timestamps"]

然后，我们对输出进行处理，使每个序列在 DataFrame 中都有单独的记录。 此外，我们还将用户特征与评分数据结合起来。

ratings_data_movies = ratings_data[["user_id", "movie_ids"]].explode("movie_ids", ignore_index=True
)
ratings_data_rating = ratings_data[["ratings"]].explode("ratings", ignore_index=True)
ratings_data_transformed = pd.concat([ratings_data_movies, ratings_data_rating], axis=1)
ratings_data_transformed = ratings_data_transformed.join(users.set_index("user_id"), on="user_id"
)
ratings_data_transformed.movie_ids = ratings_data_transformed.movie_ids.apply(lambda x: ",".join(x)
)
ratings_data_transformed.ratings = ratings_data_transformed.ratings.apply(lambda x: ",".join([str(v) for v in x])
)del ratings_data_transformed["zip_code"]ratings_data_transformed.rename(columns={"movie_ids": "sequence_movie_ids", "ratings": "sequence_ratings"},inplace=True,
)

在 sequence_length 为 4、step_size 为 2 的情况下，我们最终得到了 498 623 个序列。 最后，我们将数据分成训练和测试两个部分，分别包含 85% 和 15% 的实例，并将它们存储到 CSV 文件中。

random_selection = np.random.rand(len(ratings_data_transformed.index)) <= 0.85
train_data = ratings_data_transformed[random_selection]
test_data = ratings_data_transformed[~random_selection]train_data.to_csv("train_data.csv", index=False, sep="|", header=False)
test_data.to_csv("test_data.csv", index=False, sep="|", header=False)

定义元数据

CSV_HEADER = list(ratings_data_transformed.columns)CATEGORICAL_FEATURES_WITH_VOCABULARY = {"user_id": list(users.user_id.unique()),"movie_id": list(movies.movie_id.unique()),"sex": list(users.sex.unique()),"age_group": list(users.age_group.unique()),"occupation": list(users.occupation.unique()),
}USER_FEATURES = ["sex", "age_group", "occupation"]MOVIE_FEATURES = ["genres"]

创建用于训练和评估的 tf.data.Dataset

def get_dataset_from_csv(csv_file_path, shuffle=False, batch_size=128):def process(features):movie_ids_string = features["sequence_movie_ids"]sequence_movie_ids = tf.strings.split(movie_ids_string, ",").to_tensor()# The last movie id in the sequence is the target movie.features["target_movie_id"] = sequence_movie_ids[:, -1]features["sequence_movie_ids"] = sequence_movie_ids[:, :-1]ratings_string = features["sequence_ratings"]sequence_ratings = tf.strings.to_number(tf.strings.split(ratings_string, ","), tf.dtypes.float32).to_tensor()# The last rating in the sequence is the target for the model to predict.target = sequence_ratings[:, -1]features["sequence_ratings"] = sequence_ratings[:, :-1]return features, targetdataset = tf.data.experimental.make_csv_dataset(csv_file_path,batch_size=batch_size,column_names=CSV_HEADER,num_epochs=1,header=False,field_delim="|",shuffle=shuffle,).map(process)return dataset

创建模型输入

def create_model_inputs():return {"user_id": keras.Input(name="user_id", shape=(1,), dtype="string"),"sequence_movie_ids": keras.Input(name="sequence_movie_ids", shape=(sequence_length - 1,), dtype="string"),"target_movie_id": keras.Input(name="target_movie_id", shape=(1,), dtype="string"),"sequence_ratings": keras.Input(name="sequence_ratings", shape=(sequence_length - 1,), dtype=tf.float32),"sex": keras.Input(name="sex", shape=(1,), dtype="string"),"age_group": keras.Input(name="age_group", shape=(1,), dtype="string"),"occupation": keras.Input(name="occupation", shape=(1,), dtype="string"),}

输入特征编码

输入特征编码方法的工作原理如下：

每个分类用户特征都使用层嵌入（layer.Embedding）编码，嵌入维度等于特征词汇量的平方根。

电影序列中的每部电影和目标电影都使用层.嵌入编码，嵌入维度等于电影数量的平方根。

每部电影的多热点流派向量与其嵌入向量连接，并使用非线性层.密集处理，以输出具有相同电影嵌入维度的向量。
将位置嵌入添加到序列中的每个电影嵌入中，然后乘以评分序列中的评分。

将目标电影嵌入与序列电影嵌入连接起来，产生一个张量，其形状为[批量大小、序列长度、嵌入大小]，正如转换器架构的注意层所预期的那样。

该方法返回一个由两个元素组成的元组：编码转换器特征和编码其他特征。

def encode_input_features(inputs,include_user_id=True,include_user_features=True,include_movie_features=True,
):encoded_transformer_features = []encoded_other_features = []other_feature_names = []if include_user_id:other_feature_names.append("user_id")if include_user_features:other_feature_names.extend(USER_FEATURES)## Encode user featuresfor feature_name in other_feature_names:# Convert the string input values into integer indices.vocabulary = CATEGORICAL_FEATURES_WITH_VOCABULARY[feature_name]idx = StringLookup(vocabulary=vocabulary, mask_token=None, num_oov_indices=0)(inputs[feature_name])# Compute embedding dimensionsembedding_dims = int(math.sqrt(len(vocabulary)))# Create an embedding layer with the specified dimensions.embedding_encoder = layers.Embedding(input_dim=len(vocabulary),output_dim=embedding_dims,name=f"{feature_name}_embedding",)# Convert the index values to embedding representations.encoded_other_features.append(embedding_encoder(idx))## Create a single embedding vector for the user featuresif len(encoded_other_features) > 1:encoded_other_features = layers.concatenate(encoded_other_features)elif len(encoded_other_features) == 1:encoded_other_features = encoded_other_features[0]else:encoded_other_features = None## Create a movie embedding encodermovie_vocabulary = CATEGORICAL_FEATURES_WITH_VOCABULARY["movie_id"]movie_embedding_dims = int(math.sqrt(len(movie_vocabulary)))# Create a lookup to convert string values to integer indices.movie_index_lookup = StringLookup(vocabulary=movie_vocabulary,mask_token=None,num_oov_indices=0,name="movie_index_lookup",)# Create an embedding layer with the specified dimensions.movie_embedding_encoder = layers.Embedding(input_dim=len(movie_vocabulary),output_dim=movie_embedding_dims,name=f"movie_embedding",)# Create a vector lookup for movie genres.genre_vectors = movies[genres].to_numpy()movie_genres_lookup = layers.Embedding(input_dim=genre_vectors.shape[0],output_dim=genre_vectors.shape[1],embeddings_initializer=keras.initializers.Constant(genre_vectors),trainable=False,name="genres_vector",)# Create a processing layer for genres.movie_embedding_processor = layers.Dense(units=movie_embedding_dims,activation="relu",name="process_movie_embedding_with_genres",)## Define a function to encode a given movie id.def encode_movie(movie_id):# Convert the string input values into integer indices.movie_idx = movie_index_lookup(movie_id)movie_embedding = movie_embedding_encoder(movie_idx)encoded_movie = movie_embeddingif include_movie_features:movie_genres_vector = movie_genres_lookup(movie_idx)encoded_movie = movie_embedding_processor(layers.concatenate([movie_embedding, movie_genres_vector]))return encoded_movie## Encoding target_movie_idtarget_movie_id = inputs["target_movie_id"]encoded_target_movie = encode_movie(target_movie_id)## Encoding sequence movie_ids.sequence_movies_ids = inputs["sequence_movie_ids"]encoded_sequence_movies = encode_movie(sequence_movies_ids)# Create positional embedding.position_embedding_encoder = layers.Embedding(input_dim=sequence_length,output_dim=movie_embedding_dims,name="position_embedding",)positions = tf.range(start=0, limit=sequence_length - 1, delta=1)encodded_positions = position_embedding_encoder(positions)# Retrieve sequence ratings to incorporate them into the encoding of the movie.sequence_ratings = inputs["sequence_ratings"]sequence_ratings = keras.ops.expand_dims(sequence_ratings, -1)# Add the positional encoding to the movie encodings and multiply them by rating.encoded_sequence_movies_with_poistion_and_rating = layers.Multiply()([(encoded_sequence_movies + encodded_positions), sequence_ratings])# Construct the transformer inputs.for i in range(sequence_length - 1):feature = encoded_sequence_movies_with_poistion_and_rating[:, i, ...]feature = keras.ops.expand_dims(feature, 1)encoded_transformer_features.append(feature)encoded_transformer_features.append(encoded_target_movie)encoded_transformer_features = layers.concatenate(encoded_transformer_features, axis=1)return encoded_transformer_features, encoded_other_features

创建 BST 模型

include_user_id = False
include_user_features = False
include_movie_features = Falsehidden_units = [256, 128]
dropout_rate = 0.1
num_heads = 3def create_model():inputs = create_model_inputs()transformer_features, other_features = encode_input_features(inputs, include_user_id, include_user_features, include_movie_features)# Create a multi-headed attention layer.attention_output = layers.MultiHeadAttention(num_heads=num_heads, key_dim=transformer_features.shape[2], dropout=dropout_rate)(transformer_features, transformer_features)# Transformer block.attention_output = layers.Dropout(dropout_rate)(attention_output)x1 = layers.Add()([transformer_features, attention_output])x1 = layers.LayerNormalization()(x1)x2 = layers.LeakyReLU()(x1)x2 = layers.Dense(units=x2.shape[-1])(x2)x2 = layers.Dropout(dropout_rate)(x2)transformer_features = layers.Add()([x1, x2])transformer_features = layers.LayerNormalization()(transformer_features)features = layers.Flatten()(transformer_features)# Included the other features.if other_features is not None:features = layers.concatenate([features, layers.Reshape([other_features.shape[-1]])(other_features)])# Fully-connected layers.for num_units in hidden_units:features = layers.Dense(num_units)(features)features = layers.BatchNormalization()(features)features = layers.LeakyReLU()(features)features = layers.Dropout(dropout_rate)(features)outputs = layers.Dense(units=1)(features)model = keras.Model(inputs=inputs, outputs=outputs)return modelmodel = create_model()

开展培训和评估实验

# Compile the model.
model.compile(optimizer=keras.optimizers.Adagrad(learning_rate=0.01),loss=keras.losses.MeanSquaredError(),metrics=[keras.metrics.MeanAbsoluteError()],
)# Read the training data.
train_dataset = get_dataset_from_csv("train_data.csv", shuffle=True, batch_size=265)# Fit the model with the training data.
model.fit(train_dataset, epochs=5)# Read the test data.
test_dataset = get_dataset_from_csv("test_data.csv", batch_size=265)# Evaluate the model on the test data.
_, rmse = model.evaluate(test_dataset, verbose=0)
print(f"Test MAE: {round(rmse, 3)}")

Epoch 1/51600/1600 ━━━━━━━━━━━━━━━━━━━━ 19s 11ms/step - loss: 1.5762 - mean_absolute_error: 0.9892
Epoch 2/51600/1600 ━━━━━━━━━━━━━━━━━━━━ 17s 11ms/step - loss: 1.1263 - mean_absolute_error: 0.8502
Epoch 3/51600/1600 ━━━━━━━━━━━━━━━━━━━━ 17s 11ms/step - loss: 1.0885 - mean_absolute_error: 0.8361
Epoch 4/51600/1600 ━━━━━━━━━━━━━━━━━━━━ 17s 11ms/step - loss: 1.0943 - mean_absolute_error: 0.8388
Epoch 5/51600/1600 ━━━━━━━━━━━━━━━━━━━━ 17s 10ms/step - loss: 1.0360 - mean_absolute_error: 0.8142
Test MAE: 0.782

测试数据的平均绝对误差 (MAE) 应该在 0.7 左右。

---