WWW2018-Neural Attentional Rating Regression with Review-level Explanations

摘要

Introduction

评分预测任务常出现在推荐系统中。大多数推荐方法都基于协同过滤技术（如PMF等算法），这类算法的一个主要问题是缺乏对推荐结果的解释能力。研究表明，在推荐系统中提供可解释性的功能具有重要意义，并且这一特性往往能够显著提升用户体验和系统的可信度。然而，在缺乏这种特性的情况下，即使是最有效的推荐系统也无法让用户完全信任其决策依据。许多电子商务平台上用户不仅能够对商品进行评价（打分），而且这些评价信息通常会包含与产品质量相关的特征信息（如材质、质量、颜色等）。本文研究关注的是评价的有效性（即用户的评分能否帮助其决定是否购买）。如图所示，则是展示有效评论与无效评论的例子。

已有研究将用户评论信息融合到隐含因子模型中以提升其性能[3, 25–27, 39, 46] ，或是生成推荐并提供相应的解释[11, 32, 44] 。尽管取得了一定程度上的成功 ，但目前仍存在两个主要问题：首先 ，未能建立评论对被推荐项贡献度的明确模型 ，并且忽略了其他用户的使用效果；其次 ，以往研究中的解释方法通常是对词或短语进行提取 ，这可能导致评论内容出现偏差 。本文开创性地将评论的有效性纳入考量 ，并显著提升了推荐效果与可解释性 。

本文提出了一种名为Neural Attentional Regression model with Review-level Explanations (NARRE)的新模型架构。该模型通过引入注意力机制为每个评论赋予相应的权重系数。具体而言,该模型以用户信息、推荐物品特征以及评论文本作为多层神经网络的输入层,并设计了一个权重计算公式来生成每个评论的重要性评分。同时,该研究参考了现有文献[46]的方法论,采用并行的神经网络结构来提取用户与推荐物品之间的隐式因子特征:一方面,一个神经网络用于建模用户在撰写评论时的行为倾向;另一方面,另一个神经网络则专注于分析评论文本对推荐物品兴趣度的影响机制。最终,通过将隐式因子模型[21]扩展为全连接神经网络结构,该研究实现了评分预测任务的目标函数输出层结构设计。实验部分,作者在四个典型真实数据集上进行了对比实验测试:结果表明所提出的NARRE模型在预测性能方面显著优于现有领先方法如PMF[29]、NMF[24]、SVD++[20]、HFT[27]以及DeepCoNN[46]等

Related Work

近年来相关研究主要通过融合神经网络与协同过滤框架进行探索。例如（He等, 2013）提出了一种名为NCF的框架结构,该框架成功地建模了用户行为之间的复杂非线性关系,从而推动了后续研究的发展方向

Methodology

• Latent Factor Model is a method used to model any user u's rating for item i within the context of latent factor models. The relationship between users and items can be represented using the following formula.

基于卷积神经网络的文本处理器：对于一个长度为T的输入文本序列来说，在其嵌入表示矩阵中包含V1到VT这些元素的情况下，请问每个神经元如何提取其特定特征？这里用符号*代表卷积运算

，

表示第j个神经元在时间窗口t中的滑动结果，则是该神经元对应的输出结果。接着最终属于该神经元的特征将通过最大池化操作获得输出结果。而最大池化的意义在于筛选出最具有代表性的特征，并非简单地取其数值大小最高的那个特征？

。最后CNN的输出就是由其m个神经元的输出拼接出来的

。最后，输出O会输入到一个全连接层，从而得到最终结果

本文模型 NARRE Model： 该模型由两个并行的神经网络同时存在，在它们的上层结构中增加了用于用户与推荐项目的隐含因子交互以预测最终评分结果的一个聚合操作。在训练过程中，输入包括用户、推荐项目以及评论文本，在测试阶段，则将用户和推荐项目作为输入进行处理。具体而言，在处理每个评论时，每个词通过 embedding 转换为相应的向量表示后放入卷积神经网络（CNN）中的池化层进行处理得到输出向量Oi1, Oi2,...Oij. 在获得 item 的表征时，默认情况下会对这些输出向量进行平均聚合以获取最终表示这一过程也是较为常见的方法之一。而本文提出的是一种基于注意力机制的方法旨在学习不同输出向量对应的权重系数。

为了通过计算attention score这一指标来建模用户的评论有效性与行为模式,该方法采用了两个层次的人工神经网络架构,其中输入数据由每个条目对应的第l条评论及其生成特征向量(O il)构成,同时结合了评论作者的身份信息嵌入表示。这种身份信息嵌入的作用在于识别出那些经常发布低质量评论的行为模式,从而帮助模型更准确地评估用户的评论质量与价值。该attention网络的具体定义如下

，其中

最后，通过softmax进行归一化，得到

将ail作为最终的权重进行加权平均

最后通过全连接层得到item的最终表示

同理可以用同样的方法对Net_u进行建模，然后通过点乘将两部分结合，

，最终的评分计算公式为

。

模型的训练过程与传统推荐任务相同，其损失函数为

至此，本文的模型NARRE介绍完毕。

小结

该篇文章在方法上表现平平，在创新点上主要体现在巧妙地应用了Rate_Usefulness 概念的基础上进行了一些改进。其创新点在于巧妙地应用了这一概念的同时也在数据处理过程中充分利用了评论的有效性这一特点。这一发现非常有趣的是如果不考虑评论有效性的这一关键因素，则该文章与多数情感分析评分预测模型思路基本相似

参考文献

REFERENCES
[1] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint
arXiv:1409.0473 (2014).
[2] David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation.
Journal of machine Learning research 3, Jan (2003), 993–1022.
[3] Rose Catherine and William Cohen. 2017. TransNets: Learning to Transform for
Recommendation. arXiv preprint arXiv:1704.02298 (2017).
[4] Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and TatSeng Chua. 2017. Attentive collaborative ltering: Multimedia recommendation
with item-and component-level attention. In SIGIR. 335–344.
[5] Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, and Tat-Seng
Chua. 2016. SCA-CNN: Spatial and Channel-wise Attention in Convolutional
Networks for Image Captioning. arXiv preprint arXiv:1611.05594 (2016).
[6] J Cohen. 1968. Weighted kappa: nominal scale agreement with provision for
scaled disagreement or partial credit. Psychological Bulletin 70, 4 (1968), 213.
[7] Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu,
and Pavel Kuksa. 2011. Natural language processing (almost) from scratch.
Journal of Machine Learning Research 12, Aug (2011), 2493–2537.
[8] Qiming Diao, Minghui Qiu, Chao-Yuan Wu, Alexander J Smola, Jing Jiang, and
Chong Wang. 2014. Jointly modeling aspects, ratings and sentiments for movie
recommendation (jmars). In SIGKDD. 193–202.
[9] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep learning. MIT
press.
[10] Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual
evolution of fashion trends with one-class collaborative ltering. In WWW. 507–
517.
[11] Xiangnan He, Tao Chen, Min-Yen Kan, and Xiao Chen. 2015. Trirank: Reviewaware explainable recommendation by modeling aspects. In CIKM. 1661–1670.
[12] Xiangnan He and Tat-Seng Chua. 2017. Neural Factorization Machines for Sparse
Predictive Analytics. (2017).
[13] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng
Chua. 2017. Neural collaborative ltering. In WWW. 173–182.
[14] G Hinton, N Srivastava, and K Swersky. 2012. RMSProp: Divide the gradient by
a running average of its recent magnitude. Neural networks for machine learning,
Coursera lecture 6e (2012).
[15] Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag
of tricks for ecient text classication. arXiv preprint arXiv:1607.01759 (2016).
[16] Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A Convolutional
Neural Network for Modelling Sentences. Eprint Arxiv 1 (2014).
[17] Soo Min Kim, Patrick Pantel, Tim Chklovski, and Marco Pennacchiotti. 2006.
Automatically assessing review helpfulness. In EMNLP. 423–430.
[18] Yoon Kim. 2014. Convolutional neural networks for sentence classication. arXiv
preprint arXiv:1408.5882 (2014).
[19] D Kinga and J Ba Adam. 2015. A method for stochastic optimization. In ICLR.
[20] Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted
collaborative ltering model. In SIGKDD. 426–434.
[21] Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009).
[22] J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement
for categorical data. biometrics (1977), 159–174.
[23] Quoc V. Le and Tomas Mikolov. 2014. Distributed Representations of Sentences
and Documents. 4 (2014), II–1188.
[24] Daniel D Lee and H Sebastian Seung. 2001. Algorithms for non-negative matrix
factorization. In Advances in neural information processing systems. 556–562.
[25] Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and Wai Lam. 2017. Neural
Rating Regression with Abstractive Tips Generation for Recommendation. (2017).
[26] Guang Ling, Michael R Lyu, and Irwin King. 2014. Ratings meet reviews, a
combined approach to recommend. In RecSys. 105–112.
[27] Julian McAuley and Jure Leskovec. 2013. Hidden factors and hidden topics:
understanding rating dimensions with review text. In RecSys. 165–172.
[28] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jerey Dean. 2013.
Distributed representations of words and phrases and their compositionality. In
NIPS. 3111–3119.
[29] Andriy Mnih and Ruslan R Salakhutdinov. 2008. Probabilistic matrix factorization.
In Advances in neural information processing systems. 1257–1264.
[30] Vinod Nair and Georey E. Hinton. 2010. Rectied linear units improve restricted
boltzmann machines. In ICML. 807–814.
[31] Jerey Pennington, Richard Socher, and Christopher Manning. 2014. Glove:
Global Vectors for Word Representation. In EMNLP. 1532–1543.
[32] Zhaochun Ren, Shangsong Liang, Piji Li, Shuaiqiang Wang, and Maarten de Rijke.
2017. Social collaborative viewpoint regression with explainable recommendations. In WSDM. 485–494.
[33] Steen Rendle. 2010. Factorization machines. In ICDM. 995–1000.
[34] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i
trust you?: Explaining the predictions of any classier. In SIGKDD. 1135–1144.
[35] Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. In Recommender systems handbook. Springer,
1–35.
[36] Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention
model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685
(2015).
[37] Nitish Srivastava, Georey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan
Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from
overtting. Journal of machine learning research 15, 1 (2014), 1929–1958.
[38] Xiaoyuan Su and Taghi M Khoshgoftaar. 2009. A survey of collaborative ltering
techniques. Advances in articial intelligence 2009 (2009), 4.
[39] Yunzhi Tan, Min Zhang, Yiqun Liu, and Shaoping Ma. 2016. Rating-Boosted
Latent Topics: Understanding Users and Items with Ratings and Reviews.. In
IJCAI. 2640–2646.
[40] Jesse Vig, Shilad Sen, and John Riedl. 2009. Tagsplanations: explaining recommendations using tags. In IUI. 47–56.
[41] Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative deep learning
for recommender systems. In SIGKDD. 1235–1244.
[42] Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua.
2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv preprint arXiv:1708.04617 (2017).
[43] Chenyan Xiong, Jimie Callan, and Tie-Yen Liu. 2017. Learning to attend and to
rank with word-entity duets. In SIGIR.
[44] Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping
Ma. 2014. Explicit factor models for explainable recommendation based on
phrase-level sentiment analysis. In SIGIR. 83–92.
[45] Yongfeng Zhang, Yunzhi Tan, Min Zhang, Yiqun Liu, Tat-Seng Chua, and Shaoping Ma. 2015. Catch the Black Sheep: Unied Framework for Shilling Attack
Detection Based on Fraudulent Action Propagation.. In IJCAI. 2408–2414.
[46] Lei Zheng, Vahid Noroozi, and Philip S Yu. 2017. Joint deep modeling of users
and items using reviews for recommendation. In WSDM. 425–434.