【第三课】kaggle案例分析三

Evernote Export

比赛题目介绍

• 作为全球领先的第三方移动数据分析平台,TalkingData致力于为中国用户提供最丰富、最真实的移动设备用户行为数据。
• 该平台目前拥有超过二十万真实用户数据样本,这些数据全部经过严格脱敏处理,按照年龄性别以及地理位置等关键维度进行细致分类,例如:22-25岁的男性用户与30-35岁的女性用户。
• 用户画像的本质就是对用户的多维特征识别系统,根据企业业务需求动态调整标签体系,既包括固定不变的基础属性标签,也支持灵活多变的行为特征标签。
• 在这一过程中,我们主要关注的维度包括:地理位置、人口统计信息(年龄性别)、职业背景、文化特征以及消费能力等方面的关键指标。
• 在产品层面的具体刻画维度主要包括:产品类别使用频率、活跃度指数以及用户的兴趣偏好等方面的数据记录。
• 在技术架构上,我们的模型主要采用无监督学习算法为主流方案,同时结合半监督学习方法提升模型的适应性与泛化能力。
• 用户画像的核心作用体现在以下几个方面:
  • 精准营销:基于用户的画像特征制定个性化的营销策略
  • 数据驱动分析:通过挖掘用户的消费模式与行为轨迹建立智能推荐系统
  • 智能服务优化:利用画像分析结果优化个性化服务体验
  • 竞争情报支持:为企业提供竞争对手分析的重要数据依据
  • 用户运营指导:为精准定位目标群体提供科学依据

人工神经网络原理

• 知识地图
  • 由单层感知器发展出多层感知器体系
  • 多层感知器向自编码器技术的演进
  • 自编码器基础上延伸出卷积神经网络体系
  • 卷积神经网络与深度残差网络结合应用
  • 递归神经网络技术衍生出LSTM模型
  • 单层感知机向Hopfield神经网络过渡
  • Hopfield神经网络扩展至Bolazmann机研究
  • 神经机器学习框架下RBM成为关键组件
• 神经网络要素
  • 网络结构特征包含全连接层、分层架构、时滞回路的存在与否、权值共享机制以及激活函数的选择
  • 运行机制涵盖异步更新与同步更新两种模式，并包含前馈计算的基本流程
• 训练算法
  • 基于小批量数据的批量归一化技术应用广泛
  • 数据预处理方法直接影响模型性能表现
• 训练数据
  • 输入样本与目标输出数据构成训练集的主要内容
• 单层感知器
  • 输入节点的定义及其作用功能分析
  • 输出节点在信号传递过程中的角色定位
  • 权向量参数的具体表示形式及意义解析
  • 偏置因子对模型性能的影响机制探讨
• 激活函数

sng(w1​x1​+w2​x2​+w3x3​)=0

单层感知器类比于线性分类器

感知器学习的规则

在1958年首次提出了一种基于单层计算单元的神经网络结构，并将其命名为感知器

r=dj​−oj​

在公式中，dj代表期望输出；其中oj=f(Wj^T X)所示的感知器采用了一种与阈值转移函数具有类似特性的符号转换机制；其数学表达式如下所示

f(WjT​X)=sgn(WjT​X)={1,−1,​(WjT​≥0)(WjT​<0)​

因此，权值调整公式为：

ΔWj​=±2ηX

感知器学习规则仅限于二进制神经元范畴，在设定初始参数时具有一定的灵活性；该规则代表一种有导师的学习机制，在神经网络领域占据重要地位。其中，在神经网络的学习过程中扮演着基础角色，并且其对训练过程的指导作用不容小觑。

多层前馈神经网络（BP网络）

• 隐藏层与隐藏节点
• 前馈-----每一层的节点仅和下一层节点相连

BP学习算法 本质就是梯度下降法

神经网络的两个过程：1.训练过程 2.推断过程

总结：从数学角度来看，梯度下降法通过构建基于误差平方和的损失函数来实现权值优化。该方法通过反向传播机制计算损失函数对各权重参数的偏导数，在推断过程中，误差信号会沿着反向传播路径传递，在完成这一系列计算后，则确定了相应的权重参数。

在Python深度学习领域中，Keras提供了一个简便易用的框架来进行基础神经网络的设计与实现。Keras的功能模块能够处理数据或序列数据流的操作图结构，并且将各个功能环节独立化处理以提高灵活性。此外，默认情况下Keras基于TensorFlow运行，若要切换到Theano框架，则需修改指定路径下的json配置文件。

keras的基本使用

• 基于顺序架构构建Sequential模型
• 该序贯架构由多个网络层按顺序连接构成
• 通过将一个layers列表传递给Sequential模型来实现该架构

 from keras.models import Sequential

    
 from keras.layers import Dense,Activation
    
 model = Sequential([    
    
 Dense(32,units=784),    
    
 Activation('relu'),    
    
 Dense(10),    
    
 Activation('softmax'),])

• 也可以通过.add()方法一个个的将layer加入模型中

 model = Sequential()

    
 model.add(Dense(32,input_shape=(784,))
    
 )model.add(Activation('relu'))

• 指定输入数据的shape
  • input shape（元祖型数据）
  • input dim

 model = Sequential()

    
 model.add(Dense(32,input_dim=784))

• input length

 model = Sequentail()

    
 model.add(Dense(32,input_shape=784))

• batch_size

评分标准

选手计算用户在各个分组中的概率，在现实中每个用户仅属于一个分组。在理想情况下，如果某个用户的概率被精确分配为1，则属于该分组的概率为1；其余情况则为0。这种分配方式不会带来任何损失。通常情况下，在实际应用中同一个用户可能会以较大的或较小的概率被分配到多个组别；这种情况下就会产生概率上的损失；这种损失的大小直接反映了答案的质量水平；我们所使用的优化函数或者评估指标正是基于这样的损失函数来衡量模型性能的

logloss=−N1​i=1∑N​j=1∑M​yij​log(pij​)

max(min(p,1−10−15),10−15)

%23%23%23%20%E6%AF%94%E8%B5%9B%E9%A2%98%E7%9B%AE%E4%BB%8B%E7%BB%8D%0A*%20TalkingData%E6%98%AF%E4%B8%AD%E5%9B%BD%E6%9C%80%E5%A4%A7%E7%9A%84%E7%AC%AC%E4%B8%89%E6%96%B9%E7%A7%BB%E5%8A%A8%E6%95%B0%E6%8D%AE%E5%B9%B3%E5%8F%B0%EF%BC%8C%E7%A7%BB%E5%8A%A8%E8%AE%BE%E5%A4%87%E7%94%A8%E6%88%B7%E6%97%A5%E5%B8%B8%E7%9A%84%E9%80%89%E6%8B%A9%E5%92%8C%E8%A1%8C%E4%B8%BA%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E3%80%82%E7%9B%AE%E5%89%8D%EF%BC%8CTalkingData%E6%AD%A3%E5%9C%A8%E5%AF%BB%E6%B1%82%E6%AF%8F%E5%A4%A9%E5%9C%A8%E4%B8%AD%E5%9B%BD%E6%B4%BB%E8%B7%83%E7%9A%845%E4%BA%BF%E7%A7%BB%E5%8A%A8%E8%AE%BE%E5%A4%8770%25%E4%BB%A5%E4%B8%8A%E7%9A%84%E8%A1%8C%E4%B8%BA%E6%95%B0%E6%8D%AE%EF%BC%8C%E5%B8%AE%E5%8A%A9%E5%AE%A2%E6%88%B7%E6%9B%B4%E5%A5%BD%E7%9A%84%E4%BA%86%E8%A7%A3%E4%B8%8E%E5%85%B6%E7%94%A8%E6%88%B7%E4%BA%92%E5%8A%A8%E3%80%82%0A*%20TalkingData%E6%8F%90%E4%BE%9B%E4%BA%86%E5%A4%A7%E7%BA%A620%E4%B8%87%E7%94%A8%E6%88%B7%E7%9A%84%E6%95%B0%E6%8D%AE(%E5%85%A8%E9%83%A8%E7%9C%9F%E5%AE%9E%E4%B8%94%E7%BB%8F%E8%BF%87%E8%84%B1%E6%95%8F%E5%A4%84%E7%90%86)%EF%BC%8C%E6%8C%89%E7%85%A7%E5%B9%B4%E9%BE%84%E5%92%8C%E6%80%A7%E5%88%AB%E5%88%86%E6%88%9012%E4%B8%AA%E7%BB%84%EF%BC%8C%E6%AF%94%E5%A6%82%E7%94%B7%E6%80%A722%E5%88%B025%E5%B2%81%EF%BC%8C%E5%A5%B3%E6%80%A730%E5%88%B035%E5%B2%81%EF%BC%8C%E5%90%8C%E6%97%B6%E6%8F%90%E4%BE%9B%E4%BA%86%E7%94%A8%E6%88%B7%E8%A1%8C%E4%B8%BA%E5%B1%9E%E6%80%A7%EF%BC%8C%E6%AF%94%E5%A6%82%E5%9C%A8%E4%BB%80%E4%B9%88%E6%A0%B7%E7%9A%84%E6%97%B6%E9%97%B4%E7%82%B9%E5%87%BA%E7%8E%B0%E4%BB%80%E4%B9%88%E6%A0%B7%E7%9A%84%E5%9C%B0%E7%90%86%E4%BD%8D%E7%BD%AE%E7%AD%89%E7%AD%89%EF%BC%8C%E9%80%89%E6%89%8B%E9%80%9A%E8%BF%87%E8%BF%99%E4%BA%9B%E4%BF%A1%E6%81%AF%E5%8E%BB%E6%8E%A8%E6%B5%8B%E7%94%A8%E6%88%B7%E6%98%AF%E5%88%86%E5%9C%A8%E5%93%AA%E4%B8%80%E4%B8%AA%E6%80%A7%E5%88%AB%E5%B9%B4%E9%BE%84%E9%87%8C%E9%9D%A2%E3%80%82%0A*%20**%E4%BB%80%E4%B9%88%E6%98%AF%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F**%0A*%20%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E5%8F%AF%E4%BB%A5%E7%90%86%E8%A7%A3%E4%B8%BA%E5%B0%B1%E6%98%AF%E4%B8%BA%E7%94%A8%E6%88%B7%E7%94%BB%E6%A0%87%E7%AD%BE%EF%BC%8C%E6%A0%87%E7%AD%BE%E6%98%AF%E4%B8%8D%E5%9B%BA%E5%AE%9A%E7%9A%84%EF%BC%8C%E4%BE%9D%E7%85%A7%E4%BC%81%E4%B8%9A%E4%B8%9A%E5%8A%A1%E5%9C%BA%E6%99%AF%E5%8A%A8%E6%80%81%E6%B7%BB%E5%8A%A0%E3%80%81%E5%88%A0%E9%99%A4%E7%AD%89%EF%BC%8C%E6%9C%89%E7%9A%84%E6%A0%87%E7%AD%BE%E6%98%AF%E6%B0%B8%E4%B9%85%E7%9A%84%EF%BC%8C%E6%9C%89%E7%9A%84%E6%A0%87%E7%AD%BE%E6%98%AF%E5%8F%AF%E4%BB%A5%E5%8A%A8%E6%80%81%E5%8F%98%E6%9B%B4%E7%9A%84%0A*%20%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E5%8F%AF%E4%BB%A5%E7%9C%8B%E5%81%9A%E6%98%AF%E7%BB%BC%E5%90%88%E6%80%A7%E7%9A%84%E6%A0%87%E7%AD%BE%E7%B3%BB%E7%BB%9F%EF%BC%8C%E5%9C%A8%E8%87%AA%E8%BA%AB%E6%8B%A5%E6%9C%89(%E6%88%96%E4%BB%8E%E7%AC%AC%E4%B8%89%E6%96%B9%E8%8E%B7%E5%BE%97)%E7%9A%84%E5%8E%9F%E5%A7%8B%E6%95%B0%E6%8D%AE%E4%B8%AD%EF%BC%8C%E4%BB%8E%E5%A4%9A%E4%B8%AA%E7%BB%B4%E5%BA%A6%E5%AF%B9%E7%94%A8%E6%88%B7%E8%BF%9B%E8%A1%8C%E4%BA%A7%E5%93%81%E8%BF%9B%E8%A1%8C%E5%88%BB%E7%94%BB%EF%BC%8C%E6%8F%90%E5%8F%96%E5%87%BA%E5%95%86%E4%B8%9A%E4%BB%B7%E5%80%BC%E6%BD%9C%E5%8A%9B%E7%9A%84%E8%AF%AD%E4%B9%89%E4%BF%A1%E6%81%AF%EF%BC%8C%E6%98%AF%E5%B8%B8%E8%A7%81%E7%9A%84%E4%BC%81%E4%B8%9A%E7%BA%A7%E5%A4%A7%E6%95%B0%E6%8D%AE%E5%BA%94%E7%94%A8%0A*%20%E5%AF%B9%E4%BA%8E%E4%BA%BA%E5%B8%B8%E8%A7%81%E6%A0%87%E7%AD%BE%E4%B8%8E%E7%BB%B4%E5%BA%A6%EF%BC%8C%E5%9C%B0%E5%9F%9F%E3%80%81%E5%B9%B4%E9%BE%84%E3%80%81%E6%80%A7%E5%88%AB%E3%80%81%E6%96%87%E5%8C%96%E3%80%81%E8%81%8C%E4%B8%9A%E3%80%81%E6%94%B6%E5%85%A5%E3%80%81%E7%94%9F%E6%B4%BB%E4%B9%A0%E6%83%AF%E3%80%81%E6%B6%88%E8%B4%B9%E4%B9%A0%E6%83%AF%E7%AD%89%0A*%20%E5%AF%B9%E4%BA%8E%E4%BA%A7%E5%93%81%E5%B8%B8%E8%A7%81%E5%88%BB%E7%94%BB%E7%BB%B4%E5%BA%A6%EF%BC%8C%E4%BA%A7%E5%93%81%E7%B1%BB%E5%88%AB%E3%80%81%E6%B4%BB%E8%B7%83%E9%A2%91%E7%8E%87%E3%80%81%E4%BA%A7%E5%93%81%E5%96%9C%E5%A5%BD%E3%80%81%E4%BA%A7%E5%93%81%E9%A9%B1%E5%8A%A8%E3%80%81%E4%BD%BF%E7%94%A8%E4%B9%A0%E6%83%AF%E3%80%81%E4%BA%A7%E5%93%81%E6%B6%88%E8%B4%B9%E7%AD%89%0A*%20%E6%8A%80%E6%9C%AF%E6%A0%88%EF%BC%9A%E6%97%A0%E7%9B%91%E7%9D%A3%E5%AD%A6%E4%B9%A0%E3%80%81%E5%8D%8A%E7%9B%91%E7%9D%A3%E5%AD%A6%E4%B9%A0(%E6%8A%80%E5%B7%A7%E6%80%A7%E5%BE%88%E5%BC%BA)%EF%BC%8C%E5%A4%A7%E5%A4%9A%E6%95%B0%E6%98%AF%E6%9C%89%E7%9B%91%E7%9D%A3%E5%AD%A6%E4%B9%A0%0A*%20**%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E7%9A%84%E4%BD%9C%E7%94%A8**%0A*%20%E7%B2%BE%E5%87%86%E8%90%A5%E9%94%80%EF%BC%8C%E5%88%86%E6%9E%90%E4%BA%A7%E5%93%81%E6%BD%9C%E5%9C%A8%E5%AE%A2%E6%88%B7%EF%BC%8C%E9%92%88%E5%AF%B9%E7%89%B9%E5%AE%9A%E7%BE%A4%E4%BD%93%E5%88%A9%E7%94%A8%E7%9F%AD%E4%BF%A1%E9%82%AE%E4%BB%B6%E7%AD%89%E6%96%B9%E5%BC%8F%E8%BF%9B%E8%A1%8C%E8%90%A5%E9%94%80%0A*%20%E7%94%A8%E6%88%B7%E7%BB%9F%E8%AE%A1%EF%BC%8C%E6%AF%94%E5%A6%82%E4%B8%AD%E5%9B%BD%E5%A4%A7%E5%AD%A6%E8%B4%AD%E4%B9%B0%E4%B9%A6%E7%B1%8D%E4%BA%BA%E6%95%B0TOP10%EF%BC%8C%E5%85%A8%E5%9B%BD%E5%88%86%E5%9F%8E%E5%B8%82%E6%8C%87%E6%95%B0%E7%AD%89%0A*%20%E6%95%B0%E6%8D%AE%E6%8C%96%E6%8E%98%EF%BC%8C%E6%9E%84%E5%BB%BA%E6%99%BA%E8%83%BD%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9F%EF%BC%8C%E5%88%A9%E7%94%A8%E5%85%B3%E8%81%94%E8%A7%84%E5%88%99%E8%AE%A1%E7%AE%97%EF%BC%8C%E5%96%9C%E6%AC%A2%E7%BA%A2%E9%85%92%E7%9A%84%E9%80%9A%E5%B8%B8%E5%96%9C%E6%AC%A2%E4%BB%80%E4%B9%88%E8%BF%90%E5%8A%A8%E5%93%81%E7%89%8C%EF%BC%8C%E5%88%A9%E7%94%A8%E8%81%9A%E7%B1%BB%E7%AE%97%E6%B3%95%E5%88%86%E6%9E%90%EF%BC%8C%E5%96%9C%E6%AC%A2%E7%BA%A2%E9%85%92%E7%9A%84%E4%BA%BA%E7%9A%84%E5%B9%B4%E9%BE%84%E6%AE%B5%E5%88%86%E5%B8%83%E6%83%85%E5%86%B5%0A*%20%E8%BF%9B%E8%A1%8C%E6%95%88%E6%9E%9C%E8%AF%84%E4%BC%B0%EF%BC%8C%E5%AE%8C%E5%96%84%E4%BA%A7%E5%93%81%E8%BF%90%E7%94%A8%EF%BC%8C%E6%8F%90%E5%8D%87%E6%9C%8D%E5%8A%A1%E8%B4%A8%E9%87%8F%EF%BC%8C%E5%85%B6%E5%AE%9E%E8%BF%99%E4%B9%9F%E5%B0%B1%E6%98%AF%E7%9B%B8%E5%BD%93%E4%BA%8E%E5%B8%82%E5%9C%BA%E8%B0%83%E7%A0%94%E3%80%81%E7%94%A8%E6%88%B7%E8%B0%83%E7%A0%94%EF%BC%8C%E8%BF%85%E9%80%9F%E4%B8%8B%E5%AE%9A%E4%BD%8D%E6%9C%8D%E5%8A%A1%E7%BE%A4%E4%BD%93%EF%BC%8C%E6%8F%90%E4%BE%9B%E9%AB%98%E6%B0%B4%E5%B9%B3%E7%9A%84%E6%9C%8D%E5%8A%A1%0A*%20%E5%AF%B9%E6%9C%8D%E5%8A%A1%E6%88%96%E4%BA%A7%E5%93%81%E8%BF%9B%E8%A1%8C%E7%A7%81%E4%BA%BA%E5%AE%9A%E5%88%B6%EF%BC%8C%E5%8D%B3%E4%B8%AA%E6%80%A7%E5%8C%96%E7%9A%84%E6%9C%8D%E5%8A%A1%E6%9F%90%E7%B1%BB%E7%BE%A4%E4%BD%93%E7%94%9A%E8%87%B3%E6%AF%8F%E4%B8%80%E4%BD%8D%E7%94%A8%E6%88%B7(%E8%BF%99%E6%98%AF%E6%9C%AA%E6%9D%A5%E7%9A%84%E6%B6%88%E8%B4%B9%E8%B6%8B%E5%8A%BF)%E6%AF%94%E5%A6%82%EF%BC%8C%E6%9F%90%E5%85%AC%E5%8F%B8%E9%80%80%E5%87%BA%E4%B8%80%E6%AC%BE5-10%E5%B2%81%E5%84%BF%E7%AB%A5%E7%9A%84%E7%8E%A9%E5%85%B7%EF%BC%8C%E9%80%9A%E8%BF%87%E7%94%A8%E6%88%B7%E7%94%BB%E5%83%8F%E5%88%86%E6%9E%90%EF%BC%8C%E5%8F%91%E7%8E%B0%E6%9F%90%E4%BA%9B%E7%89%B9%E5%BE%81%E7%9A%84%E6%AF%94%E9%87%8D%E6%9C%80%E5%A4%A7%EF%BC%8C%E5%B0%B1%E7%BB%99%E4%BA%A7%E5%93%81%E6%8F%90%E4%BE%9B%E4%BA%86%E9%9D%9E%E5%B8%B8%E5%8F%AF%E8%A7%82%E7%9A%84%E5%86%B3%E7%AD%96%E4%BE%9D%E6%8D%AE%0A*%20%E4%B8%9A%E5%8A%A1%E7%BB%8F%E8%90%A5%E5%88%86%E6%9E%90%E4%BB%A5%E5%8F%8A%E7%AB%9E%E4%BA%89%E5%88%86%E6%9E%90%EF%BC%8C%E5%BD%B1%E5%93%8D%E4%BC%81%E4%B8%9A%E5%8F%91%E5%B1%95%E6%88%98%E7%95%A5%0A%0A%23%23%23%23%20%E4%BA%BA%E5%B7%A5%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E5%8E%9F%E7%90%86%0A*%20**%E7%9F%A5%E8%AF%86%E5%9C%B0%E5%9B%BE**%0A*%20%E4%BB%8E%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%0A*%20%E4%BB%8E%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0%E8%87%AA%E7%BC%96%E7%A0%81%E5%99%A8%0A*%20%E4%BB%8E%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0%E5%8D%B7%E7%A7%AF%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%EF%BC%8C%E5%86%8D%E5%88%B0%E6%B7%B1%E5%BA%A6%E6%AE%8B%E5%B7%AE%E7%BD%91%E7%BB%9C%0A*%20%E4%BB%8E%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0%E9%80%92%E5%BD%92%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%EF%BC%8C%E5%86%8D%E5%88%B0LSTM%0A*%20%E4%BB%8E%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%88%B0Hopfield%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%EF%BC%8C%E5%86%8D%E5%88%B0Bolazmann%E6%9C%BA%E5%92%8CRBM%0A*%20%E7%94%A8RBM%E5%A0%86%E5%8F%A0%E6%88%90DBN%EF%BC%8CDBN%E4%B8%8E%E5%A4%9A%E5%B1%82%E8%87%AA%E7%BC%96%E7%A0%81%E5%99%A8%E7%BB%93%E5%90%88%E6%88%90DBN-DNN%0A*%20**%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E8%A6%81%E7%B4%A0**%0A*%20%E7%BD%91%E8%B7%AF%E7%BB%93%E6%9E%84(%E5%85%A8%E8%BF%9E%E6%8E%A5%E3%80%81%E5%88%86%E5%B1%82%E3%80%81%E6%9C%89%E6%97%B6%E6%BB%9E%E5%9B%9E%E8%B7%AF%E3%80%81%E6%9D%83%E5%80%BC%E5%85%B1%E4%BA%AB%E3%80%81%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0)%0A*%20%E8%BF%90%E8%A1%8C%E6%9C%BA%E5%88%B6(%E5%BC%82%E6%AD%A5%E6%9B%B4%E6%96%B0%E3%80%81%E5%90%8C%E6%AD%A5%E6%9B%B4%E6%96%B0%E3%80%81%E5%89%8D%E9%A6%88)%0A*%20%E8%AE%AD%E7%BB%83%E7%AE%97%E6%B3%95%EF%BC%8C%E8%AE%AD%E7%BB%83%E4%B8%AD%E4%BD%BF%E7%94%A8%E7%9A%84trick(mini%20batch%20BN%20drop%20out%E7%AD%89)%EF%BC%8C%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%E7%9A%84%E5%AE%9A%E4%B9%89%0A*%20%E8%AE%AD%E7%BB%83%E6%95%B0%E6%8D%AE(%E6%95%B0%E6%8D%AE%E7%9A%84%E9%A2%84%E5%A4%84%E7%90%86%EF%BC%8C%E8%BE%93%E5%85%A5%E5%92%8C%E8%BE%93%E5%87%BA%E7%9A%84%E6%9E%84%E6%88%90%E7%AD%89)%0A*%20**%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8**%0A*%20%E8%BE%93%E5%85%A5%E8%8A%82%E7%82%B9%0A*%20%E8%BE%93%E5%87%BA%E8%8A%82%E7%82%B9%0A*%20%E6%9D%83%E5%90%91%E9%87%8F%0A*%20%E5%81%8F%E7%BD%AE%E5%9B%A0%E5%AD%90%0A*%20%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0%0A*%20%E5%AD%A6%E4%B9%A0%E7%8E%87%0A*%20%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E5%AD%A6%E4%B9%A0%E7%AE%97%E6%B3%95%0A*%20%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E7%9A%84%E5%B1%80%E9%99%90%0A%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0%0A%24%24sng(w_1x_1%2Bw_2x_2%2Bw3x_3)%3D0%24%24%0A%E5%8D%95%E5%B1%82%E6%84%9F%E7%9F%A5%E5%99%A8%E7%B1%BB%E6%AF%94%E4%BA%8E%E7%BA%BF%E6%80%A7%E5%88%86%E7%B1%BB%E5%99%A8%0A%23%23%23%23%20%E6%84%9F%E7%9F%A5%E5%99%A8%E5%AD%A6%E4%B9%A0%E7%9A%84%E8%A7%84%E5%88%99%0A1958%E5%B9%B4%EF%BC%8C%E9%A6%96%E5%85%88%E5%AE%9A%E4%B9%89%E4%BA%86%E4%B8%80%E4%B8%AA%E5%85%B7%E6%9C%89%E5%8D%95%E5%B1%82%E8%AE%A1%E7%AE%97%E5%8D%95%E5%85%83%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%BB%93%E6%9E%84%EF%BC%8C%E7%A7%B0%E4%B8%BA%E6%84%9F%E7%9F%A5%E5%99%A8%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E7%9A%84%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99%E8%A7%84%E5%AE%9A%EF%BC%8C%E5%AD%A6%E4%B9%A0%E4%BF%A1%E5%8F%B7%E7%AD%89%E4%BA%8E%E7%A5%9E%E7%BB%8F%E5%85%83%E6%9C%9F%E6%9C%9B%E8%BE%93%E5%87%BA(%E6%95%99%E5%B8%88%E4%BF%A1%E5%8F%B7)%E4%B8%8E%E5%AE%9E%E9%99%85%E8%BE%93%E5%87%BA%E4%B9%8B%E5%B7%AE%0A%24%24r%3Dd_j-o_j%24%24%0A%E5%BC%8F%E4%B8%AD%EF%BC%8C%24d_j%24%E4%B8%BA%E6%9C%9F%E6%9C%9B%E7%9A%84%E8%BE%93%E5%87%BA%EF%BC%8C%24o_j%3Df(W%5ET_jX)%24%E6%84%9F%E7%9F%A5%E5%99%A8%E9%87%87%E7%94%A8%E4%BA%86%E4%B8%8E%E9%98%88%E5%80%BC%E8%BD%AC%E7%A7%BB%E5%87%BD%E6%95%B0%E7%B1%BB%E4%BC%BC%E7%9A%84%E7%AC%A6%E5%8F%B7%E8%BD%AC%E7%A7%BB%E5%87%BD%E6%95%B0%EF%BC%8C%E5%85%B6%E8%A1%A8%E8%BE%BE%E5%BC%8F%E4%B8%BA%0A%0A%24%24%20f(W%5ET_jX)%3Dsgn(W%5ET_jX)%3D%20%5Cbegin%7Bcases%7D%201%2C%20%26%20%5Ctext%20%7B%24(W%5ET_j%20%5Cgeq%200)%24%7D%20%5C%5C%20-1%2C%20%26%20%5Ctext%7B%24(W%5ET_j%20%3C%200)%24%7D%20%5Cend%7Bcases%7D%20%24%24%0A%0A%E5%9B%A0%E6%AD%A4%EF%BC%8C%E6%9D%83%E5%80%BC%E8%B0%83%E6%95%B4%E5%85%AC%E5%BC%8F%E4%B8%BA%EF%BC%9A%0A%24%24%5CDelta%20W_j%20%3D%5Cpm%20%202%20%5Ceta%20X%24%24%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99%E5%8F%AA%E9%80%82%E7%94%A8%E4%BA%8E%E4%BA%8C%E8%BF%9B%E5%88%B6%E7%A5%9E%E7%BB%8F%E5%85%83%EF%BC%8C%E5%88%9D%E5%A7%8B%E5%80%BC%E5%8F%AF%E5%8F%96%E4%BB%BB%E6%84%8F%E5%80%BC%E3%80%82%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99%E4%BB%A3%E8%A1%A8%E4%B8%80%E7%A7%8D%E6%9C%89%E5%AF%BC%E5%B8%88%E5%AD%A6%E4%B9%A0%EF%BC%8C%E7%94%B1%E4%BA%8E%E6%84%9F%E7%9F%A5%E5%99%A8%E7%90%86%E8%AE%BA%E6%98%AF%E7%A0%94%E7%A9%B6%E5%85%B6%E4%BB%96%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9A%84%E5%9F%BA%E7%A1%80%EF%BC%8C%E8%AF%A5%E8%A7%84%E5%88%99%E5%AF%B9%E4%BA%8E%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9A%84%E6%9C%89%E5%AF%BC%E5%B8%88%E5%AD%A6%E4%B9%A0%E5%85%B7%E6%9C%89%E6%9E%81%E4%B8%BA%E9%87%8D%E8%A6%81%E7%9A%84%E6%84%8F%E4%B9%89%E3%80%82%0A%0A**%E5%A4%9A%E5%B1%82%E5%89%8D%E9%A6%88%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%EF%BC%88BP%E7%BD%91%E7%BB%9C%EF%BC%89**%0A*%20%E9%9A%90%E8%97%8F%E5%B1%82%E4%B8%8E%E9%9A%90%E8%97%8F%E8%8A%82%E7%82%B9%0A*%20%E5%89%8D%E9%A6%88-----%E6%AF%8F%E4%B8%80%E5%B1%82%E7%9A%84%E8%8A%82%E7%82%B9%E4%BB%85%E5%92%8C%E4%B8%8B%E4%B8%80%E5%B1%82%E8%8A%82%E7%82%B9%E7%9B%B8%E8%BF%9E%0A%0A!%5B88f5c6cf69add3012dd2319faa0f31e1.png%5D(en-resource%3A%2F%2Fdatabase%2F1352%3A1)%0A%0ABP%E5%AD%A6%E4%B9%A0%E7%AE%97%E6%B3%95%E6%9C%AC%E8%B4%A8%E5%B0%B1%E6%98%AF%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%0A%3E%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9A%84%E4%B8%A4%E4%B8%AA%E8%BF%87%E7%A8%8B%EF%BC%9A1.%E8%AE%AD%E7%BB%83%E8%BF%87%E7%A8%8B%202.%E6%8E%A8%E6%96%AD%E8%BF%87%E7%A8%8B%0A%0A%3E%E5%B0%8F%E7%BB%93%EF%BC%9A%E6%95%B0%E5%AD%A6%E8%83%8C%E6%99%AF%EF%BC%9A%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95(%E4%BD%BF%E7%94%A8%E8%AF%AF%E5%B7%AE%E5%B9%B3%E6%96%B9%E5%92%8C%E6%9E%84%E9%80%A0%E8%AF%AF%E5%B7%AE%E5%87%BD%E6%95%B0)%0A%3E%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%E7%BB%99%E5%87%BA%E6%9D%83%E5%80%BC%EF%BC%8C%E9%80%9A%E8%BF%87%E5%8F%8D%E5%90%91%E4%BC%A0%E6%92%AD%E8%AF%AF%E5%B7%AE%E9%80%90%E5%B1%82%E8%AE%AD%E7%BB%83%EF%BC%8C%E6%8E%A8%E6%96%AD%E6%97%B6%E8%AF%AF%E5%B7%AE%E5%BE%80%E5%90%8E%E8%B5%B0%EF%BC%8C%E6%9E%84%E9%80%A0%E5%AE%8C%E5%90%8E%E7%BB%99%E5%87%BA%E6%9D%83%E5%80%BC%EF%BC%8C%E8%AF%AF%E5%B7%AE%E4%BF%A1%E5%8F%B7%E9%80%9A%E8%BF%87%E5%85%AC%E5%BC%8F%E8%AE%A1%E7%AE%97%E6%A2%AF%E5%BA%A6%EF%BC%8C%E5%BE%97%E5%87%BA%E6%A2%AF%E5%BA%A6%E5%AF%B9%E6%9D%83%E5%80%BC%E8%BF%9B%E8%A1%8C%E8%BF%AD%E4%BB%A3%E6%94%B9%E5%8F%98%EF%BC%8C%E7%9F%A5%E9%81%93%E6%9D%83%E5%80%BC%E4%B8%8D%E5%86%8D%E5%8F%98%E5%8C%96%E6%88%96%E8%80%85%E6%AC%A1%E6%95%B0%E8%B6%B3%E5%A4%9F%E6%94%B6%E6%95%9B%EF%BC%8C%E5%B0%B1%E7%BB%93%E6%9D%9F%E6%95%B4%E4%B8%AA%E8%AE%AD%E7%BB%83%E8%BF%87%E7%A8%8B%0A%0A%3E%E5%9C%A8python%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E4%B8%AD%EF%BC%8C%E5%8F%AF%E4%BB%A5%E4%BD%BF%E7%94%A8keras%E8%BF%9B%E8%A1%8C%E7%AE%80%E5%8D%95%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%AD%89%E8%AE%BE%E8%AE%A1%0A%3Ekeras%E6%A1%86%E6%9E%B6%E6%98%AF%E7%AE%80%E5%8D%95%E6%96%B9%E4%BE%BF%E7%9A%84%EF%BC%8C%E6%A8%A1%E5%9D%97%E5%8F%AF%E4%BB%A5%E4%BD%9C%E4%B8%BA%E6%95%B0%E6%8D%AE%E6%88%96%E5%BA%8F%E5%88%97%E7%9A%84%E8%BF%90%E7%AE%97%E5%9B%BE%E7%AD%89%EF%BC%8Ckeras%E4%B8%AD%E7%9A%84%E5%90%84%E4%B8%AA%E8%AE%BE%E8%AE%A1%E7%8E%AF%E8%8A%82%E9%83%BD%E6%98%AF%E5%8D%95%E7%8B%AC%E7%9A%84%E4%B8%80%E4%B8%AA%E6%A8%A1%E5%9D%97%EF%BC%8C%E6%89%80%E4%BB%A5%E9%9C%80%E8%A6%81%E6%A0%B9%E6%8D%AE%E8%87%AA%E5%B7%B1%E7%9A%84%E9%9C%80%E6%B1%82%E6%9D%A5%E8%AE%BE%E8%AE%A1%0A%3E**%5Bkeras%E4%B8%AD%E6%96%87%E6%96%87%E6%A1%A3%5D(http%3A%2F%2Fkeras-cn.readthedocs%2Fen%2Flatest)%0A%3Ekeras%E9%BB%98%E8%AE%A4%E4%BD%BF%E7%94%A8tensorflow%EF%BC%8C%E5%A6%82%E6%9E%9C%E4%BD%BF%E7%94%A8theano%E9%9C%80%E8%A6%81%E4%BF%AE%E6%94%B9%E6%96%87%E4%BB%B6%E4%B8%AD%E7%9A%84%E8%B7%AF%E5%BE%84%E5%86%85%E7%9A%84json%E6%96%87%E4%BB%B6%0A%0A%23%23%23%23%20keras%E7%9A%84%E5%9F%BA%E6%9C%AC%E4%BD%BF%E7%94%A8%0A*%20Sequential%20%E6%A8%A1%E5%9E%8B%E6%9E%84%E5%BB%BA%0A*%20%E5%BA%8F%E8%B4%AF%E6%A8%A1%E5%9E%8B%E6%98%AF%E5%A4%9A%E4%B8%AA%E7%BD%91%E7%BB%9C%E5%B1%82%E7%9A%84%E7%BA%BF%E6%80%A7%E5%A0%86%E5%8F%A0%0A*%20%E5%8F%AF%E4%BB%A5%E9%80%9A%E8%BF%87%E5%90%91Sequential%E6%A8%A1%E5%9E%8B%E4%BC%A0%E9%80%92%E4%B8%80%E4%B8%AAlayer%E7%9A%84list%E6%9D%A5%E6%9E%84%E9%80%A0%E8%AF%A5%E6%A8%A1%E5%9E%8B%0A%60%60%60python%0Afrom%20keras.models%20import%20Sequential%0Afrom%20keras.layers%20import%20Dense%2CActivation%0Amodel%20%3D%20Sequential(%5B%C2%A0%C2%A0%C2%A0%20%0ADense(32%2Cunits%3D784)%2C%C2%A0%C2%A0%C2%A0%20%0AActivation('relu')%2C%C2%A0%C2%A0%C2%A0%20%0ADense(10)%2C%C2%A0%C2%A0%C2%A0%20%0AActivation('softmax')%2C%5D)%0A%60%60%60%0A%0A*%20%E4%B9%9F%E5%8F%AF%E4%BB%A5%E9%80%9A%E8%BF%87.add()%E6%96%B9%E6%B3%95%E4%B8%80%E4%B8%AA%E4%B8%AA%E7%9A%84%E5%B0%86layer%E5%8A%A0%E5%85%A5%E6%A8%A1%E5%9E%8B%E4%B8%AD%0A%60%60%60python%0Amodel%20%3D%20Sequential()%0Amodel.add(Dense(32%2Cinput_shape%3D(784%2C))%0A)model.add(Activation('relu'))%0A%60%60%60%0A%0A*%20%E6%8C%87%E5%AE%9A%E8%BE%93%E5%85%A5%E6%95%B0%E6%8D%AE%E7%9A%84shape%0A*%20input%20shape%EF%BC%88%E5%85%83%E7%A5%96%E5%9E%8B%E6%95%B0%E6%8D%AE%EF%BC%89%0A*%20input%20dim%20%0A%60%60%60python%0Amodel%20%3D%20Sequential()%0Amodel.add(Dense(32%2Cinput_dim%3D784))%0A%60%60%60%0A*%20input%20length%0A%60%60%60python%0Amodel%20%3D%20Sequentail()%0Amodel.add(Dense(32%2Cinput_shape%3D784))%0A%60%60%60%0A*%20batch_size%0A%0A%23%23%23%23%20%E8%AF%84%E5%88%86%E6%A0%87%E5%87%86%0A*%20%E9%80%89%E6%89%8B%E7%AE%97%E5%87%BA%E7%94%A8%E6%88%B7%E5%9C%A8%E4%B8%8D%E5%90%8C%E5%88%86%E7%BB%84%E4%B8%8A%E7%9A%84%E6%A6%82%E7%8E%87%EF%BC%8C%E7%8E%B0%E5%AE%9E%E4%B8%AD%E4%B8%80%E4%B8%AA%E7%94%A8%E6%88%B7%E5%8F%AA%E8%83%BD%E5%9C%A8%E4%B8%80%E4%B8%AA%E5%88%86%E7%BB%84%E3%80%82%E7%90%86%E6%83%B3%E7%8A%B6%E6%80%81%E4%B8%8B%E5%A6%82%E6%9E%9C%E8%83%BD%E7%AE%97%E5%87%BA%E8%BF%99%E4%B8%AA%E6%A6%82%E7%8E%87%E6%98%AF1%EF%BC%8C%E5%85%B6%E4%BB%96%E6%98%AF0%E7%9A%84%E8%AF%9D%EF%BC%8C%E8%BF%99%E4%B8%AA%E7%AD%94%E6%A1%88%E5%B0%B1%E6%98%AF%E6%B2%A1%E6%9C%89%E4%BB%BB%E4%BD%95%E6%8D%9F%E5%A4%B1%E7%9A%84%E3%80%82%E4%B8%80%E8%88%AC%E6%9D%A5%E8%AF%B4%EF%BC%8C%E6%8F%90%E4%BA%A4%E7%9A%84%E7%AD%94%E6%A1%88%E4%B8%AD%EF%BC%8C%E6%9F%90%E4%B8%AA%E7%94%A8%E6%88%B7%E4%BC%9A%E6%9C%89%E6%88%96%E5%A4%A7%E6%88%96%E5%B0%8F%E7%9A%84%E6%A6%82%E7%8E%87%E5%B1%9E%E4%BA%8E%E5%A4%9A%E4%B8%AA%E7%BB%84%E5%88%AB%EF%BC%8C%E8%BF%99%E4%B8%AA%E6%97%B6%E5%80%99%E5%B0%B1%E6%9C%89%E6%A6%82%E7%8E%87%E4%B8%8A%E7%9A%84%E6%8D%9F%E5%A4%B1%EF%BC%8C%E8%BF%99%E4%B8%AA%E6%8D%9F%E5%A4%B1%E7%9A%84%E9%AB%98%E4%BD%8E%E4%BB%A3%E8%A1%A8%E7%AD%94%E6%A1%88%E7%9A%84%E6%B0%B4%E5%B9%B3%E3%80%82%E6%88%91%E4%BB%AC%E4%BC%98%E5%8C%96%E5%87%BD%E6%95%B0%E6%88%96%E8%80%85%E8%AF%B4%E6%98%AF%E8%AF%84%E4%BC%B0%E6%8C%87%E6%A0%87%E5%B0%B1%E6%98%AF%E4%B8%8B%E9%9D%A2%E7%9A%84%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%0A%24%24logloss%3D-%5Cfrac%7B1%7D%7BN%7D%20%5Csum%5EN_%7Bi%3D1%7D%20%5Csum%5EM_%7Bj%3D1%7D%20y_%7Bij%7D%20log(p_%7Bij%7D)%24%24%0A%24%24max(min(p%2C1-10%5E%7B-15%7D)%2C10%5E%7B-15%7D)%24%24%0A%0A!%5Bc430046ff391af67322b0c4d02f2c6a7.png%5D(en-resource%3A%2F%2Fdatabase%2F1354%3A0)%0A%0A%3E%E6%96%B0%E6%80%9D%E8%B7%AF%0A%3E%E7%89%B9%E5%BE%81**%0A%3E%20*%20Xgboost%3A%E4%B8%BB%E8%A6%81%E4%BD%BF%E7%94%A8count%E7%89%B9%E5%BE%81%E4%B8%8ETF-IDF%E7%89%B9%E5%BE%81%0A%3E%20*%20Keras%EF%BC%9A%E4%B8%BB%E8%A6%81%E4%BD%BF%E7%94%A8bag%20of%20apps%E7%89%B9%E5%BE%81%0A%3E%20**%E4%B8%A4%E6%AE%B5%E9%A2%84%E6%B5%8B**%0A%3E%20%E7%94%9F%E6%88%90%E7%89%B9%E5%BE%81%E9%9B%86%0A%3E%20%E9%A2%84%E6%B5%8B%E7%94%A8%E6%88%B7%E7%9A%84%E6%80%A7%E5%88%AB%E6%A6%82%E7%8E%87(Stage%201)%0A%3E%20%E4%BD%BF%E7%94%A8%E9%A2%84%E6%B5%8B%E7%9A%84%E6%80%A7%E5%88%AB%E4%BD%9C%E4%B8%BA%E9%A2%9D%E5%A4%96%E7%9A%84%E7%89%B9%E5%BE%81%EF%BC%8C%E9%A2%84%E6%B5%8B%E5%90%84%E4%B8%AA%E5%B9%B4%E9%BE%84%E6%AE%B5%E7%9A%84%E6%A6%82%E7%8E%87%EF%BC%8C%E5%85%88%E5%81%87%E8%AE%BE%E7%94%A8%E6%88%B7%E6%98%AF%E5%A5%B3%E6%80%A7%EF%BC%8C%E9%A2%84%E6%B5%8B%E5%85%B6%E5%B1%9E%E4%BA%8E%E5%90%84%E4%B8%AA%E5%B9%B4%E9%BE%84%E6%AE%B5%E7%9A%84%E6%A6%82%E7%8E%87%EF%BC%8C%E5%BE%97%E5%88%B0%24P(A_i%7CF)%24%EF%BC%8C%E5%86%8D%E5%81%87%E8%AE%BE%E7%94%A8%E6%88%B7%E4%B8%BA%E7%94%B7%E6%80%A7%EF%BC%8C%E9%A2%84%E6%B5%8B%E5%85%B6%E5%B1%9E%E4%BA%8E%E5%90%84%E4%B8%AA%E5%B9%B4%E9%BE%84%E6%AE%B5%E7%9A%84%E6%A6%82%E7%8E%87(Stage%201)%0A%3E%20%E6%A0%B9%E6%8D%AE%E6%9D%A1%E4%BB%B6%E6%A6%82%E7%8E%87%E5%85%AC%E5%BC%8F%E8%8E%B7%E5%BE%97%E7%94%A8%E6%88%B7%E7%9A%84%E4%BA%BA%E5%8F%A3%E7%89%B9%E5%BE%81%E5%88%86%E7%BB%84%E6%A6%82%E7%8E%87%0A%3E%20%24P(A_i%2CF)%3DP(A_i%7CF)P(F)%24%20for%20i...6%20and%20%24P(A_i%2CM)%3DP(A_i%7CM)P(M)%24%20for%20i%3D1....6%20%24A_i%24%E4%BB%A3%E8%A1%A8%E8%BF%99%E4%B8%AA%E5%B9%B4%E9%BE%84%E7%9A%846%E4%B8%AA%E5%88%86%E7%BB%84%EF%BC%8C1%20to%206%20%E4%BB%A3%E8%A1%A8%E7%9A%84%E6%98%AFF%E5%A5%B3%E6%80%A7%EF%BC%8CM%E7%94%B7%E6%80%A7

转载于:https://www.cnblogs.com/pandaboy1123/p/10394562.html