深度学习资源汇总
机器学习作为一种最能体现人工智能本质的领域，在过去几年里取得了显著的进步。与此同时，在这一过程中占据主导地位的是人工智能技术对全球格局重塑的能力。值得注意的是，在过去几天里，“AlphaGo以3:0击败柯洁”的事件再次将焦点转向了深度学习技术的发展及其深远影响。本文系统地整理并介绍了深度学习领域的优质资源，并旨在为在探索这一前沿领域前行的朋友提供有价值的参考与指导
深度 learning 是一门实践性学科，在实践中不断进行实验是取得进步的关键。BigQuant 人工智能量化投资平台 是一个整合了多种深度 learning 与 machine learning 开源框架的优质平台。它不仅支持主流的深度 learning 框架如 TensorFlow 和 Keras，并且还包含 XGBoost 等经典的 machine learning 算法，并为开发者提供了便捷的操作界面。
简介
机器学习领域中，深度学习是一种重要的分支学科。其核心研究方向主要聚焦于"深度神经网络"结构的构建与优化。自20世纪40年代至60年代间，在"控制论"理论指导下取得重要进展；随后从1986年到1995年之间,"联结主义"理论发展迅速；而近年来随着计算技术的突破性发展,"深度学习"逐渐成为机器智能领域的主流方法之一。基于GPU和TPU等高性能计算硬件的发展需求，在计算机视觉、语音识别、自然语言处理以及推荐系统等多个领域均取得了显著的应用成果
框架 （排名不分先后）
TensorFlow是由谷歌开源的机器学习框架提供支持，并由该框架提供可靠且高效的Python及C++接口；目前仅支持Python和C++接口，并计划未来逐步扩展至包括Go语言与Java语言的支持。
Theano is a classic deep learning framework that supports Python APIs and demonstrates strong capabilities in abstracting computational graphs.
Torch基于Lua语言构建了计算框架，并提供了大量预定义的模型实例；方便用户自定义层并支持GPU加速运算；主要不适用于递归神经网络的场景。
Caffe是一个广泛应用于机器视觉领域的核心库，在图像处理方面表现出色，在涉及文本信息、语音信号等其他类型的数据时则不建议使用。
CNTK由微软开源提供作为深度学习工具包使用的主要框架，并主要包含前馈神经网络架构、卷积神经网络模型以及循环神经网络结构。
MXNet是由陈天奇及其开发团队开发的深度学习框架。目前已被亚马逊认可并正式采用为其深度学习平台。其显著特点在于运行速度极快且内存管理效率极高。
Keras被视为一个功能强大的深度学习框架，在Python编程语言的基础上构建，并支持TensorFlow或Theano等后端框架的操作。其主要目标在于加速实验过程，在研究过程中尽量延迟从理论到实际成果之间的过渡是一个关键点。
XGBoost
该文档介绍了一种高效的机器学习算法——XGBoost
XGBoost被视为一个开放源代码软件包，并被广泛应用于多个编程语言环境中。该工具支持基于树的梯度提升方法，并且能够被集成到C++、Java、Python、R以及Julia等多种编程语言中。此外，在主要操作系统平台包括Linux操作系统族及其主流分支版本（如Ubuntu系列）、Windows操作系统的主流版本以及macOS平台之上均可运行
Pylearn2 是一个机器学习库。它 largely relies on Theano for its functionality. This means that you can develop plugins for Pylearn2 using mathematical expressions (such as new models and algorithms). Theano optimizes and stabilizes these expressions, compiling them into the backend of your choice, whether it's a CPU or GPU.
Chainer 是一种基于 Python 的开源框架，在深度学习领域具有广泛的应用。该框架提供了灵活、直观且性能优越的方法来构建并训练各种类型的深度学习模型，并支持最新的神经网络架构实现
Neon是Nervana利用Python开发的一种深度学习库。 Nervana致力于为金融机构提供一套全面部署深度学习的技术方案。
由于将BigQuant定位在该领域，并且整合了包括TensorFlow、Theano、XgBoost等在内的多个开源工具。该平台的研究者无需自行部署这些工具。
NeuralTalk是一个基于Python + numpy的项目,旨在实现多模态循环神经网络用于学习如何通过语言描述图像
在线书籍
Deep Learning: A Comprehensive Guide to the Field of Deep Learning by Yoshua Bengio, Ian Goodfellow and Aaron Courville
介绍：一本关于深度学习的教材，在涵盖的知识领域方面非常全面，并且在全面覆盖相关领域的同时，在深入探讨各个知识点上也达到了很高的水平。这本教材被广泛认为是系统深入学习该领域的参考资料。
人工神经网络与深度学习著Michael Nielsen
概述：这是一本无偿提供的在线资源，在系统地阐述神经网络与深度学习的关键原理与实践应用方面具有重要价值。
Deep Learning: 理论与应用
本书专门阐述基于深度学习的一般思路及其在信号与信息处理任务中的应用情况，并针对语言、文字处理、信息检索以及计算机视觉等领域展开具体运用介绍。
Comprehensive Guide to Deep Learning tutorial available at http://link.zhihu.com/?target=http%3A//deeplearning.net/tutorial/deeplearning.pdf. Research Group at the University of Montreal, January 6, 2015
该实验室发布的深度学习教材系统介绍了包括卷积神经网络、长短期记忆网络（LSTM）以及循环神经网络（RNN）等模型，并通过丰富的案例帮助读者更好地理解相关技术原理与实现细节；案例丰富且具有较强的实践指导价值
关于遗传算法的概览
关于遗传算法的概览
介绍：遗传算法简介
介绍：现为第3版，并已在全球 hundred+个国家由超过 thousand 所大学采用作为教科书，并提供 free online AI courses. 主要从 mathematical perspectives 展开讨论人工智能相关技术包括 problem-solving, knowledge representation, 和 reasoning processes.
Deep learning within neural networks: A comprehensive survey
本文对监督学习进行了全面的梳理，并详细讨论了其中包含反向传播算法的监督学习方法；同时系统介绍了无监督学习、强化学习以及进化学习的基础理论与应用情况。
课 程
Deep Learning Techniques: A Summer School in Montreal, 2015
Deep Learning Techniques: A Summer School in Montreal, 2015
蒙特利尔2015夏季深度学习暑期学校专为研究生学生、工程师以及研究人员设计，这些参与者已掌握机器学习的基本知识，并渴望进一步深入探索深度学习领域的研究方向。
Deep Learning Summer School, Montreal 2016
介绍：蒙特利尔2016年深度学习暑期班
介绍：该研究来自多伦多大学统计学系的Ruslan Salakhutdinov。该研究详细阐述了深度学习模型在电力系统中的实际应用，并探讨了如何构建智能电网。
斯坦福大学的机器学习课程
介绍：这门由吴恩达教授开设的世界著名机器学习课程。非常适合刚开始学习机器学习的人，这门课程内容安排得非常完整。它被收录在网易公开课上。
Stanford大学开设的深度学习与自然语言处理相关课程（UFLDL教程 - Ufldl）
本教程旨在深入探讨无监督特征学习与深度学习的核心概念。通过系统的学习过程，您将掌握多个具体算法的功能。观察到这些方法在实际中展现出的有效性。学会灵活运用这些思想到新问题中去。为了使您能够更好地理解这些方法，请提前掌握机器学习的基本知识（尤其是监督学习、逻辑回归以及梯度下降的相关概念），我们特别强调这一背景。
李宏毅老师开设的《深度学习》课程
介绍：台大李宏毅的课程讲义，以Theano为编程语言，适合大二以上的学生。
周莫煩對 Python 的 CNN 進行了詳細講解（卷積神經網路）
卷积神经网络作为一种新型的人工神经网络架构，在过去几年逐渐兴起，并因其在图像识别与语音识别等领域的显著优势而备受关注。这一技术已被广泛传播并得到广泛应用，在多个领域发挥着重要作用。
其最常见的应用场景之一便是计算机视觉中的图像识别任务；尽管如此；它还被成功应用于视频分析；自然语言处理以及药物研发等多个领域；尤其是最近风靡全球的人工智能Alpha Go不仅实现了计算机对围棋的理解与掌握；并且也展示了人工智能在这一领域的卓越能力。
该视频以简洁明了的方式；在短短五分钟内全面介绍了卷积神经网络的基本概念及其重要性。
CS224d: 高级自然语言处理与深度学习课程（Stanford大学）
斯坦福大学开设深度学习在自然语言处理领域的应用型在线课程，并涵盖搜索引擎优化（如网页搜索）、广告投放策略（如广告相关）、邮件营销策略（如电子邮件）、客户关系管理（如客户服务）、机器翻译技术（如语言翻译）以及医学影像分析（如放射学报告）等多个方面。
加州理工学院机器学习视频资源库](http://link.zhihu.com/?target=http://work.caltech.edu/library/)
介绍：加州理工大学 机器学习视频课程，课程里面的图表做得特别漂亮。
MIT 的 Underactuated Robotics 自 2014 年 10 月 1 日起开课，并是一门针对 MIT 研究生的课程。欢迎对机器人技术及非线性动力学领域感兴趣的朋友们前来探索这门课程！
官方课程：余凯与张潼联合打造的机器学习教学视频
百度推出的机器学习课程。如果你从事搜索引擎技术、网络广告投放、用户行为分析与预测、图像识别技术、自然语言理解与处理、生物信息学以及智能机器人技术等领域的研究或工作背景，则建议深入掌握这门核心课程的知识对于你的职业发展至关重要。
机器学习课程由斯坦福大学Andrew Ng教授在Coursera平台上从2010年到2014年期间教授。
介绍：斯坦福大学机器学习课程是一门广受欢迎的专业课程。该课程旨在全面涵盖机器学习、数据挖掘以及统计模式识别的核心内容。具体来说：
(i) 监督式学习模块将深入探讨参数与非参数算法的特性及其应用，并详细阐述支持向量机的基本原理以及核函数的设计与优化；同时涵盖神经网络模型的架构与训练方法。
(ii) 无监督学习部分则聚焦于集群分析与降维技术的基本原理及其在实际中的应用；同时深入讨论推荐系统的设计与优化，并结合深度 learning 模型提升其性能。
(iii) 在实例分析中，则会深入探讨偏差-方差权衡理论及其对模型性能的影响，并关注当前 machine learning 在人工智能领域的创新应用和发展趋势
机器学习 - 由Yaser Abu-Mostafa在2012至2014年间教授该课程
由费曼奖得主Yaser Abu-Mostafa教授亲授课程。系统地阐述机器学习的核心概念与核心技术。强调深刻的理解而非仅仅表面的知识。
机器学习 - 塎泉学院由汤米·米切尔教授在春季学期授课（2024年春季）
介绍：卡内基梅隆大学2011年春季机器学习课程，主讲人是Tom Mitchell
Introduces Neural Networks for Machine Learning - Geoffrey Hinton taught by Coursera (2012)
由多伦多大学深度学习领域的知名学者Geoffrey Hinton教授授课的公开课深入讲解了人工神经网络的基本原理，并探讨了机器学习的应用方法。可用于语音识别、图像分割以及人体姿态分析等多个领域。学习该课程需具备一定的微积分知识和Python编程基础。
Neural networks class](http://link.zhihu.com/?target=https%3A//www.youtube.com/playlist%3Flist%3DPL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH) - Mr. Hugo Larochelle from the University of Sherbrooke (2013)
介绍：是Hugo Larochelle在Sherbrooke大学神经网络课程的视频。
机器学习课程 - NYU CILVR实验室 (2014年)
介绍：CILVR机构（计算智能与技术融合的研究中心）提供深度学习专业课程模块。该中心汇聚了众多教师、博士研究生以及学术专家，在计算机视觉技术、机器学习算法以及医疗健康技术等领域的研究与教学并重。
课程视频 | 人工智能 | 电气工程与计算机科学 | MIT OpenCourseWare - 该平台由麻省理工学院（MIT）教授帕特里克·亨利·温斯顿提供，并提供人工智能领域的课程资源（课程于2010年秋季学期授课）
介绍：麻省理工学院人工智能课程
Visual perception and learning: computers and brains
视觉感知与学习：视觉感知与学习
麻省理工学院的教学内容着重融合计算与生物学视角来探索相关学习方法。该教学计划涵盖当前领域的最新研究成果及其局限性分析，在面部表情识别技术方面深入探讨计算机与人类大脑之间的交互机制，并结合马尔可夫决策过程模型研究人工智能在复杂认知活动中的应用潜力
Convolutional Neural Networks for Visual Recognition - Stanford University, by Fei-Fei Li and Andrej Karpathy (2016 year)]
介绍：着重介绍视觉识别的卷积神经网络。为2017年春季课程，比较新。
Deep Learning for Natural Language Processing - Stanford
介绍：斯坦福大学自然语言处理课程。
Neural Networks - usherbrooke
介绍：由雨果Larochelle教授开设的神经网络在线课程专为研究生设计，在课程中深入探讨了多个前沿主题包括自动编码器、稀疏编码技术以及卷积神经网络等。
Machine Learning (ML) - University of Oxford (2014-2015)
Oxford University's Machine Learning and Deep Learning course,授课教师为Nando de Freitas ，授课时间为2014至2015学年。
NVIDIA Deep Learning & AI Training Programs Nvidia (2015)
NVIDIA深度学习研究所（DLI）特意为开发人员设计了一项培训计划，在这项课程中, 学员们将深入研究一系列常用的开源框架, 同时也会有机会接触并操作NVIDIA最前沿的基于GPU的深度学习加速平台.
The 2012 Graduate Summer School on Deep Learning and Feature Learning was organized in 2012 by Geoffrey Hinton, Yoshua Bengio, Yann LeCun, Andrew Ng, Nando de Freitas and several other scholars @ IPAM, UCLA.
位于美国的纯数学与应用数学研究所（IPAM）是由美国国家科学基金会资助的一个国家级研究机构；它致力于推动基础数学与各领域科学技术的深入交流与合作；作为一项由多位知名学者共同参与的重要网课项目；该课程聚焦于机器学习/深度学习等前沿领域；为研究人员提供了便捷的学习平台
深度学习（中/英） - offered by Vincent Vanhoucke and Arpan Chakraborty, a course provided by Udacity and Google (launched in 2016)
介绍：google的深度学习课程，很多案例都是Kaggle竞赛中的。
Deep Learning课程 - University of Waterloo由Professor Ali Ghodsi教授于2015年讲授
介绍：Ali Ghodsi在You Tube上的深度学习课程。
Statistical Machine Learning paradigm - taught by Professor Wasserman at Carnegie Mellon University
介绍：Larry Wasserman在You Tube上的统计机器学习课程。
Deep Learning Course - The course details Yann LeCun's Deep Learning, specified in full detail at the provided link address.
介绍：Yann LeCun2015年至2016年深度学习课程。
[Bay Area Deep Learning School] - Andrew Ng and others / Stanford University, California (2016)
介绍：斯坦福大学2016年9月的深度学习课程班。包括多为深度学习教授。
Crafting, Representing and Deciphering Deep Neural Networks at the University of California, Berkeley
介绍：加州伯克利分校深度神经网络设计、可视化课程
UVA Deep Learning Course - Master of Science in Artificial Intelligence, University of Amsterdam.
该课程作为阿姆斯特丹大学人工智能硕士专业的核心内容进行讲解。该课程深入探讨了基于大数据训练的现代深度学习模型的理论基础，并特别强调计算机视觉与语言建模技术的发展与应用。其中计算机视觉与语言建模无疑是深度学习领域最具代表性和影响力的应用领域之一。该课程由Efstratios Gavves（副 教授）, Kirill Gavrilyuk, Berkay Kicanaoglu以及Patrick Putzky教授担任授课教师。
An Introduction to Deep Learning for Self-Driving Cars
An Introduction to Deep Learning for Self-Driving Cars
麻省理工学院推出一项为期一周的教学项目,聚焦于深度学习技术,其应用覆盖自然语言处理、计算机视觉以及生成对抗网络等关键领域
深入解析强化学习算法及其在深度学习框架中的实现
本课程由加州大学伯克利分校于2017年秋季学期开设，并旨在为本科生提供深度强化学习的教学内容概述
视频和演讲
Mind, the Process of Creation - Ray Kurzweil
Ray Kurzweil's talk at TED could offer insights into how human minds form, potentially influencing deep learning research.
Deep Learning, Self-Taught Learning, and Unsupervised Feature Learning - Andrew Ng
介绍：吴恩达关于深度学习、自学习和无监督特征学习的一个演讲。
Advances in Deep Learning, recently proposed by Geoff Hinton, are widely recognized as groundbreaking innovations.
综上所述：Geoff Hinton教授在本次讲座中对深度学习领域最新的研究进展进行了详细讲解。这一视频内容属于UBC计算机科学领域的知名讲座系列之一。
The marked effectiveness of Deep Learning - Yann LeCun
Facebook人工智能研究院负责人Yann LeCun博士就卷积神经网络以及其在机器学习与计算机视觉领域中的应用进行了深入讨论（约翰·霍普金斯大学 语言与语音处理研究中心 11/18/2014 巴尔的摩 MD）
Representation Learning - Yoshua Bengio
介绍：源自GoogleTechTalks的一次演讲。
Hierarchical Temporal Memory (HTM) - Jeff Hawkins
介绍：杰夫·霍金斯的“分层时态记忆原理（HTM）：机器智能基础”一个演讲。
机器学习讨论组 - 链接 - 由亚当·科特斯教授讲授的斯坦福大学人工智能实验室深度学习课程
概述：由Adam Coates担任主讲人，斯坦福人工智能实验室近期开展了一系列研究项目，并将系统性地介绍这些项目的最新进展。此外，还将深入探讨深度学习技术的发展与应用前景。
Understanding the World through Deep Learning - Adam Coates
介绍：从标题就可以看出，讲的是深度学习在现实世界中产生的一些运用。
Explained: Unsupervised Feature Learning - Adam Coates
概述：亚当·科特斯（Adam Coates）于2012年12月7日在加州大学伯克利分校发表了一个演说,题为《神秘无监督的功能学习》.
该深度学习框架由Yann LeCun提出
介绍：源自GoogleTechTalks系列的"深度学习-视觉感知"系列演讲一讲
Next Generation of Neural Networks : Dr. Geoffrey Hinton at Google Tech Talks
本次介绍：重量级专家Hinton在一个名为GoogleTechTalks的场合上进行的一次重要演讲。其主题围绕"下一代神经网络"展开
The remarkable and striking implications of computers that can learn - Jeremy Howard at TEDxBrussels
概述：Howard在一次TED的演讲中讨论了计算机能够自主学习的引人注目的特点及其潜在危害。
CS 294A/CS 294W - Unsupervised Deep Learning - Offered by Andrew Ng at Stanford University (2011)
介绍：吴恩达在斯坦福大学2011年对无监督深度学习的讲解。
A new introduction to advanced neural networks - Natalie Hammel and Lorraine Yurshansky
通过一个简单的例子"你的手机如何识别一只狗"来了解深度神经网络入门指南。这个入门指南非常有趣。
Deep Learning: Insights from Machine Learning - led by Steve Jurvetson and a panel at VLAB, Stanford University.
介绍：在2014年的讨论中，Stanford University's 商学院探讨了"深度学习与大数据智能化"这一主题。
An overview of Artificial Neural Networks and Deep Learning Techniques, presented by Leo Isikdogan at Motorola Mobility Headquarters.
介绍：芝加哥摩托罗拉总部于2016年夏天举办了一场涉及深度学习与人工神经网络的讨论活动。
NIPS 2016 lecture and workshop videos - NIPS 2016
介绍：从09年到17年每年的NIPS会议成果主页。
评论论文
Representation Learning: A Review and New Perspectives是Yoshua Bengio及其研究团队的工作
This scholarly work delves into the architecture of deep learning systems, specifically examining the mechanisms underlying their success in artificial intelligence applications.
Deep Machine Learning – A New Frontier in Artificial Intelligence Research – a survey paper by Itamar Arel, Derek C. Rose, and Thomas P. Karnowski.
Graves, A. (Year of publication). Supervised sequential labeling utilizing recurrent neural networks(Vol. 385). Springer.
Schmidhuber, J. (2014). A comprehensive review of deep neural networks utilizing learning algorithms. Approximately 75 pages, over 850 references provided.
LeCun等四位学者共同撰写了题为[Deep learning]的相关研究论文，并将其发表于Nature期刊上
增强学习
Mnih et al. “该研究利用深度强化学习在Atari游戏中取得了显著成果.” arXIV预印本arXIV:1312.5602（2013）
Volodymyr Mnih, Nicolas Heess, Alex Graves, Koray Kavukcuoglu. “Reiterative frameworks of perception mechanisms” ArXiv working paper, 2014.
计算机视觉
ImageNet Classification with Deep Convolutional Neural Networks, Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton, NIPS 2012.
Going Deeper with Convolutions, Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich posted on 19-Sept-2014.
该文提出了一种基于层次特征学习的方法用于场景标签化，并在IEEE Transactions on Pattern Analysis and Machine Intelligence期刊上发表于2013年
The paper introduces a novel approach to learning hierarchical convolutional features for visual recognition tasks. It was authored by Koray Kavukcuoglu and a team of researchers including Pierre Sermanet, Y-Lan Boureau, Karol Gregor, Michaël Mathieu, and Yann LeCun. The study was presented at the prestigious Advances in Neural Information Processing Systems (NIPS) conference in 2010.
Graves et al., A new innovative system for handwritten recognition.
Cireşan et al. (2010) explored Substantive and substantial neural structures for the recognition of handwritten digits. Neural computation, 22(12), 3207-3220.]
Ciresan, Dan 和 Ueli Meier 等人. 基于多列深度神经网络的图像分类研究. 计算机视觉与模式识别会议 (CVPR), 2012 IEEE 会议. IEEE, 2012 年.
A group of neural networks forming a committee for traffic sign classification was presented in A committee of neural networks for traffic sign classification at the 2011 IJCNN conference.
NLP和语言处理
Joint Learning of Words and Meaning Representations for Open-Text Semantic Parsing, Antoine Bordes, Xavier Glorot, Jason Weston and Yoshua Bengio (2012), in: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS)
Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. Socher, R., Huang, E. H., Pennington, J., Ng, A. Y., and Manning, C. D. (2011a). In NIPS’2011.
用于预测情感分布的半监督递归自动编码器.Socher等（2011b）在EMNLP'2011上发表
Tomáš Mikolov: Statistical language models grounded in neural network principles. PhD thesis, University of Brno, 2012.
The work by Graves, Alex, and Jürgen Schmidhuber was published in the journal Neural Networks in the year 2005, volume 18, issue 5, covering pages 602 to 610. The paper explores techniques such as framewise phoneme classification using bidirectional LSTM networks alongside other neural network architectures.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. "Distributed representations of words and phrases along with their compositionality" appears in Advances in Neural Information Processing Systems, pages 3111–3119, 2013.
K. Cho等人提出了一种基于RNN编码器解码器架构的学习短语表示方法，在统计机器翻译领域取得了显著成果
Sutskever等作者提出了一种基于神经网络的序列到序列学习方法，并在《神经信息处理进展》会议上进行了展示。他们的研究链接为Sequence to sequence learning with neural networks
迁移学习和域适应
Raina, Rajat, et al. “Self-supervised learning: transductive learning from unlabelled data.” Proceedings of the 24th international conference on Machine Learning. ACM, 2007.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio propose an effective method for domain adaptation in large-scale sentiment classification using deep learning. This innovative approach is published in the proceedings of the Twenty-eighth International Conference on Machine Learning (ICML’11), spanning pages 97 to 110. The research is accessible via the provided link.
R. Collobert et al., Almost from scratch: Natural Language Processing. Journal of Machine Learning Research, 12:2493-2537, 2011
Mesnil, Grégoire, et al. 该挑战采用深度学习的方法来解决：无监督与转移学习研讨会（结合ICML）于2011年召开。
Ciresan et al. introduced a method for cross-lingual knowledge transfer using deep learning models to process text in Latin and Chinese scripts, which was presented at the IJCNN conference held in 2012 and published by the IEEE. Their research paper provides a detailed analysis of this approach and is available at [http://link.zhihu.com/?target=http%3A//www.idsia.ch/~ciresan/data/ijcnn2012_v9.pdf].
Goodfellow, Ian, Aaron Courville, and Yoshua Bengio. “机器学习中的大规模特征学习与稀疏编码”的论文被收入《Advances in Neural Information Processing Systems》中。第15届神经信息处理系统的进展研讨会（NIPS 2011）
实用技巧与指导
Improving neural networks while avoiding the co-adaptation of feature detectors. Hinton and his team, including Geoffrey E. Hinton, et al., have published an arXiv preprint available under the identifier arXiv:1207.0580 in 2012.
针对深层架构基于梯度的实用建议, Yoshua Bengio from the University of Montreal authorizes this arXiv report with ID 1206.5533 within The Lecture Notes in Computer Science series, Volume 7700 as part of the The second edition of the book Neural Networks: Tricks of the Trade published in year 2012
可参考[Hinton, 2006]中的详细指导材料
稀疏编码
Formation of simple-cell receptive field characteristics through the acquisition of a sparse coding mechanism in natural images, Bruno Olhausen, Nature 1996.
Kavukcuoglu, Koray, Marc'Aurelio Ranzato, and Yann LeCun. 'Efficient object detection using sparse coding techniques with applications to visual recognition'. arXIV preprint Arenx IV: 1010.3467 (2010).
Goodfellow, Ian; Courville, Aaron; Bengio, Yoshua. "Large-scale feature learning employing spike-and-slab sparse coding techniques." In Proceedings of the 31st International Conference on Machine Learning (ICML), pp. 585-593. ICML 2012.
Effective algorithms for sparse coding have been developed by Honglak Lee, Alexis Battle, Raina Rajat and Andrew Y. Ng in the proceedings of NIPS 19, 2007 as a PDF document.
Sparse representation using a redundant basis set: A method utilized by the VI algorithm.
Olshausen, Bruno A., and David J. Field. Vision research 37.23 (1997): 3311-3326.
基础理论与动机
LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep learning.” Nature 521, no. 7553(2015): 436-444.
Hinton, Geoffrey E. Deterministic Boltzmann learning executes steepest descent within the weight-space. Neural computation 1.1 (1989): 143-150.
Bengio, Yoshua, and Samy Bengio. "Constructing models for high-dimensionality in discrete datasets using deep neural architectures." Advances in Neural Information Processing Systems 12 (2000): 400-406.
Bengio, Yoshua, et al. conducted a study titled "Greedy layer-wise training within deep neural networks." Their research was published in the publications of the field of neural information processing systems, volume 19 (2007), page 153.
Bengio, Yoshua, Martin Monperrus, and Hugo Larochelle. "Nonlocal estimation of manifold structure." Neural Computation 18.10 (2006): 2509-2528.
Dr. Hinton and Dr. Salakhutdinov discuss how neural networks can lower the dimensionality of data in their study published in Science, volume 313, issue 5786 in the year 2006, covering pages 504 to 507.
Marian Ranzato, Y., Laurent Boureau, and Yann LeCun. “Efficient feature extraction through deep belief networks.” Advances in neural information processing systems 20 (2007): 1185-1192.
Bengio, Yoshua; LeCun, Yann. "Advancing learning algorithms for AI development." Large-Scale Kernel Machines 34 (2007).
Le Roux, Nicolas, and Yoshua Bengio. "Expressive capacity of RBMs and DBNs." Neural Computation 20, no. 6 (2008): 1631-1649.
Among the authors are Ilya Sutskever and Geoffrey Hinton, alongside other notable contributors. The paper introduces Temporal-Kernel Recurrent Neural Networks as a novel approach in the field of machine learning. In the year 2010, this research was published in volume 23, issue 2 of the journal Neural Networks, covering pages 239 to 243.
Le Roux, Nicolas, and Yoshua Bengio. “Deep belief networks are compact universal approximators.” 可以改写为 “深度信念网络被称作紧凑型万能逼近器。”
Bengio, Yoshua, and Olivier Delalleau. “Investigating the expressive capabilities of deep architectures within algorithmic learning theory.” Springer Berlin/Heidelberg, 2011.
Montufar et al. ("When") investigate the conditions under which a mixture of products includes a product of mixtures.
Montúfar et al., On the Number of Linear Regions of Deep Neural Networks.] arXiv preprint arXiv:1402.1869 ( Montúfar et al., 2014 ).
监督式前馈神经网络
流形的切线分类器 [https://link.zhihu.com/?target=http%3A//books.nips.cc/papers/files/nips2011_1240.pdf] ; Salah Rifai, Yann Dauphin among his co-authors; Pascal Vincent and also Yoshua Bengio and Xavier Muller; all contributing to the work presented at the NIPS 2011 conference.
判别式学习Sum-Product网络.**, Gens et al., NIPS 2012最佳学生论文.
Goodfellow et al., I. et al., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. (2013). Maxout networks. Paper published by the University of Montreal.
Hinton, Geoffrey E., et al. Enhanced neural network performance through mitigating the co-adaptation among feature detectors. (Link to original paper: https://arxiv.org/abs/1207.0580)
Wang、Sida以及Christopher Manning. "Fast dropout training." 在Proceedings of the 30th International Conference on Machine Learning (ICML-13)上发表于2013年。
Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. “Deep sparse rectifier networks.” In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. JMLR W&CP Volume, vol. 15, pp. 315-323. 2011.
ImageNet分类与深度卷积神经网络模型，分别为Alexandr Krizhevsky、Ilya Sutskever以及Geoffrey E. Hinton，在第25届神经信息处理系统研讨会（NIPS）中。
大规模深度学习
Building High-level Features Through Extensive Unsupervised Learning Techniques
Researchers Yoshua Bengio and his team have published an extensive work on Neural probabilistic language models. Specifically, the section within this paper that discusses asynchronous Stochastic Gradient Descent is Section 3.
Dean and Jeffrey along with other co-authors presented large-scale distributed deep neural networks in their work published in the journal Advances in Neural Information Processing Systems in 2012.
循环神经网络
Learning Recurrent Neural Networks, Ilya Sutskever: Doctoral Dissertation, circa 2012.
Bengio et al. present a seminal study examining the challenges associated with training recurrent neural networks. Their work highlights the difficulties in capturing long-term dependencies using conventional gradient descent methods.
Mikolov Tomáš: 基于神经网络的统计语言模型的博士论文. 博斯特罗姆大学技术学院, 2012年.
Hochreiter, Sepp und Jürgen Schmidhuber. "Long short-term memory. (1997), Neural Computation 9.8: 1735–1780.
Hochreiter et al. (2001)进行了深入研究
Schmidhuber, J. (1992). Acquiring complex, extended sequences based on the principle of history compression. Neural Computation, 4(2), 234-242.
Graves et al. (2006) introduced an innovative approach to labelling unsegmented sequence data using recurrent neural networks through their work titled "Connectionist temporal classification". Their research, presented at the 23rd International Conference on Machine Learning in August 2006, contributed significantly to the field by introducing a novel method for assigning labels to discretely unsegmented sequences within a continuous temporal framework. The paper, published in the proceedings of the conference and accessible via the provided link, offers a detailed exploration of their methodology and its applications in machine learning.
Hinton, Geoffrey E. “Deterministic Boltzmann learning converges along the direction of maximum descent within the weight space.” Neural computation 1.1 (1989): 143-150.
Decreasing the dimensionalities of data using neural networks in the year 2006.
超参数
“Practical Bayesian Optimization of Machine Learning Algorithms”, Jasper Snoek, Hugo Larochelle, Ryan Adams, NIPS 2012.
该研究团队于James Bergstra和Yoshua Bengio于2012年在《机器学习研究期刊》中提出了随机搜索方法用于超参数优化
Methods for Fine-Tuning Hyperparameters, James Bergstra, Rémy Bardenet, Yoshua Bengio and Balázs Kégl within the NIPS.2011 conference, 2011.
优化
Implementing Hessian-Free Optimization for Training Deep and Recurrent Neural Networks, as led by James Martens and Ilya Sutskever, appears in Neural Networks: Practical Tips and Tricks, 2012.
Schaul et al. ' [No More Intricate Learning Rate Challenges ] ( http://link.zhihu.com/?target=http%3A//arxiv.org/pdf/1206.1106 ).' arXiv preprint arXIV:
Le Roux et al., Topmoumoute online natural gradient algorithm presented at the NIPS conference in 2007.]
Antoine Bordes et al. propose the SGD-QN optimization algorithm for training large-scale neural networks.
Analyzing the challenges involved in training deep feedforward neural networks, Glorot, Xavier, and Yoshua Bengio presented their work: 'Understanding the difficulty of training deep feedforward neural networks.' This research was published in the proceedings of the AISTATS conference in 2010.
This paper introduces a novel approach to constructing deep sparse rectifier networks, as presented in "Deep Sparse Rectifier Networks." This work was featured in the proceedings from the 14th International Conference on Artificial Intelligence and Statistics, published in the JMLR Workshop and Conference Proceedings (JMLR W&CP), Volume 15, Year 2011.
Second-order optimization methods for deep learning via Hessian-free techniques have shown promising results in training large-scale neural networks.
Sepp Hochreiter, Jürgen Schmidhuber, und Hans Georg Beyer. '‘Flat minima.''. Neural computation in volume 9.1 (1997), pages 1–42.
Pascanu, Razvan, and Yoshua Bengio. "Re-examining the natural gradient within the context of deep neural networks." arXiv preprint arXiv:1301.3584 (2013).
Dauphin et al. Detect the saddle point issue in high-dimensional non-convex optimization: a contribution to the field of neural information processing conferences. In Advances in Neural Information Processing Systems, pp. 2933-2941. 2014
无监督特征学习
Salaхutдинов, Руслан и Георджий Е. Хинтингтон публиковали в Deep Boltzman machine работы по исследованию...
Scholarpedia article introduces Deep Belief Networks.
Deep Boltzmann Machines
An Effective Approach to Training Deep Boltzmann Machines, Ruslan Salakhutdinov and Geoffrey E. Hinton, Journal of Neural Computation, August 2012; Volume 24, Issue 8: Pages 1967–2006.
Montavon, Grégoire, and Klaus-Robert Müller. "Deep Boltzmann Machines and the Centering Trick.) " Neural Networks: Tricks of the Trade (2012): 621-637.
Salahkulin, Ruslanov, and Hugo Larochelle. “The effective learning of deep Boltzmann machines”. Efficient learning of deep boltzmann machines.] International Conference on Artificial Intelligence and Statistics. 2010.
Ruslan Salalahutdonov's dissertation focuses on deep generative modeling approaches and is available at http://link.zhihu.com/?target=http%3A//cubs.buffalo.edu/govind/CSE705-SeminarPapers/9.pdf. His work was completed at the University of Toronto in 2009.
Goodfellow and his co-authors have made significant contributions to the field of machine learning through their work on multi-prediction deep Boltzmann machines (MDDBMs), as evidenced by their publication in the prestigious Advances in Neural Information Processing Systems (ANIPS) conference proceedings. The study explores innovative approaches to enhancing the predictive capabilities of Boltzmann machines through a combination of multiple prediction strategies. This research has been widely recognized and cited by the academic community, further solidifying its impact on the development of advanced machine learning models.
无监督图像建模技术 using spike-and-slab restricted Boltzmann machines, Aaron Courville, James Bergstra and Yoshua Bengio, presented at ICML’2011.
Geoffrey Hinton,{A practical guide to training restricted Boltzmann machines} Momentum 9.1 (2010), 926
自动编码
Regularized Auto-Encoders Estimate Local Statistics, Guillaume Alain, Yoshua Bengio and Salah Rifai, Université de Montréal, arXiv report 1211.4246, 2012
An Approach to the Generation of Contractive Auto-Encoder Samples was presented at ICML 2012 in Edinburgh, Scotland.
Compressed Auto-Encoder Models: Maintaining invariant features through feature-space compression has been explored in detail by Salah Rifai, Pascal Vincent, Xavier M. Muller, Xavier Glorot & Yoshua Bengio within the proceedings of ICML 2011.
[Disentangling factors of variation for facial expression recognition](http://www-etud.iro.umontreal.ca/rifais(material/rifai_eccv_2012.pdf), Salah Rifai et al., in ECCV 2012.
Vincent, Pascal, 等人. "层次堆叠的去噪自编码器: 在深度网络中使用局部去噪准则学习有用表示." ⟨https://doi.org/10.1093/jmlr/vqq45⟩. 《机器学习研究》(Machine Learning) 12 (2015): 669-725.
Vincent, Pascal. “A connection between score matching and denoising autoencoders.” 出版于 Neural computation 23.7 (2011), 页码为 1661-1674.
Chen et al. Marginalized denoising autoencoder for domain adaptation was an arXiv preprint released in 2012.
其他
The ICML 2009 Conference on Learning Hierarchical Feature Representations webpage features a Compilation of Reading Material linked to http://link.zhihu.com/?target=http%3A//www.cs.toronto.edu/~rsalakhu/deeplearning/references.html.
the Stanford University's UFLDL Recommended Readings.
The LISApublic wiki contains areading list and additionally includes a bibliography.
Geoff Hinton has attended the NIPS 2007 tutorial, which he later referred to as one of his most influential readings.
The LISA publications database includes a collection of advanced deep learning structures, as detailed on the provided link: http://link.zhihu.com/?target=http%3A//www.iro.umontreal.ca/~lisa/publications2/index.php/topics/single/27.
An introductory overview of AI, Machine Learning, along with an exploration of deep learning concepts within Yoshua Bengio’s IFT6266 graduate course.
The Memkite platform's deep learning-based repository includes an annotated bibliography from DeepLearning.University.
An advanced deep learning repository authored by Jeremy D. Jackson, PhD
Goodfellow, Ian, et al. “Assessing variations in deep neural networks.” Advances in neural information processing systems 22 (2009): 646-654.
Bengio, Yoshua, et al. “The Enhanced Mixing through Deep Representations.” arXiv preprint, arXiv:1207.4404 (2012).
Xavier Glorot, Antoine Bordes and Yoshua Bengio. Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach. In Proceedings of the Twenty-Eighth International Conference on Machine Learning (ICML 2011), pages 97–110.
欢迎补充。
文中提供的网页链接，均来自于网络，如有问题，请站内告知。
本作品
本作品