自动机器学习(AutoML)领域论文合集(持续更新中)!!!
发布时间
阅读量:
阅读量
Awesome-AutoML-Papers includes very up-to-date overviews of the bread-and-butter techniques we need in AutoML :
- Automated Data Clean (Auto Clean)
- Automated Feature Enginnering (Auto FE)
- Hyperparameter Optimization (HPO)
- Meta-Learning
- Neural Architecture Search (NAS)

Table of Contents
-
Papers
-
Surveys
-
Automated Feature Engineering
- Expand Reduce
- Hierarchical Organization of Transformations
- Meta Learning
- Reinforcement Learning
-
Architecture Search
- Evolutionary Algorithms
- Local Search
- Meta Learning
- Reinforcement Learning
- Transfer Learning
-
Hyperparameter Optimization
- Bayesian Optimization
- Evolutionary Algorithms
- Lipschitz Functions
- Local Search
- Meta Learning
- Particle Swarm Optimization
- Random Search
- Transfer Learning
-
Performance Prediction
- Performance Prediction
-
Frameworks
-
Miscellaneous
-
-
Tutorials
- Bayesian Optimization
- Meta Learning
-
Articles
- Bayesian Optimization
- Meta Learning
-
Slides
- Bayesian Optimization
-
Books
- Meta Learning
-
Projects
-
Prominent Researchers
Papers
Surveys
- 2019 | AutoML: A Survey of the State-of-the-Art | Xin He, et al. | arXiv |
PDF - 2019 | Survey on Automated Machine Learning | Marc Zoeller, Marco F. Huber | arXiv |
PDF - 2019 | Automated Machine Learning: State-of-The-Art and Open Challenges | Radwa Elshawi, et al. | arXiv |
PDF - 2018 | Taking Human out of Learning Applications: A Survey on Automated Machine Learning | Quanming Yao, et al. | arXiv |
PDF
Automated Feature Engineering
Expand Reduce
* 2017 | AutoLearn — Automated Feature Generation and Selection | Ambika Kaul, et al. | ICDM | [`PDF`](https://ieeexplore.ieee.org/document/8215494/)
* 2017 | One button machine for automating feature engineering in relational databases | Hoang Thanh Lam, et al. | arXiv | [`PDF`](https://arxiv.org/pdf/1706.00327.pdf)
* 2016 | Automating Feature Engineering | Udayan Khurana, et al. | NIPS | [`PDF`](http://workshops.inf.ed.ac.uk/nips2016-ai4datasci/papers/NIPS2016-AI4DataSci_paper_13.pdf)
* 2016 | ExploreKit: Automatic Feature Generation and Selection | Gilad Katz, et al. | ICDM | [`PDF`](http://ieeexplore.ieee.org/document/7837936/)
* 2015 | Deep Feature Synthesis: Towards Automating Data Science Endeavors | James Max Kanter, Kalyan Veeramachaneni | DSAA | [`PDF`](http://www.jmaxkanter.com/static/papers/DSAA_DSM_2015.pdf)
Hierarchical Organization of Transformations
* 2016 | Cognito: Automated Feature Engineering for Supervised Learning | Udayan Khurana, et al. | ICDMW | [`PDF`](http://ieeexplore.ieee.org/document/7836821/)
Meta Learning
* 2017 | Learning Feature Engineering for Classification | Fatemeh Nargesian, et al. | IJCAI | [`PDF`](https://www.ijcai.org/proceedings/2017/0352.pdf)
Reinforcement Learning
* 2017 | Feature Engineering for Predictive Modeling using Reinforcement Learning | Udayan Khurana, et al. | arXiv | [`PDF`](https://arxiv.org/pdf/1709.07150.pdf)
* 2010 | Feature Selection as a One-Player Game | Romaric Gaudel, Michele Sebag | ICML | [`PDF`](https://hal.archives-ouvertes.fr/inria-00484049/document)
Architecture Search
Evolutionary Algorithms
* 2019 | Evolutionary Neural AutoML for Deep Learning | Jason Liang, et al. | arXiv | [`PDF`](https://arxiv.org/abs/1902.06827)
* 2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR | [`PDF`](https://arxiv.org/abs/1703.01041)
* 2002 | Evolving Neural Networks through Augmenting Topologies | Kenneth O.Stanley, Risto Miikkulainen | Evolutionary Computation | [`PDF`](http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf)
Local Search
* 2017 | Simple and Efficient Architecture Search for Convolutional Neural Networks | Thomoas Elsken, et al. | ICLR | [`PDF`](https://arxiv.org/pdf/1711.04528.pdf)
Meta Learning
* 2016 | Learning to Optimize | Ke Li, Jitendra Malik | arXiv | [`PDF`](https://arxiv.org/pdf/1606.01885.pdf)
Reinforcement Learning
* 2018 | AMC: AutoML for Model Compression and Acceleration on Mobile Devices | Yihui He, et al. | ECCV | [`PDF`](http://openaccess.thecvf.com/content_ECCV_2018/papers/Yihui_He_AMC_Automated_Model_ECCV_2018_paper.pdf)
* 2018 | Efficient Neural Architecture Search via Parameter Sharing | Hieu Pham, et al. | arXiv | [`PDF`](https://arxiv.org/abs/1802.03268)
* 2017 | Neural Architecture Search with Reinforcement Learning | Barret Zoph, Quoc V. Le | ICLR | [`PDF`](https://arxiv.org/pdf/1611.01578.pdf)
Transfer Learning
* 2017 | Learning Transferable Architectures for Scalable Image Recognition | Barret Zoph, et al. | arXiv | [`PDF`](https://arxiv.org/abs/1707.07012)
Network Morphism
* 2018 | Efficient Neural Architecture Search with Network Morphism | Haifeng Jin, et al. | arXiv | [`PDF`](https://arxiv.org/abs/1806.10282)
Continuous Optimization
* 2018 | Neural Architecture Optimization | Renqian Luo, et al. | arXiv | [`PDF`](https://arxiv.org/abs/1808.07233)
* 2019 | DARTS: Differentiable Architecture Search | Hanxiao Liu, et al. | ICLR | [`PDF`](https://arxiv.org/abs/1806.09055)
Frameworks
- 2019 | Towards modular and programmable architecture search | Renato Negrinho, et al. | NeurIPS |
PDF - 2019 | Evolutionary Neural AutoML for Deep Learning | Jason Liang, et al. | arXiv |
PDF - 2017 | ATM: A Distributed, Collaborative, Scalable System for Automated Machine Learning | T. Swearingen, et al. | IEEE |
PDF - 2017 | Google Vizier: A Service for Black-Box Optimization | Daniel Golovin, et al. | KDD |
PDF - 2015 | AutoCompete: A Framework for Machine Learning Competitions | Abhishek Thakur, et al. | ICML |
PDF
Hyperparameter Optimization
Bayesian Optimization
* 2018 | A Tutorial on Bayesian Optimization. | [`PDF`](https://arxiv.org/pdf/1807.02811.pdf)
* 2018 | Efficient High Dimensional Bayesian Optimization with Additivity and Quadrature Fourier Features | Mojmír Mutný, et al. | NeurIPS | [`PDF`](https://papers.nips.cc/paper/8115-efficient-high-dimensional-bayesian-optimization-with-additivity-and-quadrature-fourier-features.pdf)
* 2018 | High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups. | PMLR | [`PDF`](https://arxiv.org/pdf/1802.07028v2.pdf)
* 2016 | Bayesian Optimization with Robust Bayesian Neural Networks | Jost Tobias Springenberg, et al. | NIPS | [`PDF`](https://papers.nips.cc/paper/6117-bayesian-optimization-with-robust-bayesian-neural-networks.pdf)
* 2016 | Scalable Hyperparameter Optimization with Products of Gaussian Process Experts | Nicolas Schilling, et al. | PKDD | [`PDF`](https://link.springer.com/chapter/10.1007/978-3-319-46128-1_3)
* 2016 | Taking the Human Out of the Loop: A Review of Bayesian Optimization | Bobak Shahriari, et al. | IEEE | [`PDF`](http://ieeexplore.ieee.org/document/7352306/)
* 2016 | Towards Automatically-Tuned Neural Networks | Hector Mendoza, et al. | JMLR | [`PDF`](http://aad.informatik.uni-freiburg.de/papers/16-AUTOML-AutoNet.pdf)
* 2016 | Two-Stage Transfer Surrogate Model for Automatic Hyperparameter Optimization | Martin Wistuba, et al. | PKDD | [`PDF`](https://link.springer.com/chapter/10.1007/978-3-319-46128-1_13)
* 2015 | Efficient and Robust Automated Machine Learning | [`PDF`](https://papers.nips.cc/paper/5872-efficient-and-robust-automated-machine-learning.pdf)
* 2015 | Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | PKDD | [`PDF`](https://link.springer.com/chapter/10.1007/978-3-319-23525-7_6)
* 2015 | Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization | Martin Wistua, et al. | [`PDF`](https://dl.acm.org/citation.cfm?id=2991491)
* 2015 | Joint Model Choice and Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | ICTAI | [`PDF`](http://ieeexplore.ieee.org/abstract/document/7372120/)
* 2015 | Learning Hyperparameter Optimization Initializations | Martin Wistuba, et al. | DSAA | [`PDF`](http://ieeexplore.ieee.org/abstract/document/7344817/)
* 2015 | Scalable Bayesian optimization using deep neural networks | Jasper Snoek, et al. | ACM | [`PDF`](https://dl.acm.org/citation.cfm?id=3045349)
* 2015 | Sequential Model-free Hyperparameter Tuning | Martin Wistuba, et al. | ICDM | [`PDF`](http://ieeexplore.ieee.org/abstract/document/7373431/)
* 2013 | Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms | [`PDF`](http://www.cs.ubc.ca/labs/beta/Projects/autoweka/papers/autoweka.pdf)
* 2013 | Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures | J. Bergstra | JMLR | [`PDF`](http://proceedings.mlr.press/v28/bergstra13.pdf)
* 2012 | Practical Bayesian Optimization of Machine Learning Algorithms | [`PDF`](https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf)
* 2011 | Sequential Model-Based Optimization for General Algorithm Configuration(extended version) | [`PDF`](https://www.cs.ubc.ca/~hutter/papers/10-TR-SMAC.pdf)
Evolutionary Algorithms
* 2018 | Autostacker: A Compositional Evolutionary Learning System | Boyuan Chen, et al. | arXiv | [`PDF`](https://arxiv.org/pdf/1803.00684.pdf)
* 2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR | [`PDF`](https://arxiv.org/pdf/1703.01041.pdf)
* 2016 | Automating biomedical data science through tree-based pipeline optimization | Randal S. Olson, et al. | ECAL | [`PDF`](https://arxiv.org/pdf/1601.07925.pdf)
* 2016 | Evaluation of a tree-based pipeline optimization tool for automating data science | Randal S. Olson, et al. | GECCO | [`PDF`](https://dl.acm.org/citation.cfm?id=2908918)
Lipschitz Functions
* 2017 | Global Optimization of Lipschitz functions | C´edric Malherbe, Nicolas Vayatis | arXiv | [`PDF`](https://arxiv.org/pdf/1703.02628.pdf)
Local Search
* 2009 | ParamILS: An Automatic Algorithm Configuration Framework | Frank Hutter, et al. | JAIR | [`PDF`](https://arxiv.org/pdf/1401.3492.pdf)
Meta Learning
* 2008 | Cross-Disciplinary Perspectives on Meta-Learning for Algorithm Selection | [`PDF`](https://dl.acm.org/citation.cfm?id=1456656)
* 2019 | SMARTML: A Meta Learning-Based Framework for Automated Selection and Hyperparameter Tuning for Machine Learning Algorithms | [`PDF`](http://openproceedings.org/2019/conf/edbt/EDBT19_paper_235.pdf)
Particle Swarm Optimization
* 2017 | Particle Swarm Optimization for Hyper-parameter Selection in Deep Neural Networks | Pablo Ribalta Lorenzo, et al. | GECCO | [`PDF`](https://dl.acm.org/citation.cfm?id=3071208)
* 2008 | Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines | Shih-Wei Lin, et al. | Expert Systems with Applications | [`PDF`](http://www.sciencedirect.com/science/article/pii/S0957417407003752)
Random Search
* 2016 | Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization | Lisha Li, et al. | arXiv | [`PDF`](https://arxiv.org/pdf/1603.06560.pdf)
* 2012 | Random Search for Hyper-Parameter Optimization | James Bergstra, Yoshua Bengio | JMLR | [`PDF`](http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf)
* 2011 | Algorithms for Hyper-parameter Optimization | James Bergstra, et al. | NIPS | [`PDF`](https://dl.acm.org/citation.cfm?id=2986743)
Transfer Learning
* 2016 | Efficient Transfer Learning Method for Automatic Hyperparameter Tuning | Dani Yogatama, Gideon Mann | JMLR | [`PDF`](https://pdfs.semanticscholar.org/75f2/6734972ebaffc6b43d45abd3048ef75f15a5.pdf)
* 2016 | Flexible Transfer Learning Framework for Bayesian Optimisation | Tinu Theckel Joy, et al. | PAKDD | [`PDF`](https://link.springer.com/chapter/10.1007/978-3-319-31753-3_9)
* 2016 | Hyperparameter Optimization Machines | Martin Wistuba, et al. | DSAA | [`PDF`](http://ieeexplore.ieee.org/abstract/document/7796889/)
* 2013 | Collaborative Hyperparameter Tuning | R´emi Bardenet, et al. | ICML | [`PDF`](http://proceedings.mlr.press/v28/bardenet13.pdf)
Miscellaneous
- 2018 | Accelerating Neural Architecture Search using Performance Prediction | Bowen Baker, et al. | ICLR |
PDF - 2017 | Automatic Frankensteining: Creating Complex Ensembles Autonomously | Martin Wistuba, et al. | SIAM |
PDF
Tutorials
Bayesian Optimization
- 2010 | A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning |
PDF
Meta Learning
- 2008 | Metalearning - A Tutorial |
PDF
Blog
| Type | Blog Title | Link |
|---|---|---|
| HPO | Bayesian Optimization for Hyperparameter Tuning | Link |
| Meta-Learning | Learning to learn | Link |
| Meta-Learning | Why Meta-learning is Crucial for Further Advances of Artificial Intelligence? | Link |
Books
| Year of Publication | Type | Book Title | Authors | Publisher | Link |
|---|---|---|---|---|---|
| 2009 | Meta-Learning | Metalearning - Applications to Data Mining | Brazdil, P., Giraud Carrier, C., Soares, C., Vilalta, R. | Springer | Download |
| 2019 | HPO, Meta-Learning, NAS | AutoML: Methods, Systems, Challenges | Frank Hutter, Lars Kotthoff, Joaquin Vanschoren | Download |
Projects
| Project | Type | Language | License | Link |
|---|---|---|---|---|
| AdaNet | NAS | Python | Apache-2.0 | Github |
| Advisor | HPO | Python | Apache-2.0 | Github |
| AMLA | HPO, NAS | Python | Apache-2.0 | Github |
| ATM | HPO | Python | MIT | Github |
| Auger | HPO | Python | Commercial | Homepage |
| Auto-Keras | NAS | Python | License |
Github |
| AutoML Vision | NAS | Python | Commercial | Homepage |
| AutoML Video Intelligence | NAS | Python | Commercial | Homepage |
| AutoML Natural Language | NAS | Python | Commercial | Homepage |
| AutoML Translation | NAS | Python | Commercial | Homepage |
| AutoML Tables | AutoFE, HPO | Python | Commercial | Homepage |
| auto-sklearn | HPO | Python | License |
Github |
| auto_ml | HPO | Python | MIT | Github |
| BayesianOptimization | HPO | Python | MIT | Github |
| BayesOpt | HPO | C++ | AGPL-3.0 | Github |
| comet | HPO | Python | Commercial | Homepage |
| DataRobot | HPO | Python | Commercial | Homepage |
| DEvol | NAS | Python | MIT | Github |
| DeepArchitect | NAS | Python | MIT | Github |
| Driverless AI | AutoFE | Python | Commercial | Homepage |
| FAR-HO | HPO | Python | MIT | Github |
| H2O AutoML | HPO | Python, R, Java, Scala | Apache-2.0 | Github |
| HpBandSter | HPO | Python | BSD-3-Clause | Github |
| HyperBand | HPO | Python | License |
Github |
| Hyperopt | HPO | Python | License |
Github |
| Hyperopt-sklearn | HPO | Python | License |
Github |
| Hyperparameter Hunter | HPO | Python | MIT | Github |
| Katib | HPO | Python | Apache-2.0 | Github |
| MateLabs | HPO | Python | Commercial | Github |
| Milano | HPO | Python | Apache-2.0 | Github |
| MLJAR | HPO | Python | Commercial | Homepage |
| nasbot | NAS | Python | MIT | Github |
| neptune | HPO | Python | Commercial | Homepage |
| NNI | HPO, NAS | Python | MIT | Github |
| Optunity | HPO | Python | License |
Github |
| R2.ai | HPO | Commercial | Homepage |
|
| RBFOpt | HPO | Python | License |
Github |
| RoBO | HPO | Python | BSD-3-Clause | Github |
| Scikit-Optimize | HPO | Python | License |
Github |
| SigOpt | HPO | Python | Commercial | Homepage |
| SMAC3 | HPO | Python | License |
Github |
| TPOT | AutoFE, HPO | Python | LGPL-3.0 | Github |
| TransmogrifAI | HPO | Scala | BSD-3-Clause | Github |
| Tune | HPO | Python | Apache-2.0 | Github |
| Xcessiv | HPO | Python | Apache-2.0 | Github |
| SmartML | HPO | R | GPL-3.0 | Github |
| MLBox | AutoFE, HPO | Python | BSD-3 License | Github |
| AutoAI Watson | AutoFE, HPO | Commercial | Homepage |
Slides
| Type | Slide Title | Authors | Link |
|---|---|---|---|
| AutoFE | Automated Feature Engineering for Predictive Modeling | Udyan Khurana, etc al. | Download |
| HPO | A Tutorial on Bayesian Optimization for Machine Learning | Ryan P. Adams | Download |
| HPO | Bayesian Optimisation | Gilles Louppe | Download |
全部评论 (0)
还没有任何评论哟~
