preface

This article was developed in collaboration with Kabuto_hui, a member of the ISN State Key Laboratory. For more technical articles please visit, Kabuto_hui CSDN blog () link: http://blog.csdn.net/kabuto_hui

takeaway

Under the strategy of AI First, Google has successively released AI-related products and technologies. According to a new report, Google’s AutoML system writes its own machine-learning code, in some ways more efficiently than professional engineers. AutoML’s goal is not to take humans out of the development process, or to develop entirely new AI, but to keep ai at a pace to change the world. In my opinion, it is nonsense that machine programming will soon replace programmers.

Here’s a study guide.

AutoML- The origin of Automatic machine learning

At this year’s ICML 2017International Conference on Machine Learning, ECMLPKDD 2017(European Conference on Machine Learning and Principles and Practice of Knowledge Discovery In Databases, ECMLPKDD), the European Conference on Machine Learning and Principles and Practices of Database Knowledge Discovery will hold a seminar on AutoML. So what is AutoML?

Overview: What is AutoML?

Machine Learning (ML) has achieved considerable success in recent years, and more and more disciplines need to rely on it. The key to this success, however, requires human machine learning engineers to do the following:

  • Preprocessed data
  • Select the appropriate features
  • Select an appropriate model selection series
  • Optimize model hyperparameters
  • Post processing machine learning model
  • Analyze the results rigorously

As the complexity of these tasks often exceeds the capabilities of non-machine learning specialists, the rapid growth of machine learning applications has created a need for off-the-shelf machine learning methods that are simple to use and do not require specialized knowledge. We call the research field aiming at the progressive automation of Machine Learning AutoML(Automatic Machine Learning).

While its end user base is for those without specialized machine learning knowledge, AutoML offers some new tools for machine learning professionals, such as:

  • Perform architecture search for deep representations

  • Analyze the importance of hyperparameters

Following the “optimized programming” paradigm, AutoML advocates the development of flexible software packages that can be automatically instantiated in a data-driven manner.

AutoML architecture

The design of AutoML networks is the result of years of careful experimentation and refinement from the initial version of the convolutional architecture.

In AutoML, a controller neural network can propose a “sub-model” architecture, and then conduct training and quality evaluation for a specific task. The feedback to the controller is then used to improve the next round of proposals. We repeat this process thousands of times — resulting in a new architecture that is then tested and fed back to the controller to learn from. Ultimately, the controller will learn to assign high probabilities to good architectures in order to achieve greater accuracy on continuing validation data sets, with little difference for architecture space. As shown in the figure below:

If AutoML is successful, it could inspire new kinds of neural networks that can be created not just by experts, but by machine learning that benefits everyone.

An example of AutoML

AutoML aims to create software that can be used “out of the box” by ML novice. Some recent examples that might be shown:

  • AutoWEKA is a method that can select both the machine learning algorithm and its corresponding hyperparameters. Using WEKA packages, you can automatically generate good models for various data sets.
  • Deep neural networks are known to rely on their hyperparameters, and some modern optimizers are already better at setting the parameters than humans (e.g. Bergstra et al., Snoek et al.).
  • Create a model search science: A complex computer vision architecture can be automatically instantiated to obtain state-of-the-art results in 3 different tasks: face matching, facial recognition, and object recognition.

Methods used by AutoML

AutoML draws on many disciplines of machine learning, mainly:

  • Bayesian optimization
  • Regression models for structured data and big data
  • Meta learning
  • Transfer of learning
  • Combinatorial optimization

Hyperparameter optimization system for AutoML

A machine learning hyperparameter system for Bayesian optimization can promote AutoML, including:

  • Hyperopt, including the TPE algorithm

  • Algorithm Configuration based on Sequential Model (SMAC)

  • [Spearmint](https://github.com/JasperSnoek/spearmint

In addition, two packages are available for hyperparameter optimization:

  • Robo-robust Bayesian Optimization Framework

  • Python implementation of smAC3-SMAC algorithm

​

Function analysis of AutoML

In general, machine learning requires a large amount of training data, and then machine learning engineers/data scientists analyze the data and design algorithms to form training models. It takes a lot of expertise. However, if we use AutoML, it is like using a tool, we only need to pass the training data set to AutoML, then the tool will automatically generate parameters and models for us to form the training model, so that even without deep expertise in machine learning can do machine learning work.

Currently, humans are needed to use AutoML to configure parameters; In the future, AutoML Box will be generated through a large amount of artificial intelligence, and then data will be passed to AutoML Box to produce training model, as shown in the following picture:

The resources

  • AutoML

  • http://www.ml4aad.org/automl/workshops/

  • http://slides.com/streguer/automl/fullscreen#/