As we already know, machine learning is a way of automating complex problem-solving. But can machine learning itself be automated? That’s what we’ll explore in this article.
By its end, we’ll have answered that question and shown practical ways it can be accomplished.
Automated Machine Learning (AutoML)
When applying machine learning models, we’d usually do data pre-processing, feature engineering, feature extraction and, feature selection. After this, we’d select the best algorithm and tune our parameters in order to obtain the best results. AutoML is a series of concepts and techniques used to automate these processes.
Benefits of AutoML
Applying machine learning models to our problems usually requires computer science skills, domain expertise, and mathematical expertise. Getting an expert with all these skills is not always a walk in the park.
AutoML also reduces bias and errors that occur when a human being is designing the machine learning models. An organization can also reduce the cost of hiring many experts by applying AutoML in their data pipeline. AutoML also reduces the amount of time it would take to develop and test a machine learning model.
Drawbacks of AutoML
AutoML is a fairly new concept in the machine learning world. It is, therefore, important to exercise caution while applying some of the current AutoML solutions. This is because some of these technologies are still under development.
Another major challenge is the time it takes to run the AutoML models. This will really depend on the computational power of the machine we’re running. As we shall see soon, some of the AutoML solutions run well on our local machines, but some require an accelerated solution such as Google Colab.
There are two major concepts to grasp as far as AutoML is concerned: neural architecture search and transfer learning.
Neural Architecture Search
Neural architecture search is the process of automating the design of neural networks. Usually, reinforcement learning or evolutionary algorithms are used in the design of these networks. In reinforcement learning, models are punished for low accuracies and rewarded for high accuracies. Using this technique the model will always strive to obtain higher accuracies.
Several neural architecture search papers have been published such as Learning Transferable Architectures for Scalable Image Recognition, Efficient Neural Architecture Search (ENAS), and Regularized Evolution for Image Classifier Architecture Search just to mention a few.
Transfer learning, as the name suggests, is a technique where one uses pre-trained models to transfer what its learned when applying the model to a new but similar dataset. This enables us to obtain high accuracies while using less computation time and power. Neural architecture search is good for problems that require the discovering of new architectures, while transfer learning works best for problems where the datasets are similar to the ones used in pre-training models.
Let’s now look at some of the available automated machine learning solutions.
According to the official site:
It can be installed using a simple pip command:
Auto-Keras is still undergoing final testing before its final release. The official site warns that they will not be held liable for any loss incurred as a result of using the libraries on the site. This package is based on the Keras deep learning package.
In Ubuntu, a C++11 building environment and SWIG are required:
Installation via Anaconda is as follows:
It isn’t possible to run Auto-Sklearn on Windows. However, one can try some hacks such as a docker image or running via a virtual machine.
The Tree-Based Pipeline Optimization Tool (TPOT)
According to its official site:
This software is open source and is available on GitHub.
Its official site states that:
Google’s AutoML solution is not open source. Its pricing can be viewed here.
H2O is an open source distributed in-memory machine learning platform. It is available in both R and Python. This package provides support for statistical & machine learning algorithms.
Applying AutoML to Real-World Problems
Now let’s see how we’d use Auto-Keras and Auto-Sklearn to solve a real problem.
I would highly recommend running the following example on Google Colab unless we’re using a machine with high computational power. It’s also important to ensure that we’re using the GPU runtime on our Google Colab. The first step here is to install Auto-Keras on our Colab runtime.
!pip install autokeras
We’re going to run image classification on the MNIST dataset. The first step is to import that dataset and the image classifier. The dataset is imported from Keras, whereas the image classifier is imported from Auto-Keras. Since we are building a model that will recognize handwritten digits based on pre-trained models, we classify this as a supervised learning problem. We then test the accuracy of the model on images of digits that it hasn’t encountered.
from keras.datasets import mnist
from autokeras.image.image_supervised import ImageClassifier
In this example, the images and labels have already been formatted to numpy arrays, which is required in machine learning. The next step is to separate the data we just loaded into a training set and test set as shown below.
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))
Once the data has been separated into a training set and a test set, the next step is to fit the image classifier.
- Specifying Verbose as True means that the search process will be printed on the screen for us to see.
- In the fit method, time_limit refers to the time limit for the search in seconds.
- final_fit is the last training that occurs after the model has found the best architecture. Specifying retrain as true means that the weights of the model will be reinitialized.
- Printing y will show us the accuracy after evaluating the model on the test set.
clf = ImageClassifier(verbose=True)
clf.fit(x_train, y_train, time_limit=12 * 60 * 60)
clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
y = clf.evaluate(x_test, y_test)
This is all we need to classify images using Auto-Keras. Very few lines of code, and Auto-Keras will do all the heavy lifting for us.
Implementation on Auto-Sklearn is very similar to the Auto-Keras implementation above. We’ll run a similar classification using the digits dataset. First, we need to get a few imports out of the way:
- autosklearn.classification that we’ll use to load the classifier later
- sklearn.model_selection for model selection
- sklearn.datasets for loading in the dataset
- import sklearn.metrics for measuring model performance
As usual, we load in the dataset and split it into a training and test set. We then import the AutoSklearnClassifier from autosklearn.classification. Once this is done we fit the classifier to our dataset, make predictions and check the accuracy. That’s all you need to do.
X, y = sklearn.datasets.load_digits(return_X_y=True)
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, random_state=1)
automl = autosklearn.classification.AutoSklearnClassifier()
y_hat = automl.predict(X_test)
print("Score of Accuracy", sklearn.metrics.accuracy_score(y_test, y_hat))
Automated machine learning packages are still under active development. We expect to see more advancements in that arena in 2019. One can keep tabs on the progress of these packages by following the official documentation sites. One can also contribute to these packages by making a pull request on GitHub. More information and examples about Auto-Keras and Auto-Sklearn can be found on their official site.