Example: Hyperparameter tuning¶

---

This example shows an advanced example on how to optimize your model's hyperparameters for multi-metric runs.

Import the breast cancer dataset from sklearn.datasets. This is a small and easy to train dataset whose goal is to predict whether a patient has breast cancer or not.

Load the data¶

In [1]:

# Import packages
from sklearn.datasets import load_breast_cancer
from optuna.distributions import IntDistribution
from atom import ATOMClassifier

# Import packages from sklearn.datasets import load_breast_cancer from optuna.distributions import IntDistribution from atom import ATOMClassifier

In [2]:

# Load the data
X, y = load_breast_cancer(return_X_y=True)

# Load the data X, y = load_breast_cancer(return_X_y=True)

Run the pipeline¶

In [3]:

# Initialize atom
atom = ATOMClassifier(X, y, n_jobs=4, verbose=2, random_state=1)

# Initialize atom atom = ATOMClassifier(X, y, n_jobs=4, verbose=2, random_state=1)

In [4]:

# Train a MultiLayerPerceptron model on two metrics
# using a custom number of hidden layers
atom.run(
    models="MLP",
    metric=["f1", "ap"],
    n_trials=10,
    est_params={"activation": "relu"},
    ht_params={
        "distributions": {
            "hidden_layer_1": IntDistribution(2, 4),
            "hidden_layer_2": IntDistribution(10, 20),
            "hidden_layer_3": IntDistribution(10, 20),
            "hidden_layer_4": IntDistribution(2, 4),
        }
    },
    errors='raise'
)

# Train a MultiLayerPerceptron model on two metrics # using a custom number of hidden layers atom.run( models="MLP", metric=["f1", "ap"], n_trials=10, est_params={"activation": "relu"}, ht_params={ "distributions": { "hidden_layer_1": IntDistribution(2, 4), "hidden_layer_2": IntDistribution(10, 20), "hidden_layer_3": IntDistribution(10, 20), "hidden_layer_4": IntDistribution(2, 4), } }, errors='raise' )

In [5]:

# For multi-metric runs, the selected best trial is the first in the Pareto front
atom.mlp.best_trial

# For multi-metric runs, the selected best trial is the first in the Pareto front atom.mlp.best_trial

In [6]:

atom.plot_pareto_front()

atom.plot_pareto_front()

In [7]:

# If you are unhappy with the results, it's possible to conitnue the study
atom.mlp.hyperparameter_tuning(n_trials=5)

# If you are unhappy with the results, it's possible to conitnue the study atom.mlp.hyperparameter_tuning(n_trials=5)

In [8]:

# The trials attribute gives an overview of the trial results
atom.mlp.trials

# The trials attribute gives an overview of the trial results atom.mlp.trials

In [9]:

# Select a custom best trial...
atom.mlp.best_trial = 2

# ...and check that the best parameters are now those in the selected trial
atom.mlp.best_params

# Select a custom best trial... atom.mlp.best_trial = 2 # ...and check that the best parameters are now those in the selected trial atom.mlp.best_params

In [10]:

# Lastly, fit the model on the complete training set 
# using the new combination of hyperparameters
atom.mlp.fit()

# Lastly, fit the model on the complete training set # using the new combination of hyperparameters atom.mlp.fit()

Analyze the results¶

In [11]:

atom.plot_trials()

atom.plot_trials()

In [12]:

atom.plot_parallel_coordinate()

atom.plot_parallel_coordinate()