Example: Utilities¶

---

This example shows various useful utilities that can be used to improve atom's pipelines.

The data used is a variation on the Australian weather dataset from Kaggle. You can download it from here. The goal of this dataset is to predict whether or not it will rain tomorrow training a binary classifier on target RainTomorrow.

Load the data¶

In [1]:

# Import packages
import pandas as pd
from sklearn.metrics import fbeta_score
from atom import ATOMClassifier, ATOMLoader

# Import packages import pandas as pd from sklearn.metrics import fbeta_score from atom import ATOMClassifier, ATOMLoader

In [2]:

# Load data
X = pd.read_csv("./datasets/weatherAUS.csv")

# Let's have a look
X.head()

# Load data X = pd.read_csv("./datasets/weatherAUS.csv") # Let's have a look X.head()

Out[2]:

	Location	MinTemp	MaxTemp	Rainfall	Evaporation	Sunshine	WindGustDir	WindGustSpeed	WindDir9am	WindDir3pm	...	Humidity9am	Humidity3pm	Pressure9am	Pressure3pm	Cloud9am	Cloud3pm	Temp9am	Temp3pm	RainToday	RainTomorrow
0	MelbourneAirport	18.0	26.9	21.4	7.0	8.9	SSE	41.0	W	SSE	...	95.0	54.0	1019.5	1017.0	8.0	5.0	18.5	26.0	Yes	0
1	Adelaide	17.2	23.4	0.0	NaN	NaN	S	41.0	S	WSW	...	59.0	36.0	1015.7	1015.7	NaN	NaN	17.7	21.9	No	0
2	Cairns	18.6	24.6	7.4	3.0	6.1	SSE	54.0	SSE	SE	...	78.0	57.0	1018.7	1016.6	3.0	3.0	20.8	24.1	Yes	0
3	Portland	13.6	16.8	4.2	1.2	0.0	ESE	39.0	ESE	ESE	...	76.0	74.0	1021.4	1020.5	7.0	8.0	15.6	16.0	Yes	1
4	Walpole	16.4	19.9	0.0	NaN	NaN	SE	44.0	SE	SE	...	78.0	70.0	1019.4	1018.9	NaN	NaN	17.4	18.1	No	0

5 rows × 22 columns

Use the utility attributes¶

In [3]:

atom = ATOMClassifier(X, random_state=1)
atom.clean()

# Quickly check what columns have missing values
print(f"Columns with missing values:\n{atom.nans}")

# Or what columns are categorical
print(f"\nCategorical columns: {atom.categorical}")

# Or if the dataset is scaled
print(f"\nIs the dataset scaled? {atom.scaled}")

atom = ATOMClassifier(X, random_state=1) atom.clean() # Quickly check what columns have missing values print(f"Columns with missing values:\n{atom.nans}") # Or what columns are categorical print(f"\nCategorical columns: {atom.categorical}") # Or if the dataset is scaled print(f"\nIs the dataset scaled? {atom.scaled}")

Columns with missing values:
MinTemp            637
MaxTemp            322
Rainfall          1406
Evaporation      60843
Sunshine         67816
WindGustDir       9330
WindGustSpeed     9270
WindDir9am       10013
WindDir3pm        3778
WindSpeed9am      1348
WindSpeed3pm      2630
Humidity9am       1774
Humidity3pm       3610
Pressure9am      14014
Pressure3pm      13981
Cloud9am         53657
Cloud3pm         57094
Temp9am            904
Temp3pm           2726
RainToday         1406
dtype: int64

Categorical columns: Index(['Location', 'WindGustDir', 'WindDir9am', 'WindDir3pm', 'RainToday'], dtype='object')

Is the dataset scaled? False

Use the stats method to assess changes in the dataset¶

In [4]:

# Note the number of missing values and categorical columns
atom.stats()

# Note the number of missing values and categorical columns atom.stats()

Dataset stats ==================== >>
Shape: (142193, 22)
Memory: 61.69 MB
Scaled: False
Missing values: 316559 (10.1%)
Categorical features: 5 (23.8%)
Duplicate samples: 45 (0.0%)
-------------------------------------
Train set size: 113755
Test set size: 28438
-------------------------------------
|   |   dataset |     train |      test |
| - | --------- | --------- | --------- |
| 0 |   0 (0.0) |   0 (0.0) |   0 (0.0) |
| 1 |   0 (0.0) |   0 (0.0) |   0 (0.0) |

In [5]:

# Now, let's impute and encode the dataset...
atom.impute()
atom.encode()

# ... and the values are gone
atom.stats()

# Now, let's impute and encode the dataset... atom.impute() atom.encode() # ... and the values are gone atom.stats()

Dataset stats ==================== >>
Shape: (56420, 22)
Memory: 9.93 MB
Scaled: False
Outlier values: 3203 (0.3%)
-------------------------------------
Train set size: 45075
Test set size: 11345
-------------------------------------
|   |   dataset |     train |      test |
| - | --------- | --------- | --------- |
| 0 |   0 (0.0) |   0 (0.0) |   0 (0.0) |
| 1 |   0 (0.0) |   0 (0.0) |   0 (0.0) |

Inspect feature distributions¶

In [6]:

# Compare the relationship of multiple columns with a scatter maxtrix
atom.plot_relationships(columns=slice(0, 5))

# Compare the relationship of multiple columns with a scatter maxtrix atom.plot_relationships(columns=slice(0, 5))

In [7]:

# Check which distribution fits a column best
atom.distribution(columns="Rainfall")

# Check which distribution fits a column best atom.distribution(columns="Rainfall")

Out[7]:

		Rainfall
dist	stat
beta	score	0.6506
	p_value	0.0
expon	score	0.6506
	p_value	0.0
gamma	score	0.6465
	p_value	0.0
invgauss	score	0.6257
	p_value	0.0
lognorm	score	0.6485
	p_value	0.0
norm	score	0.3807
	p_value	0.0
pearson3	score	0.6506
	p_value	0.0
triang	score	0.7191
	p_value	0.0
uniform	score	0.8914
	p_value	0.0
weibull_min	score	0.6506
	p_value	0.0
weibull_max	score	0.8896
	p_value	0.0

In [8]:

# Investigate a column's distribution
atom.plot_distribution(columns="MinTemp", distributions="beta")
atom.plot_qq(columns="MinTemp", distributions="beta")

# Investigate a column's distribution atom.plot_distribution(columns="MinTemp", distributions="beta") atom.plot_qq(columns="MinTemp", distributions="beta")

Change the data mid-pipeline¶

There are two ways to quickly transform the dataset mid-pipeline. The first way is through the property's @setter. The downside for this approach is that the transformation is not stored in atom's pipeline, so the transformation is not applied on new data. Therefore, we recommend using the second approach, through the add method.

In [9]:

# Note that we can only replace a dataframe with a new dataframe!
atom.X = atom.X.assign(AvgTemp=(atom.X["MaxTemp"] + atom.X["MinTemp"])/2)

# This will automatically update all other data attributes
assert "AvgTemp" in atom

# But it's not saved to atom's pipeline
atom.pipeline

# Note that we can only replace a dataframe with a new dataframe! atom.X = atom.X.assign(AvgTemp=(atom.X["MaxTemp"] + atom.X["MinTemp"])/2) # This will automatically update all other data attributes assert "AvgTemp" in atom # But it's not saved to atom's pipeline atom.pipeline

Out[9]:

0    Cleaner()
1    Imputer()
2    Encoder()
dtype: object

In [12]:

# Same transformation, different approach (AvgTemp is overwritten)
def transform(df):
    df["AvgTemp"] = (df.MaxTemp + df.MinTemp) / 2
    return df

atom.apply(transform)

assert "AvgTemp" in atom

# Same transformation, different approach (AvgTemp is overwritten) def transform(df): df["AvgTemp"] = (df.MaxTemp + df.MinTemp) / 2 return df atom.apply(transform) assert "AvgTemp" in atom

In [13]:

# Now the function appears in the pipeline
atom.pipeline

# Now the function appears in the pipeline atom.pipeline

Out[13]:

0                                            Cleaner()
1                                            Imputer()
2                                            Encoder()
3    FunctionTransformer(func=<function transform a...
dtype: object

Get an overview of the available models¶

In [14]:

atom.available_models()

atom.available_models()

Out[14]:

	acronym	model	estimator	module	needs_scaling	accepts_sparse	has_validation	supports_engines
0	AdaB	AdaBoost	AdaBoostClassifier	sklearnensemble	False	True	False	sklearn
1	Bag	Bagging	BaggingClassifier	sklearnensemble	False	True	False	sklearn
2	BNB	BernoulliNB	BernoulliNB	sklearnnaive_bayes	False	True	False	sklearn, cuml
3	CatB	CatBoost	CatBoostClassifier	catboostcatboost	True	True	True	catboost
4	CatNB	CategoricalNB	CategoricalNB	sklearnnaive_bayes	False	True	False	sklearn, cuml
5	CNB	ComplementNB	ComplementNB	sklearnnaive_bayes	False	True	False	sklearn, cuml
6	Tree	DecisionTree	DecisionTreeClassifier	sklearntree	False	True	False	sklearn
7	Dummy	Dummy	DummyClassifier	sklearndummy	False	False	False	sklearn
8	ETree	ExtraTree	ExtraTreeClassifier	sklearntree	False	True	False	sklearn
9	ET	ExtraTrees	ExtraTreesClassifier	sklearnensemble	False	True	False	sklearn
10	GNB	GaussianNB	GaussianNB	sklearnnaive_bayes	False	False	False	sklearn, cuml
11	GP	GaussianProcess	GaussianProcessClassifier	sklearngaussian_process	False	False	False	sklearn
12	GBM	GradientBoosting	GradientBoostingClassifier	sklearnensemble	False	True	False	sklearn
13	hGBM	HistGradientBoosting	HistGradientBoostingClassifier	sklearnensemble	False	False	False	sklearn
14	KNN	KNearestNeighbors	KNeighborsClassifier	sklearnneighbors	True	True	False	sklearn, sklearnex, cuml
15	LGB	LightGBM	LGBMClassifier	lightgbmlightgbm.sklearn	True	True	True	lightgbm
16	LDA	LinearDiscriminantAnalysis	LinearDiscriminantAnalysis	sklearndiscriminant_analysis	False	False	False	sklearn
17	lSVM	LinearSVM	LinearSVC	sklearnsvm	True	True	False	sklearn, cuml
18	LR	LogisticRegression	LogisticRegression	sklearnlinear_model	True	True	False	sklearn, sklearnex, cuml
19	MLP	MultiLayerPerceptron	MLPClassifier	sklearnneural_network	True	True	True	sklearn
20	MNB	MultinomialNB	MultinomialNB	sklearnnaive_bayes	False	True	False	sklearn, cuml
21	PA	PassiveAggressive	PassiveAggressiveClassifier	sklearnlinear_model	True	True	True	sklearn
22	Perc	Perceptron	Perceptron	sklearnlinear_model	True	False	True	sklearn
23	QDA	QuadraticDiscriminantAnalysis	QuadraticDiscriminantAnalysis	sklearndiscriminant_analysis	False	False	False	sklearn
24	RNN	RadiusNearestNeighbors	RadiusNeighborsClassifier	sklearnneighbors	True	True	False	sklearn
25	RF	RandomForest	RandomForestClassifier	sklearnensemble	False	True	False	sklearn, sklearnex, cuml
26	Ridge	Ridge	RidgeClassifier	sklearnlinear_model	True	True	False	sklearn, sklearnex, cuml
27	SGD	StochasticGradientDescent	SGDClassifier	sklearnlinear_model	True	True	True	sklearn
28	SVM	SupportVectorMachine	SVC	sklearnsvm	True	True	False	sklearn, sklearnex, cuml
29	XGB	XGBoost	XGBClassifier	xgboostxgboost	True	True	True	xgboost

Use a custom metric¶

In [15]:

atom.verbose = 1

# Define a custom metric
def f2(y_true, y_pred):
    return fbeta_score(y_true, y_pred, beta=2)

# Use the greater_is_better, needs_proba and needs_threshold parameters if necessary
atom.run(models="LR", metric=f2)

atom.verbose = 1 # Define a custom metric def f2(y_true, y_pred): return fbeta_score(y_true, y_pred, beta=2) # Use the greater_is_better, needs_proba and needs_threshold parameters if necessary atom.run(models="LR", metric=f2)

Training ========================= >>
Models: LR
Metric: f2


Results for LogisticRegression:
Fit ---------------------------------------------
Train evaluation --> f2: 0.5693
Test evaluation --> f2: 0.5709
Time elapsed: 0.299s
-------------------------------------------------
Total time: 0.299s


Final results ==================== >>
Total time: 0.301s
-------------------------------------
LogisticRegression --> f2: 0.5709

Customize the estimator's parameters¶

In [16]:

# You can use the est_params parameter to customize the estimator
# Let's run AdaBoost using LR instead of a decision tree as base estimator
atom.run("AdaB", est_params={"base_estimator": atom.lr.estimator})

# You can use the est_params parameter to customize the estimator # Let's run AdaBoost using LR instead of a decision tree as base estimator atom.run("AdaB", est_params={"base_estimator": atom.lr.estimator})

Training ========================= >>
Models: AdaB
Metric: f2


Results for AdaBoost:
Fit ---------------------------------------------
Train evaluation --> f2: 0.5579
Test evaluation --> f2: 0.5639
Time elapsed: 2.248s
-------------------------------------------------
Total time: 2.248s


Final results ==================== >>
Total time: 2.250s
-------------------------------------
AdaBoost --> f2: 0.5639

In [17]:

atom.adab.estimator

atom.adab.estimator

Out[17]:

AdaBoostClassifier(base_estimator=LogisticRegression(n_jobs=1, random_state=1),
                   random_state=1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [19]:

# Note that parameters specified by est_params are not optimized in the BO
atom.run(
    models="Tree",
    n_trials=10,
    est_params={
        "criterion": "gini",
        "splitter": "best",
        "min_samples_leaf": 1,
        "ccp_alpha": 0.035,
    },
    verbose=2,
)

# Note that parameters specified by est_params are not optimized in the BO atom.run( models="Tree", n_trials=10, est_params={ "criterion": "gini", "splitter": "best", "min_samples_leaf": 1, "ccp_alpha": 0.035, }, verbose=2, )

Training ========================= >>
Models: Tree
Metric: f2


Running hyperparameter tuning for DecisionTree...
| trial | max_depth | min_samples_split | max_features |      f2 | best_f2 | time_trial | time_ht |    state |
| ----- | --------- | ----------------- | ------------ | ------- | ------- | ---------- | ------- | -------- |
| 0     |        13 |                12 |          0.5 |  0.4362 |  0.4362 |     1.791s |  1.791s | COMPLETE |
| 1     |        14 |                16 |         log2 |  0.4729 |  0.4729 |     1.375s |  3.166s | COMPLETE |
| 2     |        16 |                13 |          0.8 |  0.4626 |  0.4729 |     1.631s |  4.798s | COMPLETE |
| 3     |         9 |                 6 |         None |  0.4903 |  0.4903 |     1.588s |  6.386s | COMPLETE |
| 4     |         5 |                 2 |         log2 |  0.4889 |  0.4903 |     1.497s |  7.883s | COMPLETE |
| 5     |         1 |                15 |          0.5 |  0.4953 |  0.4953 |     1.567s |  9.450s | COMPLETE |
| 6     |        15 |                 9 |         sqrt |  0.5004 |  0.5004 |     1.640s | 11.091s | COMPLETE |
| 7     |        13 |                20 |         None |  0.5004 |  0.5004 |     1.833s | 12.924s | COMPLETE |
| 8     |         3 |                19 |          0.5 |  0.4936 |  0.5004 |     1.506s | 14.430s | COMPLETE |
| 9     |        15 |                20 |         sqrt |  0.4762 |  0.5004 |     1.575s | 16.005s | COMPLETE |
Hyperparameter tuning ---------------------------
Best trial --> 6
Best parameters:
 --> max_depth: 15
 --> min_samples_split: 9
 --> max_features: sqrt
Best evaluation --> f2: 0.5004
Time elapsed: 16.005s
Fit ---------------------------------------------
Train evaluation --> f2: 0.4925
Test evaluation --> f2: 0.4925
Time elapsed: 0.167s
-------------------------------------------------
Total time: 16.172s


Final results ==================== >>
Total time: 16.199s
-------------------------------------
DecisionTree --> f2: 0.4925

Save & load¶

Note that both instances need to be initialized with the same data and use the same metric for model training to be able to merge.

In [20]:

# Save the atom instance as a pickle
# Use save_data=False to save the instance without the data
atom.save("atom", save_data=False)

# Save the atom instance as a pickle # Use save_data=False to save the instance without the data atom.save("atom", save_data=False)

ATOMClassifier successfully saved.

In [21]:

# Load the instance again with ATOMLoader
# No need to store the transformed data, providing the original dataset to
# the loader automatically transforms it through all the steps in the pipeline
atom_2 = ATOMLoader("atom", data=(X,), verbose=2)

# Load the instance again with ATOMLoader # No need to store the transformed data, providing the original dataset to # the loader automatically transforms it through all the steps in the pipeline atom_2 = ATOMLoader("atom", data=(X,), verbose=2)

Cleaning the data...
 --> Label-encoding the target column.
Imputing missing values...
 --> Dropping 637 samples due to missing values in feature MinTemp.
 --> Dropping 322 samples due to missing values in feature MaxTemp.
 --> Dropping 1406 samples due to missing values in feature Rainfall.
 --> Dropping 60843 samples due to missing values in feature Evaporation.
 --> Dropping 67816 samples due to missing values in feature Sunshine.
 --> Dropping 9330 samples due to missing values in feature WindGustDir.
 --> Dropping 9270 samples due to missing values in feature WindGustSpeed.
 --> Dropping 10013 samples due to missing values in feature WindDir9am.
 --> Dropping 3778 samples due to missing values in feature WindDir3pm.
 --> Dropping 1348 samples due to missing values in feature WindSpeed9am.
 --> Dropping 2630 samples due to missing values in feature WindSpeed3pm.
 --> Dropping 1774 samples due to missing values in feature Humidity9am.
 --> Dropping 3610 samples due to missing values in feature Humidity3pm.
 --> Dropping 14014 samples due to missing values in feature Pressure9am.
 --> Dropping 13981 samples due to missing values in feature Pressure3pm.
 --> Dropping 53657 samples due to missing values in feature Cloud9am.
 --> Dropping 57094 samples due to missing values in feature Cloud3pm.
 --> Dropping 904 samples due to missing values in feature Temp9am.
 --> Dropping 2726 samples due to missing values in feature Temp3pm.
 --> Dropping 1406 samples due to missing values in feature RainToday.
Encoding categorical columns...
 --> LeaveOneOut-encoding feature Location. Contains 26 classes.
 --> LeaveOneOut-encoding feature WindGustDir. Contains 16 classes.
 --> LeaveOneOut-encoding feature WindDir9am. Contains 16 classes.
 --> LeaveOneOut-encoding feature WindDir3pm. Contains 16 classes.
 --> Ordinal-encoding feature RainToday. Contains 2 classes.
ATOMClassifier successfully loaded.

In [24]:

# Create a separate instance with its own branch and model
atom_3 = ATOMClassifier(X, verbose=0, random_state=1)
atom_3.branch.name = "lightgbm"
atom_3.impute()
atom_3.encode()
atom_3.run("LGB", metric=f2)

# Create a separate instance with its own branch and model atom_3 = ATOMClassifier(X, verbose=0, random_state=1) atom_3.branch.name = "lightgbm" atom_3.impute() atom_3.encode() atom_3.run("LGB", metric=f2)

In [25]:

# Merge the instances
atom_2.merge(atom_3)

# Merge the instances atom_2.merge(atom_3)

Merging instances...
 --> Merging branch lightgbm.
 --> Merging model LGB.
 --> Merging attributes.

In [26]:

# Note that it now contains both branches and all models
atom_2

# Note that it now contains both branches and all models atom_2

Out[26]:

ATOMClassifier
 --> Branches:
   --> master !
   --> lightgbm
 --> Models: LR, AdaB, Tree, LGB
 --> Metric: f2
 --> Errors: 0

In [27]:

atom_2.results

atom_2.results

Out[27]:

	score_ht	time_ht	score_train	score_test	time_fit	time
LR	NaN	NaN	0.5693	0.5709	0.299272	0.299272
AdaB	NaN	NaN	0.5579	0.5639	2.247532	2.247532
Tree	0.500381	16.00508	0.4925	0.4925	0.167152	16.172232
LGB	NaN	NaN	0.6578	0.5909	3.562652	3.562652