Dummy
When doing supervised learning, a simple sanity check consists of comparing one's estimator against simple rules of thumb. The prediction methods completely ignore the input data. Do not use this model for real problems. Use it only as a simple baseline to compare with other models.
Corresponding estimators are:
- DummyClassifier for classification tasks.
- DummyRegressor for regression tasks.
Read more in sklearn's documentation.
See Also
Example
>>> from atom import ATOMClassifier
>>> from sklearn.datasets import load_breast_cancer
>>> X, y = load_breast_cancer(return_X_y=True, as_frame=True)
>>> atom = ATOMClassifier(X, y, random_state=1)
>>> atom.run(models="Dummy", metric="f1", verbose=2)
Training ========================= >>
Models: Dummy
Metric: f1
Results for Dummy:
Fit ---------------------------------------------
Train evaluation --> f1: 0.7709
Test evaluation --> f1: 0.7717
Time elapsed: 0.030s
-------------------------------------------------
Time: 0.030s
Final results ==================== >>
Total time: 0.033s
-------------------------------------
Dummy --> f1: 0.7717
Hyperparameters
Attributes
Data attributes
| Attributes | pipeline: Pipeline Pipeline of transformers.
mapping: dict[str, dict[str, int | float]]Models that used automated feature scaling have the scaler added. Tip Use the plot_pipeline method to visualize the pipeline. Encoded values and their respective mapped values.
dataset: pd.DataFrameThe column name is the key to its mapping dictionary. Only for columns mapped to a single column (e.g., Ordinal, Leave-one-out, etc...). Complete data set.
train: pd.DataFrameTraining set.
test: pd.DataFrameTest set.
X: pd.DataFrameFeature set.
y: pd.Series | pd.DataFrameTarget column(s).
X_train: pd.DataFrameFeatures of the training set.
y_train: pd.Series | pd.DataFrameTarget column of the training set.
X_test: pd.DataFrameFeatures of the test set.
y_test: pd.Series | pd.DataFrameTarget column(s) of the test set.
X_holdout: pd.DataFrame | NoneFeatures of the holdout set.
y_holdout: pd.Series | pd.DataFrame | NoneTarget column of the holdout set.
shape: tuple[Int, Int]Shape of the dataset (n_rows, n_columns).
columns: pd.IndexName of all the columns.
n_columns: intNumber of columns.
features: pd.IndexName of the features.
n_features: intNumber of features.
target: str | list[str]Name of the target column(s).
|
Utility attributes
| Attributes | name: str Name of the model.
run: RunUse the property's Mlflow run corresponding to this model.
study: StudyThis property is only available for models that with mlflow tracking enabled. Optuna study used for hyperparameter tuning.
trials: pd.DataFrameThis property is only available for models that ran hyperparameter tuning. Overview of the trials' results.
best_trial: FrozenTrialThis property is only available for models that ran hyperparameter tuning. All durations are in seconds. Columns include:
Trial that returned the highest score.
best_params: dict[str, Any]For multi-metric runs, the best trial is the trial that
performed best on the main metric. Use the property's Estimator's parameters in the best trial.
estimator: PredictorThis property is only available for models that ran hyperparameter tuning. Estimator fitted on the training set.
bootstrap: pd.DataFrameOverview of the bootstrapping scores.
results: pd.SeriesThe dataframe has shape=(n_bootstrap, metric) and shows the
score obtained by every bootstrapped sample for every metric.
Using Overview of the model results.
feature_importance: pd.SeriesAll durations are in seconds. Possible values include:
Normalized feature importance scores.
The sum of importances for all features is 1. The scores are
extracted from the estimator's |
Methods
The plots can be called directly from the model. The remaining utility methods can be found hereunder.
| bootstrapping | Apply a bootstrap algorithm. |
| calibrate | Calibrate the model. |
| canvas | Create a figure with multiple plots. |
| clear | Reset attributes and clear cache from the model. |
| create_app | Create an interactive app to test model predictions. |
| create_dashboard | Create an interactive dashboard to analyze the model. |
| cross_validate | Evaluate the model using cross-validation. |
| decision_function | Get confidence scores on new data or existing rows. |
| evaluate | Get the model's scores for the provided metrics. |
| export_pipeline | Export the transformer pipeline with final estimator. |
| fit | Fit and validate the model. |
| full_train | Train the estimator on the complete dataset. |
| get_best_threshold | Get the threshold that maximizes the ROC curve. |
| get_tags | Get the model's tags. |
| hyperparameter_tuning | Run the hyperparameter tuning algorithm. |
| inverse_transform | Inversely transform new data through the pipeline. |
| predict | Get predictions on new data or existing rows. |
| predict_log_proba | Get class log-probabilities on new data or existing rows. |
| predict_proba | Get class probabilities on new data or existing rows. |
| register | Register the model in mlflow's model registry. |
| reset_aesthetics | Reset the plot aesthetics to their default values. |
| save_estimator | Save the estimator to a pickle file. |
| score | Get a metric score on new data. |
| serve | Serve the model as rest API endpoint for inference. |
| transform | Transform new data through the pipeline. |
| update_layout | Update the properties of the plot's layout. |
| update_traces | Update the properties of the plot's traces. |
Take bootstrapped samples from the training set and test them on the test set to get a distribution of the model's results.
| Parameters | n_bootstrap: int
umber of bootstrapped samples to fit on.
reset: bool, default=False
Whether to start a new run or continue the existing one.
|
Applies probability calibration on the model. The estimator
is trained via cross-validation on a subset of the training
data, using the rest to fit the calibrator. The new classifier
will replace the estimator attribute. If there is an active
mlflow experiment, a new run is started using the name
[model_name]_calibrate. Since the estimator changed, the
model is cleared. Only for classifiers.
| Parameters | **kwargs
Additional keyword arguments for sklearn's CCV. Using
cv="prefit" will use the trained model and use the test
set for calibration. Use this only if you have another,
independent set for testing (holdout set).
|
This @contextmanager allows you to draw many plots in one
figure. The default option is to add two plots side by side.
See the user guide for an example.
| Parameters | rows: int, default=1
Number of plots in length.
cols: int, default=2
Number of plots in width.
sharex: bool, default=False
If True, hide the label and ticks from non-border subplots
on the x-axis.
sharey: bool, default=False
If True, hide the label and ticks from non-border subplots
on the y-axis.
hspace: float, default=0.05
Space between subplot rows in normalized plot coordinates.
The spacing is relative to the figure's size.
vspace: float, default=0.07
Space between subplot cols in normalized plot coordinates.
The spacing is relative to the figure's size.
title: str, dict or None, default=None
Title for the plot.
legend: bool, str or dict, default="out"
Legend for the plot. See the user guide for
an extended description of the choices.
figsize: tuple or None, default=None
Figure's size in pixels, format as (x, y). If None, it
adapts the size to the number of plots in the canvas.
filename: str, Path or None, default=None
Save the plot using this name. Use "auto" for automatic
naming. The type of the file depends on the provided name
(.html, .png, .pdf, etc...). If display: bool, default=Truefilename has no file type,
the plot is saved as html. If None, the plot is not saved.
Whether to render the plot.
|
| Yields | go.Figure
Plot object.
|
Reset certain model attributes to their initial state, deleting potentially large data arrays. Use this method to free some memory before saving the instance. The affected attributes are:
- In-training validation scores
- Cached predictions.
- Shap values
- App instance
- Dashboard instance
- Calculated holdout data sets
Demo your machine learning model with a friendly web interface.
This app launches directly in the notebook or on an external
browser page. The created Interface instance can be accessed
through the app attribute.
| Parameters | **kwargs
Additional keyword arguments for the Interface instance
or the Interface.launch method.
|
ATOM uses the explainerdashboard package to provide a quick and easy way to analyze and explain the predictions and workings of the model. The dashboard allows you to investigate SHAP values, permutation importances, interaction effects, partial dependence plots, all kinds of performance plots, and even individual decision trees.
By default, the dashboard renders in a new tab in your default
browser, but if preferable, you can render it inside the
notebook using the mode="inline" parameter. The created
ExplainerDashboard instance can be accessed through the
dashboard attribute. This method is not available for
multioutput tasks.
Note
Plots displayed by the dashboard are not created by ATOM and can differ from those retrieved through this package.
| Parameters | rows: hashable, segment, sequence or dataframe, default="test"
Selection of rows to get the
report from.
filename: str, Path or None, default=None
Filename or pathlib.Path of the file to save. None to not
save anything.
**kwargs
Additional keyword arguments for the ExplainerDashboard
instance.
|
This method cross-validates the whole pipeline on the complete
dataset. Use it to assess the robustness of the model's
performance. If the scoring method is not specified in kwargs,
it uses atom's metric.
| Parameters | **kwargs
Additional keyword arguments for one of these functions.
|
| Returns | pd.DataFrame
Overview of the results.
|
New data is first transformed through the model's pipeline.
Transformers that are only applied on the training set are
skipped. The estimator must have a decision_function method.
Read more in the user guide.
| Parameters | X: hashable, segment, sequence or dataframe-like
Selection of rows or feature
set with shape=(n_samples, n_features) to make predictions
on.
verbose: int or None, default=None
Verbosity level for the transformers in the pipeline. If
None, it uses the pipeline's verbosity.
|
| Returns | series or dataframe
Predicted confidence scores with shape=(n_samples,) for
binary classification tasks (log likelihood ratio of the
positive class) or shape=(n_samples, n_classes) for
multiclass classification tasks.
|
Tip
Use the self-get_best_threshold or plot_threshold
method to determine a suitable value for the threshold
parameter.
| Parameters | metric: str, func, scorer, sequence or None, default=None
Metrics to calculate. If None, a selection of the most
common metrics per task are used.
rows: hashable, segment, sequence or dataframe, default="test"
Selection of rows to calculate
metric on.
threshold: float or sequence, default=0.5
Threshold between 0 and 1 to convert predicted probabilities
to class labels. Only used when:
For multilabel classification tasks, it's possible to provide a sequence of thresholds (one per target column, as returned by the get_best_threshold method). If float, the same threshold is applied to all target columns. |
| Returns | pd.Series
Scores of the model.
|
The returned pipeline is already fitted on the training set. Note that if the model used automated feature scaling, the Scaler is added to the pipeline.
| Returns | Pipeline
Current branch as a sklearn-like Pipeline object.
|
The estimator is fitted using the best hyperparameters found during hyperparameter tuning. Afterwards, the estimator is evaluated on the test set. Only use this method to re-fit the model after having continued the study.
In some cases, it might be desirable to use all available data
to train a final model. Note that doing this means that the
estimator can no longer be evaluated on the test set. The newly
retrained estimator will replace the estimator attribute. If
there is an active mlflow experiment, a new run is started
with the name [model_name]_full_train. Since the estimator
changed, the model is cleared.
Warning
Although the model is trained on the complete dataset, the
pipeline is not. To get a fully trained pipeline, use:
pipeline = atom.export_pipeline().fit(atom.X, atom.y).
Only available for models with a predict_proba method in a
binary or multilabel classification task.
| Parameters | rows: hashable, segment, sequence or dataframe
Selection of rows on which to
calculate the threshold.
|
| Returns | float or list
The best threshold or list of thresholds for multilabel tasks.
|
Return class parameters that provide general information about the model's characteristics.
| Returns | dict
Model's tags.
|
Search for the best combination of hyperparameters. The function to optimize is evaluated either with a K-fold cross-validation on the training set or using a random train and validation split every trial. Use this method to continue the optimization.
| Parameters | n_trials: int
Number of trials for the hyperparameter tuning.
reset: bool, default=False
Whether to start a new study or continue the existing one.
|
Transformers that are only applied on the training set are
skipped. The rest should all implement an inverse_transform
method. If only X or only y is provided, it ignores
transformers that require the other parameter. This can be
of use to, for example, inversely transform only the target
column. If called from a model that used automated feature
scaling, the scaling is inverted as well.
New data is first transformed through the model's pipeline.
Transformers that are only applied on the training set are
skipped. The estimator must have a predict method.
Read more in the user guide.
| Parameters | X: hashable, segment, sequence or dataframe-like
Selection of rows or feature
set with shape=(n_samples, n_features) to make predictions
on.
inverse: bool, default=True
Whether to inversely transform the output through the
pipeline. This doesn't affect the predictions if there are
no transformers in the pipeline or if the transformers have
no verbose: int or None, default=Noneinverse_transform method or don't apply to y.
Verbosity level for the transformers in the pipeline. If
None, it uses the pipeline's verbosity.
|
| Returns | series or dataframe
Predictions with shape=(n_samples,) or shape=(n_samples,
n_targets) for multioutput tasks.
|
New data is first transformed through the model's pipeline.
Transformers that are only applied on the training set are
skipped. The estimator must have a predict_log_proba method.
Read more in the user guide.
| Parameters | X: hashable, segment, sequence or dataframe-like
Selection of rows or feature
set with shape=(n_samples, n_features) to make predictions
on.
verbose: int or None, default=None
Verbosity level for the transformers in the pipeline. If
None, it uses the pipeline's verbosity.
|
| Returns | dataframe
Predicted class log-probabilities with shape=(n_samples,
n_classes) or shape=(n_samples * n_classes, n_targets) with
a multiindex format for multioutput tasks.
|
New data is first transformed through the model's pipeline.
Transformers that are only applied on the training set are
skipped. The estimator must have a predict_proba method.
Read more in the user guide.
| Parameters | X: hashable, segment, sequence or dataframe-like
Selection of rows or feature
set with shape=(n_samples, n_features) to make predictions
on.
verbose: int or None, default=None
Verbosity level for the transformers in the pipeline. If
None, it uses the pipeline's verbosity.
|
| Returns | dataframe
Predicted class probabilities with shape=(n_samples,
n_classes) or shape=(n_samples * n_classes, n_targets) with
a multiindex format for multioutput tasks.
|
This method is only available when model tracking is enabled using one of the following URI schemes: databricks, http, https, postgresql, mysql, sqlite, mssql.
| Parameters | filename: str or Path, default="auto"
Filename or pathlib.Path of the file to save. Use
"auto" for automatic naming.
|
New data is first transformed through the model's pipeline. Transformers that are only applied on the training set are skipped.
Read more in the user guide.
Info
If the metric parameter is left to its default value, the
method returns atom's metric score, not the metric returned
by sklearn's score method for estimators.
| Parameters | X: hashable, segment, sequence or dataframe-like
Selection of rows or feature
set with shape=(n_samples, n_features) to make predictions
on.
y: int, str, sequence, dataframe-like or None, default=None
Target column(s) corresponding to metric: str, func, scorer or None, default=NoneX.
Metric to calculate. Choose from any of sklearn's scorers,
a function with signature sample_weight: sequence or None, default=Nonemetric(y_true, y_pred) -> score
or a scorer object. If None, it uses atom's metric (the main
metric for multi-metric runs).
Sample weights corresponding to verbose: int or None, default=Noney.
Verbosity level for the transformers in the pipeline. If
None, it uses the pipeline's verbosity.
|
| Returns | float
Metric score of X with respect to y.
|
The complete pipeline is served with the model. The inference
data must be supplied as json to the HTTP request, e.g.
requests.get("http://127.0.0.1:8000/", json=X.to_json()).
The deployment is done on a ray cluster. The default host
and port parameters deploy to localhost.
Tip
Use import ray; ray.serve.shutdown() to close the
endpoint after finishing.
Transformers that are only applied on the training set are
skipped. If only X or only y is provided, it ignores
transformers that require the other parameter. This can be
of use to, for example, transform only the target column. If
called from a model that used automated feature scaling, the
data is scaled as well.
Recursively update the structure of the original layout with the values in the arguments.
| Parameters | **kwargs
Keyword arguments for the figure's update_layout method.
|
Recursively update the structure of the original traces with the values in the arguments.
| Parameters | **kwargs
Keyword arguments for the figure's update_traces method.
|