Skip to content

Lasso


Lassoneeds scalingaccept sparsesupports acceleration

Linear least squares with l1 regularization.

Corresponding estimators are:

  • Lasso for regression tasks.

Read more in sklearn's documentation.


See Also

ElasticNet

Linear Regression with elasticnet regularization.

OrdinaryLeastSquares

Linear Regression.

Ridge

Linear least squares with l2 regularization.


Example

>>> from atom import ATOMRegressor
>>> from sklearn.datasets import fetch_california_housing

>>> X, y = fetch_california_housing(return_X_y=True)

>>> atom = ATOMRegressor(X, y)
>>> atom.run(models="Lasso", metric="r2", verbose=2)

Training ========================= >>
Models: Lasso
Metric: r2


Results for Lasso:
Fit ---------------------------------------------
Train evaluation --> r2: 0.6681
Test evaluation --> r2: 0.6466
Time elapsed: 0.020s
-------------------------------------------------
Total time: 0.020s


Final results ==================== >>
Total time: 0.021s
-------------------------------------
Lasso --> r2: 0.6466



Hyperparameters

Parametersalpha
FloatDistribution(high=10.0, log=True, low=0.001, step=None)
selection
CategoricalDistribution(choices=('cyclic', 'random'))

Parametersalpha
FloatDistribution(high=10.0, log=True, low=0.001, step=None)
selection
CategoricalDistribution(choices=('cyclic', 'random'))

Parametersalpha
FloatDistribution(high=10.0, log=True, low=0.001, step=None)
selection
CategoricalDistribution(choices=('cyclic', 'random'))

Parametersalpha
FloatDistribution(high=10.0, log=True, low=0.001, step=None)
selection
CategoricalDistribution(choices=('cyclic', 'random'))





Attributes

Data attributes

Attributespipeline: pd.Series
Transformers fitted on the data.

Models that used automated feature scaling have the scaler added. Use this attribute only to access the individual instances. To visualize the pipeline, use the plot_pipeline method.

mapping: dict
Encoded values and their respective mapped values.

The column name is the key to its mapping dictionary. Only for columns mapped to a single column (e.g. Ordinal, Leave-one-out, etc...).

dataset: dataframe
Complete data set.
train: dataframe
Training set.
test: dataframe
Test set.
X: dataframe
Feature set.
y: series | dataframe
Target column(s).
X_train: dataframe
Features of the training set.
y_train: series | dataframe
Target column(s) of the training set.
X_test: dataframe
Features of the test set.
y_test: series | dataframe
Target column(s) of the test set.
shape: tuple[int, int]
Shape of the dataset (n_rows, n_columns).
columns: series
Name of all the columns.
n_columns: int
Number of columns.
features: series
Name of the features.
n_features: int
Number of features.
target: str | list[str]
Name of the target column(s).


Utility attributes

Attributesname: str
Name of the model.

Use the property's @setter to change the model's name. The acronym always stays at the beginning of the model's name. If the model is being tracked by mlflow, the name of the corresponding run also changes.

study: Study | None
Optuna study used for hyperparameter tuning.
trials: pd.DataFrame | None
Overview of the trials' results.

All durations are in seconds. Columns include:

  • params: Parameters used for this trial.
  • estimator: Estimator used for this trial.
  • score: Objective score(s) of the trial.
  • time_trial: Duration of the trial.
  • time_ht: Duration of the hyperparameter tuning.
  • state: Trial's state (COMPLETE, PRUNED, FAIL).
best_trial: Trial | None
Trial that returned the highest score.

For multi-metric runs, the best trial is the trial that performed best on the main metric. Use the property's @setter to change the best trial. See here an example.

best_params: dict
Hyperparameters used by the best trial.
score_ht: float | list[float] | None
Metric score obtained by the best trial.
time_ht: int | None
Duration of the hyperparameter tuning (in seconds).
estimator: Predictor
Estimator fitted on the training set.
score_train: float | list[float]
Metric score on the training set.
score_test: float | list[float]
Metric score on the test set.
score_holdout: float | list[float]
Metric score on the holdout set.
time_fit: int
Duration of the model fitting on the train set (in seconds).
bootstrap: pd.DataFrame | None
Overview of the bootstrapping scores.

The dataframe has shape=(n_bootstrap, metric) and shows the score obtained by every bootstrapped sample for every metric. Using atom.bootstrap.mean() yields the same values as score_bootstrap.

score_bootstrap: float | list[float] | None
Mean metric score on the bootstrapped samples.
time_bootstrap: int | None
Duration of the bootstrapping (in seconds).
time: int
Total duration of the run (in seconds).
feature_importance: pd.Series | None
Normalized feature importance scores.

The sum of importances for all features is 1. The scores are extracted from the estimator's scores_, coef_ or feature_importances_ attribute, checked in that order. Returns None for estimators without any of those attributes.

results: pd.Series
Overview of the training results.

All durations are in seconds. Values include:

  • score_ht: Score obtained by the hyperparameter tuning.
  • time_ht: Duration of the hyperparameter tuning.
  • score_train: Metric score on the train set.
  • score_test: Metric score on the test set.
  • time_fit: Duration of the model fitting on the train set.
  • score_bootstrap: Mean score on the bootstrapped samples.
  • time_bootstrap: Duration of the bootstrapping.
  • time: Total duration of the run.


Prediction attributes

The prediction attributes are not calculated until the attribute is called for the first time. This mechanism avoids having to calculate attributes that are never used, saving time and memory.

Attributespredict_train: series | dataframe
Class predictions on the training set.

The shape of the output depends on the task:

  • (n_samples,) for non-multioutput tasks.
  • (n_samples, n_targets) for multioutput tasks.
predict_test: series | dataframe
Class predictions on the test set.

The shape of the output depends on the task:

  • (n_samples,) for non-multioutput tasks.
  • (n_samples, n_targets) for multioutput tasks.
predict_holdout: series | dataframe | None
Class predictions on the holdout set.

The shape of the output depends on the task:

  • (n_samples,) for non-multioutput tasks.
  • (n_samples, n_targets) for multioutput tasks.



Methods

The plots and prediction methods can be called directly from the model. The remaining utility methods can be found hereunder.

bootstrappingApply a bootstrap algorithm.
calibrateCalibrate the model.
clearReset attributes and clear cache from the model.
create_appCreate an interactive app to test model predictions.
create_dashboardCreate an interactive dashboard to analyze the model.
cross_validateEvaluate the model using cross-validation.
evaluateGet the model's scores for the provided metrics.
export_pipelineExport the model's pipeline to a sklearn-like object.
fitFit and validate the model.
full_trainTrain the estimator on the complete dataset.
get_best_thresholdGet the threshold that maximizes the ROC curve.
hyperparameter_tuningRun the hyperparameter tuning algorithm.
inverse_transformInversely transform new data through the pipeline.
save_estimatorSave the estimator to a pickle file.
serveServe the model as rest API endpoint for inference.
registerRegister the model in mlflow's model registry.
transformTransform new data through the pipeline.


method bootstrapping(n_bootstrap, reset=False)[source]
Apply a bootstrap algorithm.

Take bootstrapped samples from the training set and test them on the test set to get a distribution of the model's results.

Parametersn_bootstrap: int
umber of bootstrapped samples to fit on.

reset: bool, default=False
Whether to start a new run or continue the existing one.



method calibrate(**kwargs)[source]
Calibrate the model.

Applies probability calibration on the model. The estimator is trained via cross-validation on a subset of the training data, using the rest to fit the calibrator. The new classifier will replace the estimator attribute. If there is an active mlflow experiment, a new run is started using the name [model_name]_calibrate. Since the estimator changed, the model is cleared. Only for classifiers.

Parameters**kwargs
Additional keyword arguments for sklearn's CCV. Using cv="prefit" will use the trained model and fit the calibrator on the test set. Use this only if you have another, independent set for testing.



method clear()[source]
Reset attributes and clear cache from the model.

Reset certain model attributes to their initial state, deleting potentially large data arrays. Use this method to free some memory before saving the instance. The affected attributes are:



method create_app(**kwargs)[source]
Create an interactive app to test model predictions.

Demo your machine learning model with a friendly web interface. This app launches directly in the notebook or on an external browser page. The created Interface instance can be accessed through the app attribute.

Parameters**kwargs
Additional keyword arguments for the Interface instance or the Interface.launch method.



method create_dashboard(dataset="test", filename=None, **kwargs)[source]
Create an interactive dashboard to analyze the model.

ATOM uses the explainerdashboard package to provide a quick and easy way to analyze and explain the predictions and workings of the model. The dashboard allows you to investigate SHAP values, permutation importances, interaction effects, partial dependence plots, all kinds of performance plots, and even individual decision trees.

By default, the dashboard renders in a new tab in your default browser, but if preferable, you can render it inside the notebook using the mode="inline" parameter. The created ExplainerDashboard instance can be accessed through the dashboard attribute. This method is not available for multioutput tasks.

Note

Plots displayed by the dashboard are not created by ATOM and can differ from those retrieved through this package.

Parametersdataset: str, default="test"
Data set to get the report from. Choose from: "train", "test", "both" (train and test) or "holdout".

filename: str or None, default=None
Name to save the file with (as .html). None to not save anything.

**kwargs
Additional keyword arguments for the ExplainerDashboard instance.



method cross_validate(**kwargs)[source]
Evaluate the model using cross-validation.

This method cross-validates the whole pipeline on the complete dataset. Use it to assess the robustness of the solution's performance.

Parameters**kwargs
Additional keyword arguments for sklearn's cross_validate function. If the scoring method is not specified, it uses atom's metric.

Returnspd.DataFrame
Overview of the results.



method evaluate(metric=None, dataset="test", threshold=0.5, sample_weight=None)[source]
Get the model's scores for the provided metrics.

Tip

Use the self-get_best_threshold or plot_threshold method to determine a suitable value for the threshold parameter.

Parametersmetric: str, func, scorer, sequence or None, default=None
Metrics to calculate. If None, a selection of the most common metrics per task are used.

dataset: str, default="test"
Data set on which to calculate the metric. Choose from: "train", "test" or "holdout".

threshold: float or sequence, default=0.5
Threshold between 0 and 1 to convert predicted probabilities to class labels. Only used when:

  • The task is binary or multilabel classification.
  • The model has a predict_proba method.
  • The metric evaluates predicted probabilities.

For multilabel classification tasks, it's possible to provide a sequence of thresholds (one per target column, as returned by the get_best_threshold method). If float, the same threshold is applied to all target columns.

sample_weight: sequence or None, default=None
Sample weights corresponding to y in dataset.

Returnspd.Series
Scores of the model.



method export_pipeline(memory=None, verbose=None)[source]
Export the model's pipeline to a sklearn-like object.

The returned pipeline is already fitted on the training set. Note that, if the model used automated feature scaling, the Scaler is added to the pipeline.

Info

The returned pipeline behaves similarly to sklearn's Pipeline, and additionally:

  • Accepts transformers that change the target column.
  • Accepts transformers that drop rows.
  • Accepts transformers that only are fitted on a subset of the provided dataset.
  • Always returns pandas objects.
  • Uses transformers that are only applied on the training set to fit the pipeline, not to make predictions.

Parametersmemory: bool, str, Memory or None, default=None
Used to cache the fitted transformers of the pipeline. - If None or False: No caching is performed. - If True: A default temp directory is used. - If str: Path to the caching directory. - If Memory: Object with the joblib.Memory interface.

verbose: int or None, default=None
Verbosity level of the transformers in the pipeline. If None, it leaves them to their original verbosity. Note that this is not the pipeline's own verbose parameter. To change that, use the set_params method.

ReturnsPipeline
Current branch as a sklearn-like Pipeline object.



method fit(X=None, y=None)[source]
Fit and validate the model.

The estimator is fitted using the best hyperparameters found during hyperparameter tuning. Afterwards, the estimator is evaluated on the test set. Only use this method to re-fit the model after having continued the study.

ParametersX: dataframe or None
Feature set with shape=(n_samples, n_features). If None, self.X_train is used.

y: series or None
Target column corresponding to X. If None, self.y_train is used.



method full_train(include_holdout=False)[source]
Train the estimator on the complete dataset.

In some cases it might be desirable to use all available data to train a final model. Note that doing this means that the estimator can no longer be evaluated on the test set. The newly retrained estimator will replace the estimator attribute. If there is an active mlflow experiment, a new run is started with the name [model_name]_full_train. Since the estimator changed, the model is cleared.

Warning

Although the model is trained on the complete dataset, the pipeline is not. To get a fully trained pipeline, use: pipeline = atom.export_pipeline().fit(atom.X, atom.y).

Parametersinclude_holdout: bool, default=False
Whether to include the holdout set (if available) in the training of the estimator. It's discouraged to use this option since it means the model can no longer be evaluated on any set.



method get_best_threshold(dataset="train")[source]
Get the threshold that maximizes the ROC curve.

Only available for models with a predict_proba method in a binary or multilabel classification task.

Parametersdataset: str, default="train"
Data set on which to calculate the threshold. Choose from: train, test, dataset.

Returnsfloat or list
Best threshold or list of thresholds for multilabel tasks.



method hyperparameter_tuning(n_trials, reset=False)[source]
Run the hyperparameter tuning algorithm.

Search for the best combination of hyperparameters. The function to optimize is evaluated either with a K-fold cross-validation on the training set or using a random train and validation split every trial. Use this method to continue the optimization.

Parametersn_trials: int
Number of trials for the hyperparameter tuning.

reset: bool, default=False
Whether to start a new study or continue the existing one.



method inverse_transform(X=None, y=None, verbose=None)[source]
Inversely transform new data through the pipeline.

Transformers that are only applied on the training set are skipped. The rest should all implement a inverse_transform method. If only X or only y is provided, it ignores transformers that require the other parameter. This can be of use to, for example, inversely transform only the target column. If called from a model that used automated feature scaling, the scaling is inverted as well.

ParametersX: dataframe-like or None, default=None
Transformed feature set with shape=(n_samples, n_features). If None, X is ignored in the transformers.

y: int, str, dict, sequence, dataframe or None, default=None
Target column corresponding to X.

  • If None: y is ignored.
  • If int: Position of the target column in X.
  • If str: Name of the target column in X.
  • If sequence: Target array with shape=(n_samples,) or sequence of column names or positions for multioutput tasks.
  • If dataframe: Target columns for multioutput tasks.

verbose: int or None, default=None
Verbosity level for the transformers. If None, it uses the transformer's own verbosity.

Returnsdataframe
Original feature set. Only returned if provided.

series
Original target column. Only returned if provided.



method save_estimator(filename="auto")[source]
Save the estimator to a pickle file.

Parametersfilename: str, default="auto"
Name of the file. Use "auto" for automatic naming.



method serve(method="predict", host="127.0.0.1", port=8000)[source]
Serve the model as rest API endpoint for inference.

The complete pipeline is served with the model. The inference data must be supplied as json to the HTTP request, e.g. requests.get("http://127.0.0.1:8000/", json=X.to_json()). The deployment is done on a ray cluster. The default host and port parameters deploy to localhost.

Tip

Use import ray; ray.serve.shutdown() to close the endpoint after finishing.

Parametersmethod: str, default="predict"
Estimator's method to do inference on.

host: str, default="127.0.0.1"
Host for HTTP servers to listen on. To expose serve publicly, you probably want to set this to "0.0.0.0".

port: int, default=8000
Port for HTTP server.



method register(name=None, stage="Staging")[source]
Register the model in mlflow's model registry.

This method is only available when model tracking is enabled using one of the following URI schemes: databricks, http, https, postgresql, mysql, sqlite, mssql.

Parametersname: str or None, default=None
Name for the registered model. If None, the model's full name is used.

stage: str, default="Staging"
New desired stage for the model.



method transform(X=None, y=None, verbose=None)[source]
Transform new data through the pipeline.

Transformers that are only applied on the training set are skipped. If only X or only y is provided, it ignores transformers that require the other parameter. This can be of use to, for example, transform only the target column. If called from a model that used automated feature scaling, the data is scaled as well.

ParametersX: dataframe-like or None, default=None
Feature set with shape=(n_samples, n_features). If None, X is ignored. If None, X is ignored in the transformers.

y: int, str, dict, sequence, dataframe or None, default=None
Target column corresponding to X.

  • If None: y is ignored.
  • If int: Position of the target column in X.
  • If str: Name of the target column in X.
  • If sequence: Target array with shape=(n_samples,) or sequence of column names or positions for multioutput tasks.
  • If dataframe: Target columns for multioutput tasks.

verbose: int or None, default=None
Verbosity level for the transformers. If None, it uses the transformer's own verbosity.

Returnsdataframe
Transformed feature set. Only returned if provided.

series
Transformed target column. Only returned if provided.