Normalizer

class atom.data_cleaning.Normalizer(strategy="yeojohnson", device="cpu", engine="sklearn", verbose=0, logger=None, random_state=None, **kwargs)[source]

Transform the data to follow a Normal/Gaussian distribution.

This transformation is useful for modeling issues related to heteroscedasticity (non-constant variance), or other situations where normality is desired. Missing values are disregarded in fit and maintained in transform. Categorical columns are ignored.

This class can be accessed from atom through the normalize method. Read more in the user guide.

Warning

The quantile strategy performs a non-linear transformation. This may distort linear correlations between variables measured at the same scale but renders variables measured at different scales more directly comparable.

Note

The yeojohnson and boxcox strategies scale the data after transforming. Use the kwargs to change this behaviour.

Parameters

strategy: str, default="yeojohnson"

The transforming strategy. Choose from:

"yeojohnson"
"boxcox" (only works with strictly positive values)
"quantile": Transform features using quantiles information.

device: str, default="cpu"

Device on which to train the estimators. Use any string that follows the SYCL_DEVICE_FILTER filter selector, e.g. device="gpu" to use the GPU. Read more in the user guide.

engine: str, default="sklearn"

Execution engine to use for the estimators. Refer to the user guide for an explanation regarding every choice. Choose from:

"sklearn" (only if device="cpu")
"cuml" (only if device="gpu")

verbose: int, default=0

Verbosity level of the class. Choose from:

0 to not print anything.
1 to print basic information.

logger: str, Logger or None, default=None

If None: Doesn't save a logging file.
If str: Name of the log file. Use "auto" for automatic naming.
Else: Python logging.Logger instance.

random_state: int or None, default=None

Seed used by the quantile strategy. If None, the random number generator is the RandomState used by np.random.

**kwargs

Additional keyword arguments for the strategy estimator.

Attributes

[strategy]: sklearn transformer

Object with which the data is transformed.

feature_names_in_: np.array

Names of features seen during fit.

n_features_in_: int

Number of features seen during fit.

Example

atomstand-alone

>>> from atom import ATOMClassifier
>>> from sklearn.datasets import load_breast_cancer

>>> X, y = load_breast_cancer(return_X_y=True, as_frame=True)

>>> atom = ATOMClassifier(X, y)
>>> print(atom.dataset)

     mean radius  mean texture  ...  worst fractal dimension  target
0          16.78         18.80  ...                  0.07228       0
1          15.34         14.26  ...                  0.09946       0
2          14.22         27.85  ...                  0.07796       1
3          18.31         18.58  ...                  0.06938       0
4          18.49         17.52  ...                  0.09445       0
..           ...           ...  ...                      ...     ...
564        13.44         21.58  ...                  0.07146       0
565        20.47         20.67  ...                  0.06386       0
566        12.98         19.35  ...                  0.09166       1
567        14.61         15.69  ...                  0.05695       1
568        23.27         22.04  ...                  0.09187       0

[569 rows x 31 columns]

>>> atom.plot_distribution(columns=0)

>>> atom.normalize(verbose=2)

Fitting Normalizer...
Normalizing features...

>>> print(atom.dataset)

     mean radius  mean texture  ...  worst fractal dimension  target
0       0.868700      0.010820  ...                -0.684572       0
1       0.513904     -1.257343  ...                 1.019875       0
2       0.200435      1.773390  ...                -0.226619       1
3       1.197448     -0.042755  ...                -0.945047       0
4       1.233326     -0.310726  ...                 0.786014       0
..           ...           ...  ...                      ...     ...
564    -0.041166      0.635293  ...                -0.756291       0
565     1.595052      0.440855  ...                -1.497202       0
566    -0.193933      0.141884  ...                 0.642613       1
567     0.313768     -0.816597  ...                -2.307746       1
568     2.022355      0.730259  ...                 0.653756       0

[569 rows x 31 columns]


>>> atom.plot_distribution(columns=0)

>>> from atom.data_cleaning import Normalizer
>>> from sklearn.datasets import load_breast_cancer

>>> X, y = load_breast_cancer(return_X_y=True, as_frame=True)

     mean radius  mean texture  ...  worst symmetry  worst fractal dimension
0          17.99         10.38  ...          0.4601                  0.11890
1          20.57         17.77  ...          0.2750                  0.08902
2          19.69         21.25  ...          0.3613                  0.08758
3          11.42         20.38  ...          0.6638                  0.17300
4          20.29         14.34  ...          0.2364                  0.07678
..           ...           ...  ...             ...                      ...
564        21.56         22.39  ...          0.2060                  0.07115
565        20.13         28.25  ...          0.2572                  0.06637
566        16.60         28.08  ...          0.2218                  0.07820
567        20.60         29.33  ...          0.4087                  0.12400
568         7.76         24.54  ...          0.2871                  0.07039

[569 rows x 30 columns]

>>> normalizer = Normalizer(verbose=2)
>>> X = normalizer.fit_transform(X)

Fitting Normalizer...
Normalizing features...

>>> print(X)

     mean radius  mean texture  ...  worst symmetry  worst fractal dimension
0       1.134881     -2.678666  ...        2.197206                 1.723624
1       1.619346     -0.264377  ...       -0.121997                 0.537179
2       1.464796      0.547806  ...        1.218181                 0.453955
3      -0.759262      0.357721  ...        3.250202                 2.517606
4       1.571260     -1.233520  ...       -0.943554                -0.279402
..           ...           ...  ...             ...                      ...
564     1.781795      0.785604  ...       -1.721528                -0.751459
565     1.543335      1.845150  ...       -0.480093                -1.210527
566     0.828589      1.817618  ...       -1.301164                -0.170872
567     1.624440      2.016299  ...        1.744693                 1.850944
568    -2.699432      1.203224  ...        0.103122                -0.820663

[569 rows x 30 columns]

Methods

fit	Fit to data.
fit_transform	Fit to data, then transform it.
get_params	Get parameters for this estimator.
inverse_transform	Apply the inverse transformation to the data.
log	Print message and save to log file.
save	Save the instance to a pickle file.
set_params	Set the parameters of this estimator.
transform	Apply the transformations to the data.

method fit(X, y=None)[source]

Fit to data.

Parameters	X: dataframe-like Feature set with shape=(n_samples, n_features). y: int, str, sequence, dataframe-like or None, default=None Does nothing. Implemented for continuity of the API.
Returns	Normalizer Estimator instance.

method fit_transform(X=None, y=None, **fit_params)[source]

Fit to data, then transform it.

Parameters

X: dataframe-like or None, default=None

Feature set with shape=(n_samples, n_features). If None, X is ignored.

y: int, str, sequence, dataframe-like or None, default=None

Target column corresponding to X.

If None: y is ignored.
If int: Position of the target column in X.
If str: Name of the target column in X.
If sequence: Target column with shape=(n_samples,) or sequence of column names or positions for multioutput tasks.
If dataframe-like: Target columns with shape=(n_samples, n_targets) for multioutput tasks.

**fit_params

Additional keyword arguments for the fit method.

Returns

dataframe

Transformed feature set. Only returned if provided.

series

Transformed target column. Only returned if provided.

method get_params(deep=True)[source]

Get parameters for this estimator.

Parameters	deep : bool, default=True If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns	params : dict Parameter names mapped to their values.

method inverse_transform(X, y=None)[source]

Apply the inverse transformation to the data.

Parameters	X: dataframe-like Feature set with shape=(n_samples, n_features). y: int, str, sequence, dataframe-like or None, default=None Does nothing. Implemented for continuity of the API.
Returns	dataframe Original dataframe.

method log(msg, level=0, severity="info")[source]

Print message and save to log file.

Parameters

msg: int, float or str

Message to save to the logger and print to stdout.

level: int, default=0

Minimum verbosity level to print the message.

severity: str, default="info"

Severity level of the message. Choose from: debug, info, warning, error, critical.

method save(filename="auto", save_data=True)[source]

Save the instance to a pickle file.

Parameters

filename: str, default="auto"

Name of the file. Use "auto" for automatic naming.

save_data: bool, default=True

Whether to save the dataset with the instance. This parameter is ignored if the method is not called from atom. If False, add the data to the load method.

method set_params(**params)[source]

Set the parameters of this estimator.

Parameters	**params : dict Estimator parameters.
Returns	self : estimator instance Estimator instance.

method transform(X, y=None)[source]

Apply the transformations to the data.

Parameters	X: dataframe-like Feature set with shape=(n_samples, n_features). y: int, str, sequence, dataframe-like or None, default=None Does nothing. Implemented for continuity of the API.
Returns	dataframe Normalized dataframe.