Skip to content

Vectorizer

class atom.nlp.Vectorizer(strategy="BOW", verbose=0, logger=None, *kwargs) [source]

Transform the corpus into meaningful vectors of numbers. The transformation is applied on the column named Corpus. If there is no column with that name, an exception is raised. This class can be accessed from atom through the vectorize method. Read more in the user guide.

Parameters: strategy: str, optional (default="BOW")
Strategy with which to vectorize the text. Choose from: verbose: int, optional (default=0)
Verbosity level of the class. Possible values are:
  • 0 to not print anything.
  • 1 to print basic information.
  • 2 to print detailed information.
logger: str, Logger or None, optional (default=None)
  • If None: Doesn't save a logging file.
  • If str: Name of the log file. Use "auto" for automatic naming.
  • Else: Python logging.Logger instance.

**kwargs
Additional keyword arguments passed to the strategy estimator.


Methods

fit Fit to data.
fit_transform Fit to text, then vectorize it.
get_params Get parameters for this estimator.
log Write information to the logger and print to stdout.
save Save the instance to a pickle file.
set_params Set the parameters of this estimator.
transform Transform the text.


method fit(X, y=None) [source]

Fit to text.

Parameters:

X: dict, list, tuple, np.array, sps.matrix or pd.DataFrame
Feature set with shape=(n_samples, n_features). If X is not a pd.DataFrame, it should be composed of a single feature containing the text documents.

y: int, str, sequence or None, optional (default=None)
Does nothing. Implemented for continuity of the API.

Returns: self: Vectorizer
Fitted instance of self.


method fit_transform(X, y=None) [source]

Fit to text, then vectorize it.

Parameters:

X: dict, list, tuple, np.array, sps.matrix or pd.DataFrame
Feature set with shape=(n_samples, n_features). If X is not a pd.DataFrame, it should be composed of a single feature containing the text documents.

y: int, str, sequence or None, optional (default=None)
Does nothing. Implemented for continuity of the API.
Returns:

X: pd.DataFrame
Transformed corpus.


method get_params(deep=True) [source]

Get parameters for this estimator.

Parameters:

deep: bool, optional (default=True)
If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns: params: dict
Dictionary of the parameter names mapped to their values.


method log(msg, level=0) [source]

Write a message to the logger and print it to stdout.

Parameters:

msg: str
Message to write to the logger and print to stdout.

level: int, optional (default=0)
Minimum verbosity level to print the message.


method save(filename="auto") [source]

Save the instance to a pickle file.

Parameters: filename: str, optional (default="auto")
Name of the file. Use "auto" for automatic naming.


method set_params(**params) [source]

Set the parameters of this estimator.

Parameters: **params: dict
Estimator parameters.
Returns: self: Vectorizer
Estimator instance.


method transform(X, y=None) [source]

Normalize the text.

Parameters:

X: dict, list, tuple, np.array, sps.matrix or pd.DataFrame
Feature set with shape=(n_samples, n_features). If X is not a pd.DataFrame, it should be composed of a single feature containing the text documents.

y: int, str, sequence or None, optional (default=None)
Does nothing. Implemented for continuity of the API.
Returns:

X: pd.DataFrame
Transformed corpus.


Example

from atom import ATOMClassifier

atom = ATOMClassifier(X, y)
atom.vectorize()
or 
from atom.nlp import Vectorizer

vectorizer = Vectorizer("tf-idf")
X = vectorizer.transform(X)

Back to top