Vectorizer

---

class atom.nlp.Vectorizer(strategy="BOW", verbose=0, logger=None, *kwargs) [source]

Transform the corpus into meaningful vectors of numbers. The transformation is applied on the column named corpus. If there is no column with that name, an exception is raised. The transformed columns are named after the word they are embedding (if the column is already present in the provided dataset, _[strategy] is added behind the name), and returned in sparse format only if all provided columns (except corpus) are sparse. This class can be accessed from atom through the vectorize method. Read more in the user guide.

Parameters:

strategy: str, optional (default="BOW") Strategy with which to vectorize the text. Choose from: "BOW": Uses a Bag of Words algorithm. "TF-IDF": Uses a TF-IDF algorithm. "Hashing": Uses a hashing algorithm. verbose: int, optional (default=0) Verbosity level of the class. Possible values are: 0 to not print anything. 1 to print basic information. 2 to print detailed information. logger: str, Logger or None, optional (default=None) If None: Doesn't save a logging file. If str: Name of the log file. Use "auto" for automatic naming. Else: Python logging.Logger instance. **kwargs Additional keyword arguments for the strategy estimator.

Warning

The returned dataframe is only sparse when all the columns in the provided dataset (except corpus) are sparse. Calling this transformer with other non-sparse columns forces the output matrix to be converted to its full array. This is not recommended since the process can be very slow and occupy large chunks of memory.

Methods

fit	Fit to data.
fit_transform	Fit to text, then vectorize it.
get_params	Get parameters for this estimator.
log	Write information to the logger and print to stdout.
save	Save the instance to a pickle file.
set_params	Set the parameters of this estimator.
transform	Transform the text.

method fit(X, y=None) [source]

Fit to text.

Parameters:	X: dataframe-like Feature set with shape=(n_samples, n_features). If X is not a pd.DataFrame, it should be composed of a single feature containing the text documents. y: int, str, sequence or None, optional (default=None) Does nothing. Implemented for continuity of the API.
Returns:	Vectorizer Fitted instance of self.

method fit_transform(X, y=None) [source]

Fit to text, then vectorize it.

Parameters:	X: dataframe-like Feature set with shape=(n_samples, n_features). If X is not a pd.DataFrame, it should be composed of a single feature containing the text documents. y: int, str, sequence or None, optional (default=None) Does nothing. Implemented for continuity of the API.
Returns:	X: pd.DataFrame Transformed corpus.

method get_params(deep=True) [source]

Get parameters for this estimator.

Parameters:	deep: bool, optional (default=True) If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:	dict Parameter names mapped to their values.

method log(msg, level=0) [source]

Write a message to the logger and print it to stdout.

Parameters:

msg: str Message to write to the logger and print to stdout. level: int, optional (default=0) Minimum verbosity level to print the message.

method save(filename="auto") [source]

Save the instance to a pickle file.

Parameters:

filename: str, optional (default="auto") Name of the file. Use "auto" for automatic naming.

method set_params(**params) [source]

Set the parameters of this estimator.

Parameters:	**params: dict Estimator parameters.
Returns:	Vectorizer Estimator instance.

method transform(X, y=None) [source]

Normalize the text.

Parameters:	X: dataframe-like Feature set with shape=(n_samples, n_features). If X is not a pd.DataFrame, it should be composed of a single feature containing the text documents. y: int, str, sequence or None, optional (default=None) Does nothing. Implemented for continuity of the API.
Returns:	X: pd.DataFrame Transformed corpus.

Example

from atom import ATOMClassifier

atom = ATOMClassifier(X, y)
atom.vectorize(strategy="TF-IDF")

or

from atom.nlp import Vectorizer

vectorizer = Vectorizer("TF-IDF")
X = vectorizer.transform(X)