plot_calibration

method plot_calibration(models=None, n_bins=10, title=None, figsize=(10, 10), filename=None, display=True) [source]

Plot the calibration curve for a binary classifier. Well calibrated classifiers are probabilistic classifiers for which the output of the predict_proba method can be directly interpreted as a confidence level. For instance a well calibrated (binary) classifier should classify the samples such that among the samples to which it gave a predict_proba value close to 0.8, approx. 80% actually belong to the positive class. Read more in sklearn's documentation.

This figure shows two plots: the calibration curve, where the x-axis represents the average predicted probability in each bin, and the y-axis is the fraction of positives, i.e. the proportion of samples whose class is the positive class (in each bin); and a distribution of all predicted probabilities of the classifier.

Parameters:

models: str, sequence or None, optional (default=None)
Name of the models to plot. If None, all models in the pipeline are selected.

n_bins: int, optional (default=10)
Number of bins used for calibration. Minimum of 5 required.

title: str or None, optional (default=None)
Plot's title. If None, the title is left empty.

figsize: tuple, optional (default=(10, 10))
Figure's size, format as (x, y).

filename: str or None, optional (default=None)
Name of the file. Use "auto" for automatic naming. If None, the figure is not saved.

display: bool or None, optional (default=True)
Whether to render the plot. If None, it returns the matplotlib figure.

Returns: fig: matplotlib.figure.Figure
Plot object. Only returned if display=None.

Tip

Use the calibrate method to calibrate the winning model.

Example

from atom import ATOMClassifier

atom = ATOMClassifier(X, y)
atom.run(["GNB", "LR", "LGB"], metric="average_precision")
atom.plot_calibration()