mojolearn.LogisticRegression

class mojolearn.LogisticRegression(*, penalty='l2', tol=0.0001, C=1.0, fit_intercept=True, class_weight=None, max_iter=1000, linesearch_max_iter=50, l1_ratio=None, solver='qn')

Bases: NumericModeMixin

Logistic regression classifier, solved by a quasi-Newton method.

Minimizes mean_i logloss_i + penalty(w) / (C * n_samples), which has the same minimizer as scikit-learn’s LogisticRegression(C=C). Two classes use the logistic loss; more than two use the multinomial (softmax) loss. The solver is L-BFGS (history of 5) with a backtracking Armijo line search, or OWL-QN when the penalty has an l1 part; OWL-QN produces coefficients that are exactly zero. The intercept is a solver parameter and is never penalized. This is cuML’s solver='qn'.

A fit stops when max|grad| <= tol * max(loss, tol), when the objective changes by less than tol * 0.01 * max(loss, tol) over 10 iterations, or at max_iter iterations (retcode_ 3). In identical mode the iteration count and the coefficients are the same on NVIDIA and AMD GPUs.

X is read as float32. y is a 1-D array of labels of any sortable type; classes_ is the sorted unique labels.

Parameters:

• penalty ({‘l2’, ‘l1’, ‘elasticnet’, None}, default ‘l2’) – The regularization term. 'l1' and 'elasticnet' use OWL-QN and are available for two classes only; with more classes fit raises NotImplementedError. Any other value raises ValueError.

• tol (float, default 1e-4) – Convergence tolerance (see above).

• C (float, default 1.0) – Inverse regularization strength, > 0; otherwise ValueError.

• fit_intercept (bool, default True) – Whether to fit an (unpenalized) intercept.

• class_weight (None) – Must be None; anything else raises NotImplementedError.

• max_iter (int, default 1000) – Maximum number of L-BFGS iterations. scikit-learn’s default is 100.

• linesearch_max_iter (int, default 50) – Maximum number of line-search steps per iteration.

• l1_ratio (float or None, default None) – The l1 share of the elastic-net penalty, in [0, 1]: the l1 strength is l1_ratio / C and the l2 strength (1 - l1_ratio) / C. Required with penalty='elasticnet' (ValueError if missing or out of range) and ignored (stored as None) for every other penalty.

• solver ({‘qn’}, default ‘qn’) – Only 'qn' is supported; any other value raises ValueError.

• numeric_mode ({‘identical’, ‘fast’} or None, default None) – The numeric mode for this estimator; None uses the process default (see mojolearn.set_numeric_mode). 'fast' is for Apple GPUs.

Variables:

• classes_ (list) – The sorted class labels.

• coef_ (array of shape (1, n_features) or (n_classes, n_features), float32) – Coefficients: one row for two classes, one row per class (in classes_ order) for more.

• intercept_ (array of shape (1,) or (n_classes,), float32) – Intercepts; zeros when fit_intercept is False.

• n_iter_ (array of shape (1,), int64) – The number of solver iterations run.

• objective_ (float) – The final value of the minimized objective.

• retcode_ (int) – The solver’s return code: 0 converged, 1 numeric error, 2 line search failed, 3 maximum iterations reached, 4 invalid arguments.

• n_features_in_ (int) – The number of features seen in fit.

• input_copied_ (bool) – Whether fit had to copy X to float32 C order.

Examples

>>> import numpy as np, mojolearn
>>> rng = np.random.default_rng(0)
>>> X = rng.random((1000, 8), dtype=np.float32)
>>> y = (X[:, 0] + X[:, 1] > 1.0).astype(np.int64)
>>> clf = mojolearn.LogisticRegression(C=10.0).fit(X, y)
>>> proba = clf.predict_proba(X)

__init__(*, penalty='l2', tol=0.0001, C=1.0, fit_intercept=True, class_weight=None, max_iter=1000, linesearch_max_iter=50, l1_ratio=None, solver='qn')

Methods

decision_function(X)	Return the linear scores X @ coef_.T + intercept_.
fit(X, y[, sample_weight])	Fit the classifier.
get_params([deep])
load(path)	Load a model written by save.
numeric_mode_used()	The numeric mode this estimator runs in.
predict(X)	Predict class labels.
predict_log_proba(X)	Return the natural log of predict_proba(X).
predict_proba(X)	Return class probabilities.
save(path)	Save the fitted model to path as an .npz file.
score(X, y)	Return the mean accuracy on the given data.
set_params(**params)	Set constructor parameters.
vendor_used()	The GPU backend this estimator runs on.

fit(X, y, sample_weight=None)

Fit the classifier.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Training data, read as float32.

• y (array-like of shape (n_samples,)) – Class labels, at least two distinct values. A 2-D y, a single class or a length mismatch raises ValueError.

• sample_weight (None) – Must be None; weights raise NotImplementedError.

Returns:

self (LogisticRegression) – The fitted estimator.

decision_function(X)

Return the linear scores X @ coef_.T + intercept_.

Parameters:

X (array-like of shape (n_samples, n_features)) – Samples, read as float32.

Returns:

scores (array of shape (n_samples,) or (n_samples, n_classes), float32) – For two classes, one score per row (positive means classes_[1]); for more, one score per class.

Raises:

ValueError – If the model is not fitted, or X has a different number of features than the training data.

predict(X)

Predict class labels.

For two classes a row is classes_[1] when its decision score is positive and classes_[0] otherwise. For more classes it is the class with the highest score; a tie goes to the class that comes first in classes_.

Parameters:

X (array-like of shape (n_samples, n_features)) – Samples, read as float32.

Returns:

labels (array of shape (n_samples,) or list) – An int64 array for integer labels, a float64 array for float labels, and a Python list for any other label type.

predict_proba(X)

Return class probabilities.

The sigmoid of the decision score for two classes, the softmax of each row’s scores for more, computed in float64.

Parameters:

X (array-like of shape (n_samples, n_features)) – Samples, read as float32.

Returns:

proba (array of shape (n_samples, n_classes), float64) – Column k is the probability of classes_[k].

predict_log_proba(X)

Return the natural log of predict_proba(X).

Parameters:

X (array-like of shape (n_samples, n_features)) – Samples, read as float32.

Returns:

log_proba (array of shape (n_samples, n_classes), float64) – log(p) per entry; -inf where a probability is exactly 0.

score(X, y)

Return the mean accuracy on the given data.

Parameters:

• X (array-like of shape (n_samples, n_features)) – Test samples.

• y (array-like of shape (n_samples,)) – True labels.

Returns:

score (float) – The fraction of rows where predict(X) == y.

save(path)

Save the fitted model to path as an .npz file.

The file stores the fitted coefficients and intercepts, the class labels, the feature count, fit_intercept and the numeric mode. mojolearn.host_model(path) can predict from the file on a machine without a GPU.

Parameters:

path (str or path-like) – Destination file.

Raises:

RuntimeError – If the model is not fitted.

classmethod load(path)

Load a model written by save.

The loaded model predicts with the saved coefficients, classes and numeric mode. It has no n_iter_, objective_ or retcode_.

Parameters:

path (str or path-like) – A file written by LogisticRegression.save.

Returns:

model (LogisticRegression)

Raises:

ValueError – If the file was saved by a different estimator or its fields do not match.