mojolearn.kernel_methods#

Kernel methods: kernel ridge regression and kernel feature approximations.

KernelRidge, Nystroem and RBFSampler follow scikit-learn’s estimators of the same names: every parameter means what scikit-learn’s parameter of that name means, and an invalid value raises an error that names the parameter. Inputs are converted to float32; sparse input is densified.

KernelRidge and Nystroem support the kernels ‘linear’, ‘poly’ (or ‘polynomial’), ‘rbf’, ‘sigmoid’, ‘laplacian’, ‘cosine’, ‘chi2’ and ‘additive_chi2’. KernelRidge also accepts ‘precomputed’; Nystroem refuses it with ValueError. RBFSampler approximates the RBF kernel only. gamma=None is scikit-learn’s 1 / n_features (1.0 for ‘chi2’).

A fitted model keeps every array it computed, so it can be inspected, saved with save and loaded with load; mojolearn.host_model(path) predicts or transforms from a saved model on a CPU without a GPU.

Classes

`KernelRidge`([alpha, kernel, gamma, degree, ...])	Kernel ridge regression.
`Nystroem`([kernel, gamma, degree, coef0, ...])	Approximate a kernel map with the Nystroem method.
`RBFSampler`([gamma, n_components, random_state])	Approximate the RBF kernel with random Fourier features.

class mojolearn.kernel_methods.KernelRidge(alpha=1.0, kernel='linear', gamma=None, degree=3, coef0=1.0)[source]#

Bases: _KernelMethodBase

Kernel ridge regression.

Follows sklearn.kernel_ridge.KernelRidge (and cuML’s). Fitting forms the kernel matrix K of the training rows, adds alpha to its diagonal, factors it with a Cholesky factorization and solves for the dual coefficients; prediction is K(X, X_fit_) @ dual_coef_. The training matrix is kept on the model because prediction needs it.

If K + alpha * I cannot be factored at the given alpha, fit raises an error rather than falling back to a least-squares solve; increase alpha in that case. Inputs and outputs are float32. Supports the 'identical' (default) and 'fast' numeric modes; 'fast' is available on Apple GPUs only.

Parameters:

alpha (float, default 1.0) – Regularization strength, added to the kernel matrix diagonal. Must be >= 0; zero is accepted, negative or NaN is refused.
kernel (str, default ‘linear’) – One of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘laplacian’, ‘cosine’, ‘chi2’, ‘additive_chi2’ or ‘precomputed’. ‘cosine’, ‘chi2’ and ‘additive_chi2’ are scikit-learn’s pairwise kernels; both chi2 kernels refuse negative values in X with ValueError, as scikit-learn’s do. With ‘precomputed’, X is the n x n kernel matrix at fit and the q x n cross-kernel against the training rows at predict, used as given (symmetry is not checked).
gamma (float or None, default None) – Kernel coefficient for ‘poly’, ‘rbf’, ‘sigmoid’, ‘laplacian’ and the chi2 kernels. None means 1 / n_features (1.0 for ‘chi2’, its scikit-learn default). Must be positive where the kernel reads it.
degree (int, default 3) – Degree of the ‘poly’ kernel.
coef0 (float, default 1.0) – Independent term of the ‘poly’ and ‘sigmoid’ kernels.
numeric_mode ({‘identical’, ‘fast’} or None, default None) – Numeric mode for this estimator; None uses the process default.

Variables:

dual_coef_ (Array of shape (n_samples,) or (n_samples, n_targets), float32) – Dual coefficients, 1-D when y was 1-D.
X_fit_ (Array of shape (n_samples, n_features), float32) – The training data, used by predict.
info_ (int) – Status of the factorization; always 0 on a fitted model.
n_features_in_ (int) – Number of features seen during fit.
n_targets_ (int) – Number of regression targets.

Examples

>>> import numpy as np, mojolearn
>>> rng = np.random.default_rng(0)
>>> X = rng.random((1000, 8), dtype=np.float32)
>>> y = X.sum(axis=1)
>>> model = mojolearn.KernelRidge(alpha=1.0, kernel="rbf").fit(X, y)
>>> pred = model.predict(X[:5])

fit(X, y, sample_weight=None)[source]#

Fit the kernel ridge model.

Parameters:

X (array-like of shape (n_samples, n_features)) – Training data, converted to float32. With kernel='precomputed', the square n x n kernel matrix.
y (array-like of shape (n_samples,) or (n_samples, n_targets)) – Target values.
sample_weight (float or array-like of shape (n_samples,), default None) – Per-sample weights, finite and >= 0, applied as scikit-learn does. None weights every sample equally.

Returns:

self (KernelRidge) – The fitted estimator.

predict(X)[source]#

Predict with the fitted model: K(X, X_fit_) @ dual_coef_.

Parameters:: X (array-like of shape (n_queries, n_features)) – Query data. With kernel='precomputed', the kernel between the queries and the training rows, shape (n_queries, n_samples).
Returns:: Array of shape (n_queries,) or (n_queries, n_targets), float32 – Predictions, 1-D when the fitted y was 1-D.

save(path)[source]#

Save the fitted model to path as an .npz file.

The file holds the training data, the dual coefficients, the kernel parameters as resolved at fit and the numeric mode. Load it with KernelRidge.load, or predict from it on a CPU without a GPU with mojolearn.host_model(path).

Parameters:: path (str or path-like) – Destination file.
Raises:: RuntimeError – If the model has not been fitted.

classmethod load(path)[source]#

Load a model written by save.

Parameters:: path (str or path-like) – File written by KernelRidge.save.
Returns:: KernelRidge – A fitted model, ready to predict, with the saved numeric mode.

class mojolearn.kernel_methods.Nystroem(kernel='rbf', gamma=None, degree=3, coef0=1.0, n_components=100, random_state=0)[source]#

Bases: _KernelMethodBase

Approximate a kernel map with the Nystroem method.

Follows sklearn.kernel_approximation.Nystroem. fit picks n_components training rows at random as the basis, forms their kernel matrix and computes its inverse square root with a Jacobi eigensolver on the device; transform maps rows to K(X, components_) @ normalization_.T, a feature space whose inner products approximate the kernel. Inputs and outputs are float32. Supports the 'identical' (default) and 'fast' numeric modes; 'fast' is available on Apple GPUs only.

Parameters:

kernel (str, default ‘rbf’) – One of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘laplacian’, ‘cosine’, ‘chi2’ or ‘additive_chi2’. ‘precomputed’ is refused with ValueError. Both chi2 kernels refuse negative values in X.
gamma (float or None, default None) – Kernel coefficient. None means 1 / n_features (1.0 for ‘chi2’).
degree (int, default 3) – Degree of the ‘poly’ kernel.
coef0 (float, default 1.0) – Independent term of the ‘poly’ and ‘sigmoid’ kernels.
n_components (int, default 100) – Number of basis rows. Must be positive and at most n_samples.
random_state (int, default 0) – Seed of the basis selection.
numeric_mode ({‘identical’, ‘fast’} or None, default None) – Numeric mode for this estimator; None uses the process default.

Variables:

components_ (Array of shape (n_components, n_features), float32) – The basis rows.
component_indices_ (Array of shape (n_components,), int32) – Indices of the basis rows in the training data.
normalization_ (Array of shape (n_components, n_components), float32) – Inverse square root of the basis kernel matrix.
eigenvalues_ (Array of shape (n_components,), float32) – Absolute eigenvalues of the basis kernel matrix (its singular values), descending. A negative eigenvalue keeps its sign in the right factor of normalization_, as scikit-learn’s SVD does.
eigenvectors_ (Array of shape (n_components, n_components), float32) – Eigenvectors of the basis kernel matrix.
sweeps_ (int) – Number of Jacobi sweeps the eigensolver ran.
n_features_in_ (int) – Number of features seen during fit.

Examples

>>> import numpy as np, mojolearn
>>> X = np.random.default_rng(0).random((1000, 8), dtype=np.float32)
>>> features = mojolearn.Nystroem(n_components=64).fit_transform(X)

fit(X, y=None)[source]#

Choose the basis rows and compute the normalization.

Parameters:

X (array-like of shape (n_samples, n_features)) – Training data, converted to float32.
y (ignored) – Present for scikit-learn API compatibility.

Returns:

self (Nystroem) – The fitted transformer.

transform(X)[source]#

Map X to the approximate kernel feature space.

Parameters:: X (array-like of shape (n_rows, n_features)) – Data to transform.
Returns:: Array of shape (n_rows, n_components), float32 – K(X, components_) @ normalization_.T.

fit_transform(X, y=None)[source]#

Fit to X, then transform X.

Parameters:

X (array-like of shape (n_samples, n_features)) – Training data.
y (ignored) – Present for scikit-learn API compatibility.

Returns:

Array of shape (n_samples, n_components), float32 – The transformed training data.

save(path)[source]#

Save the fitted transformer to path as an .npz file.

Load it with Nystroem.load, or transform from it on a CPU without a GPU with mojolearn.host_model(path).

Parameters:: path (str or path-like) – Destination file.
Raises:: RuntimeError – If the transformer has not been fitted.

classmethod load(path)[source]#

Load a transformer written by save.

Parameters:: path (str or path-like) – File written by Nystroem.save.
Returns:: Nystroem – A fitted transformer, ready to transform, with the saved numeric mode.

class mojolearn.kernel_methods.RBFSampler(gamma=1.0, n_components=100, random_state=0)[source]#

Bases: _KernelMethodBase

Approximate the RBF kernel with random Fourier features.

Follows sklearn.kernel_approximation.RBFSampler. fit draws random weights and offsets from random_state; transform maps rows to scale_ * cos(X @ random_weights_ + random_offset_), a feature space whose inner products approximate exp(-gamma * ||x - y||^2). fit reads only the number of features of X (and its variance when gamma='scale'). Inputs and outputs are float32. Supports the 'identical' (default) and 'fast' numeric modes; 'fast' is available on Apple GPUs only.

Parameters:

gamma (float or ‘scale’, default 1.0) – RBF kernel coefficient; must be positive. ‘scale’ is scikit-learn’s 1 / (n_features * X.var()) (1.0 when the variance is zero), computed from the exact variance of the float32 data.
n_components (int, default 100) – Number of random features. Must be positive.
random_state (int, default 0) – Seed of the random draws.
numeric_mode ({‘identical’, ‘fast’} or None, default None) – Numeric mode for this estimator; None uses the process default.

Variables:

random_weights_ (Array of shape (n_features, n_components), float32) – The random projection weights.
random_offset_ (Array of shape (n_components,), float32) – The random phase offsets.
sigma_ (float) – sqrt(2 * gamma), the standard deviation of the weights.
scale_ (float) – sqrt(2 / n_components), the output scale.
n_features_in_ (int) – Number of features seen during fit.

Examples

>>> import numpy as np, mojolearn
>>> X = np.random.default_rng(0).random((1000, 8), dtype=np.float32)
>>> features = mojolearn.RBFSampler(gamma=0.5, n_components=256).fit_transform(X)

fit(X, y=None)[source]#

Draw the random weights and offsets.

Reads only X.shape[1] (and, for gamma='scale', the variance of X), as scikit-learn does.

Parameters:

X (array-like of shape (n_samples, n_features)) – Training data, converted to float32.
y (ignored) – Present for scikit-learn API compatibility.

Returns:

self (RBFSampler) – The fitted transformer.

transform(X)[source]#

Map X to the random Fourier feature space.

Parameters:: X (array-like of shape (n_rows, n_features)) – Data to transform.
Returns:: Array of shape (n_rows, n_components), float32 – scale_ * cos(X @ random_weights_ + random_offset_).

fit_transform(X, y=None)[source]#

Fit to X, then transform X.

Parameters:

X (array-like of shape (n_samples, n_features)) – Training data.
y (ignored) – Present for scikit-learn API compatibility.

Returns:

Array of shape (n_samples, n_components), float32 – The transformed training data.

save(path)[source]#

Save the fitted sampler to path as an .npz file.

Load it with RBFSampler.load, or transform from it on a CPU without a GPU with mojolearn.host_model(path).

Parameters:: path (str or path-like) – Destination file.
Raises:: RuntimeError – If the sampler has not been fitted.

classmethod load(path)[source]#

Load a sampler written by save.

Parameters:: path (str or path-like) – File written by RBFSampler.save.
Returns:: RBFSampler – A fitted sampler with the saved weights (nothing is drawn again), ready to transform, with the saved numeric mode.