mojolearn.GaussianProcessRegressor#

class mojolearn.GaussianProcessRegressor(kernel=None, *, alpha=9.5367431640625e-07, optimizer=None, n_restarts_optimizer=0, normalize_y=False, copy_X_train=True, n_targets=None, random_state=None)[source]#

Bases: NumericModeMixin

Exact Gaussian process regression on the GPU.

Follows scikit-learn’s GaussianProcessRegressor: fit builds the kernel matrix K of the training points, adds alpha to its diagonal, factors it by Cholesky and solves for the dual coefficients; predict returns the posterior mean and, on request, the standard deviation or the full covariance. X is dense float32 in C order (other inputs are converted; sparse input is not accepted) and y is 1-D: one target per model.

Differences from scikit-learn that change results:

  • The default alpha is 2**-20, not 1e-10. Kernel matrices have a unit diagonal, and in float32 1.0 + 1e-10 is exactly 1.0, so scikit-learn’s default would add nothing.

  • The default optimizer is None: the kernel you pass is used as is. Pass optimizer="fmin_l_bfgs_b" (scikit-learn’s default) to fit the hyperparameters by maximizing the log marginal likelihood.

  • A failed Cholesky factorization does not raise. fit returns normally and records LAPACK’s code in info_; check that it is 0. A nonzero info_ means this kernel and ridge do not describe the data (common causes: duplicate training rows, a length scale far larger than the data’s spread, alpha=0). predict and log_marginal_likelihood then raise an error.

Under numeric_mode="identical" (the default), fit, predict and sample_y give the same bits on NVIDIA and AMD GPUs.

Parameters:

  • kernel (Kernel or None, default None) – A combination of RBF, Matern, ConstantKernel and WhiteKernel with + and *. None means ConstantKernel(1.0, "fixed") * RBF(1.0, "fixed"), scikit-learn’s default. Other kernel types, including scikit-learn kernel objects, raise TypeError.

  • alpha (float, default 2*-20*) – Value added to the diagonal of the training kernel matrix (a ridge). NaN, negative and infinite values are refused at fit time. Under numeric_mode="identical" only 0.0 and 2**-20 are accepted. Not to be confused with the fitted alpha_.

  • optimizer (None or “fmin_l_bfgs_b”, default None) – None fits the kernel as passed. "fmin_l_bfgs_b" maximizes the log marginal likelihood over the kernel’s free hyperparameters (those whose bounds are not "fixed") with a bounded L-BFGS that runs on the device. A callable raises NotImplementedError; any other value raises ValueError.

  • n_restarts_optimizer (int, default 0) – Number of extra optimizer runs from log-uniform random starting points within the bounds. The run with the best likelihood wins; a tie goes to the earlier run.

  • normalize_y (bool, default False) – Center and scale y by its mean and standard deviation before the fit, and scale the predictions back, as in scikit-learn. A zero standard deviation scales by one.

  • copy_X_train (bool, default True) – Only True is accepted: X is always copied to float32 memory. False raises NotImplementedError.

  • n_targets (None, default None) – Only None is accepted; multi-target models are not supported.

  • random_state (int or None, default None) – Seed for the optimizer’s restart points, an int in [0, 2**64). Required when optimizer is set and n_restarts_optimizer > 0. sample_y takes its own random_state.

  • numeric_mode (str or None, default None) – The numeric mode for this estimator; None uses the process default. See the numeric modes guide.

Variables:

  • X_train_ (Array of shape (n_train, n_features), float32) – The training inputs.

  • y_train_ (Array of shape (n_train,), float32) – The training targets, normalized when normalize_y=True (as in scikit-learn).

  • kernel_ (Kernel) – The fitted kernel: the kernel you passed when optimizer=None, otherwise a copy with the optimized hyperparameters.

  • L_ (Array of shape (n_train, n_train), float32) – Lower Cholesky factor of K + alpha * I.

  • alpha_ (Array of shape (n_train,), float32) – Dual coefficients (K + alpha * I)^-1 y (scikit-learn’s alpha_; not the alpha ridge parameter).

  • log_marginal_likelihood_value_ (float) – Log marginal likelihood of the fitted kernel. Meaningful only when info_ == 0.

  • info_ (int) – LAPACK’s Cholesky code. 0 means L_ is a valid factor; k > 0 means the leading minor of order k was not positive definite.

  • nb_ (int) – The Cholesky panel width used.

  • n_features_in_ (int) – Number of features seen during fit.

  • input_copied_ (bool) – Whether fit had to copy X to reach float32 C order.

  • clamped_ (Array of shape (n_query,), int32) – Set by predict(X, return_std=True): a per-point flag marking variances that were clamped to zero.

  • n_clamped_ (int) – Set by predict(X, return_std=True): how many variances were clamped to zero.

Examples

>>> import numpy as np, mojolearn
>>> rng = np.random.default_rng(0)
>>> X = rng.random((200, 3), dtype=np.float32)
>>> y = np.sin(X.sum(axis=1)).astype(np.float32)
>>> kernel = (mojolearn.ConstantKernel(1.0) * mojolearn.RBF([1.0, 1.0, 1.0])
...           + mojolearn.WhiteKernel(0.01))
>>> model = mojolearn.GaussianProcessRegressor(kernel, optimizer="fmin_l_bfgs_b").fit(X, y)
>>> ok = model.info_ == 0
>>> mean, std = model.predict(X[:5], return_std=True)
__init__(kernel=None, *, alpha=9.5367431640625e-07, optimizer=None, n_restarts_optimizer=0, normalize_y=False, copy_X_train=True, n_targets=None, random_state=None)[source]#

Methods

fit(X, y)

Fit the Gaussian process to training data.

get_params([deep])

load(path)

Load a model written by save.

log_marginal_likelihood([theta, ...])

Log marginal likelihood of the training data.

numeric_mode_used()

The numeric mode this estimator runs in.

predict(X[, return_std, return_cov])

Predict with the posterior of the fitted Gaussian process.

sample_y(X[, n_samples, random_state])

Draw samples from the posterior at X.

save(path)

Save the fitted model to path as an .npz file.

score(X, y)

Coefficient of determination R^2 of the predictions.

set_params(**params)

Set constructor parameters.

vendor_used()

The GPU backend this estimator runs on.
fit(X, y)[source]#

Fit the Gaussian process to training data.

With an optimizer, the kernel hyperparameters are fitted first. Then K + alpha * I is factored and the dual coefficients and log marginal likelihood are computed on the device.

Parameters:

  • X (array-like of shape (n_samples, n_features)) – Training inputs, converted to float32.

  • y (array-like of shape (n_samples,)) – Training targets. A 2-D y raises ValueError.

Returns:

self (GaussianProcessRegressor) – The fitted model. Check info_ == 0: a failed Cholesky factorization is reported there rather than raised.

predict(X, return_std=False, return_cov=False)[source]#

Predict with the posterior of the fitted Gaussian process.

Parameters:

  • X (array-like of shape (n_query, n_features)) – Query points, converted to float32.

  • return_std (bool, default False) – Also return the predictive standard deviation. Negative variances from rounding are clamped to zero; clamped_ and n_clamped_ record which points were clamped.

  • return_cov (bool, default False) – Also return the full predictive covariance k(X, X) - V^T V, exactly symmetric and not clamped. At most one of return_std and return_cov may be True (RuntimeError otherwise).

Returns:

  • mean (Array of shape (n_query,), float32)

  • std (Array of shape (n_query,), float32) – Only with return_std=True.

  • cov (Array of shape (n_query, n_query), float32) – Only with return_cov=True.

Raises:

ValueError – If the model is not fitted (predicting from the unfitted prior is not supported), or X has a different number of features than the training data.

Notes

A model whose fit failed (info_ != 0) refuses to predict. With normalize_y=True the mean, standard deviation and covariance are scaled back to the units of y.

save(path)[source]#

Save the fitted model to path as an .npz file.

The file holds everything predict reads: the training inputs, the Cholesky factor, the dual coefficients, the fitted kernel and the numeric mode. Raises RuntimeError if the model is not fitted.

Parameters:

path (str or path-like) – Destination file.

classmethod load(path)[source]#

Load a model written by save.

The loaded model predicts the mean and standard deviation. Every array is read at its saved dtype and never cast; a file saved by a different estimator or with inconsistent contents raises ValueError.

Parameters:

path (str or path-like) – File written by save.

Returns:

GaussianProcessRegressor

log_marginal_likelihood(theta=None, eval_gradient=False, clone_kernel=True)[source]#

Log marginal likelihood of the training data.

Parameters:

  • theta (sequence of float or None, default None) – None returns the value computed during fit, and raises RuntimeError if that fit’s factorization failed. Otherwise, the log of the free hyperparameters of kernel_ (one entry per kernel_.n_dims; ValueError on a length mismatch): the likelihood is evaluated at exp(theta) in float32. A factorization that fails at that theta gives -inf and a zero gradient, as in scikit-learn.

  • eval_gradient (bool, default False) – Also return the gradient with respect to theta. Requires a theta (ValueError otherwise).

  • clone_kernel (bool, default True) – Accepted for scikit-learn compatibility; kernel_ is never modified.

Returns:

  • log_likelihood (float)

  • gradient (list of float) – Only with eval_gradient=True.

sample_y(X, n_samples=1, random_state=0)[source]#

Draw samples from the posterior at X.

Each sample is the predictive mean plus a factor of the predictive covariance times standard normals, scaled back to the units of y under normalize_y, as in scikit-learn. The covariance is factored with a 2**-20 jitter on its diagonal. The same model, X, n_samples and random_state give the same values on every call; they are not scikit-learn’s values. A covariance that cannot be factored raises an error.

Parameters:

  • X (array-like of shape (n_query, n_features)) – Query points, converted to float32.

  • n_samples (int, default 1) – Number of joint draws.

  • random_state (int, default 0) – Seed, an int in [0, 2**64). None and a NumPy RandomState raise ValueError.

Returns:

Array of shape (n_query, n_samples), float32 – Column s is one joint draw over the query points.

Raises:

ValueError – If the model is not fitted (sampling the unfitted prior is not supported).

score(X, y)[source]#

Coefficient of determination R^2 of the predictions.

Uses scikit-learn’s definition, accumulated in float64 from the float32 predictions.

Parameters:

  • X (array-like of shape (n_samples, n_features)) – Test inputs.

  • y (array-like of shape (n_samples,)) – True targets.

Returns:

float