mojolearn.DARTClassifier#

class mojolearn.DARTClassifier(*, n_estimators=100, learning_rate=0.1, num_leaves=31, max_depth=-1, min_child_samples=20, reg_lambda=0.0, max_bin=255, drop_rate=0.1, max_drop=50, skip_drop=0.5, xgboost_dart_mode=False, uniform_drop=False, drop_seed=4, random_state=None, reg_alpha=0.0, max_delta_step=0.0, colsample_bytree=1.0, subsample=1.0, subsample_freq=0, feature_fraction_seed=2, bagging_seed=3)[source]#

Bases: _DARTBase

Gradient boosting with DART dropout, classification.

Follows LightGBM’s boosting="dart": binary log loss for two classes, multiclass softmax (one tree per class per iteration) for more. In each iteration a random subset of the earlier iterations is dropped while the new trees are fitted, and the trees are then renormalized. X is read as float32.

Each tree is grown by the forest builder (level-order, quantile bins) on the negative gradient; leaf values follow LightGBM (the L1/L2-regularized Newton step, clipped by max_delta_step). The draws (dropout, bagging, feature sampling) come from mojolearn’s own random generator, not LightGBM’s, and the starting score is a constant kept outside the trees. A fit with n_estimators * n_classes * (2 * num_leaves - 1) above 2**26, or num_leaves above 32768, raises NotImplementedError.

Parameters:

  • n_estimators (int, default=100) – Number of boosting iterations; must be at least 1.

  • learning_rate (float, default=0.1) – Shrinkage; must be > 0.

  • num_leaves (int, default=31) – Maximum leaves per tree, applied as a cap on level-order growth (LightGBM grows leaf-wise), so trees can differ from LightGBM’s.

  • max_depth (int, default=-1) – Maximum tree depth; -1 or any value <= 0 means no limit.

  • min_child_samples (int, default=20) – Minimum number of rows in each leaf.

  • reg_lambda (float, default=0.0) – L2 regularization on leaf values; must be >= 0.

  • max_bin (int, default=255) – Number of quantile bins per feature used for the split search.

  • drop_rate (float, default=0.1) – Fraction of earlier trees dropped each iteration, in [0, 1].

  • max_drop (int, default=50) – Maximum number of trees dropped per iteration.

  • skip_drop (float, default=0.5) – Probability of skipping the dropout in an iteration, in [0, 1].

  • xgboost_dart_mode (bool, default=False) – Use XGBoost’s DART normalization.

  • uniform_drop (bool, default=False) – Drop trees uniformly instead of in proportion to their weights.

  • drop_seed (int, default=4) – Seed for the dropout draws.

  • random_state (int or None, default=None) – Seed for the trees. None means seed 0.

  • reg_alpha (float, default=0.0) – L1 regularization on leaf values; must be >= 0.

  • max_delta_step (float, default=0.0) – Clip on each leaf value; 0 means no clip.

  • colsample_bytree (float, default=1.0) – Fraction of features sampled for each tree, in (0, 1].

  • subsample (float, default=1.0) – Fraction of rows bagged, in (0, 1]; used when subsample_freq > 0.

  • subsample_freq (int, default=0) – Re-draw the row bag every this many iterations; 0 disables bagging.

  • feature_fraction_seed (int, default=2) – Seed for the per-tree feature sampling.

  • bagging_seed (int, default=3) – Seed for the row bagging.

Variables:

  • classes_ (array of shape (n_classes,)) – Class labels seen in fit.

  • init_score_ (float or list of float) – Starting score (one per class for more than two classes).

  • tree_weights_ (list of float) – Final weight of each tree after DART normalization.

  • n_features_in_ (int) – Number of features seen in fit.

Examples

>>> import numpy as np, mojolearn
>>> rng = np.random.default_rng(0)
>>> X = rng.random((1000, 8), dtype=np.float32)
>>> y = (X[:, 0] + X[:, 1] > 1.0).astype(np.int32)
>>> model = mojolearn.DARTClassifier(n_estimators=50).fit(X, y)
>>> proba = model.predict_proba(X)
__init__(*, n_estimators=100, learning_rate=0.1, num_leaves=31, max_depth=-1, min_child_samples=20, reg_lambda=0.0, max_bin=255, drop_rate=0.1, max_drop=50, skip_drop=0.5, xgboost_dart_mode=False, uniform_drop=False, drop_seed=4, random_state=None, reg_alpha=0.0, max_delta_step=0.0, colsample_bytree=1.0, subsample=1.0, subsample_freq=0, feature_fraction_seed=2, bagging_seed=3)[source]#

Methods

decision_function(X)

Raw boosted scores, float64.

fit(X, y)

Fit the boosted trees on X (n_samples, n_features) and labels y.

get_params([deep])

numeric_mode_used()

The numeric mode this estimator runs in.

predict(X)

Predict the class with the highest score.

predict_proba(X)

Class probabilities, float64, shape (n_samples, n_classes).

set_params(**params)

Set constructor parameters.

vendor_used()

The GPU backend this estimator runs on.
fit(X, y)[source]#

Fit the boosted trees on X (n_samples, n_features) and labels y.

At least two classes are required.

Returns:

self

decision_function(X)[source]#

Raw boosted scores, float64.

Shape (n_samples,) for two classes (the log-odds of classes_[1]), (n_samples, n_classes) otherwise.

predict_proba(X)[source]#

Class probabilities, float64, shape (n_samples, n_classes).

For two classes the columns are [1 - p, p] with p the sigmoid of the raw score; for more, the softmax of the per-class raw scores.

predict(X)[source]#

Predict the class with the highest score.