mojolearn.StepLR#

class mojolearn.StepLR(base_lr, step_size, gamma=0.1)[source]#

Bases: _PowSched

Step-decay learning-rate schedule: multiply by gamma every step_size steps.

Computes torch’s StepLR closed form, lr = base_lr * gamma ** (epoch // step_size), as a function of the step index. lr_at(t) takes the one-based optimizer step: lr_at(1) is the rate before the first scheduler.step() in torch, and lr_at(t) the rate after t - 1 steps. Each value is the exact closed form rounded once to float32 (ties to even, values below the smallest normal flushed to +0.0), so a schedule gives the same bits on every machine.

Pass a schedule as lr_schedule= to the optimizers RMSprop, Adagrad, Lion, Adamax and NAdam, and to the recurrent estimators (LSTMClassifier, GRURegressor, …); they set lr = schedule.lr_at(t) before step t.

Parameters:

  • base_lr (float) – Learning rate of the first step. Must be finite.

  • step_size (int) – Number of steps between decays. Must be >= 1, otherwise ValueError.

  • gamma (float, default 0.1) – Multiplicative decay factor. Must be finite.

Notes

lr_at(t) raises ValueError for t < 1 and OverflowError when the value overflows float32. Per-group learning rates and last_epoch resumption are not supported: to resume, evaluate lr_at at the step you resume from.

Examples

>>> import mojolearn
>>> sched = mojolearn.StepLR(0.1, step_size=30, gamma=0.1)
>>> first, after_30 = sched.lr_at(1), sched.lr_at(31)
__init__(base_lr, step_size, gamma=0.1)[source]#

Methods

bits_at(t)

lr_at(t)

lr_values(n)

lr_at(1) .

lrs(n)

lr_at(1) .
On this page