mojolearn.ExponentialLR

class mojolearn.ExponentialLR(base_lr, gamma)

Bases: _PowSched

Exponential learning-rate schedule: multiply by gamma every step.

Computes torch’s ExponentialLR closed form, lr = base_lr * gamma ** epoch, as a function of the step index. lr_at(t) takes the one-based optimizer step (lr_at(1) is base_lr). Each value is the exact closed form rounded once to float32 (ties to even, values below the smallest normal flushed to +0.0), so a schedule gives the same bits on every machine.

Pass it as lr_schedule= to RMSprop, Adagrad, Lion, Adamax, NAdam and the recurrent estimators.

Parameters:

• base_lr (float) – Learning rate of the first step. Must be finite.

• gamma (float) – Multiplicative decay factor per step. Must be finite.

Notes

lr_at(t) raises ValueError for t < 1 and OverflowError when the value overflows float32.

Examples

>>> import mojolearn
>>> sched = mojolearn.ExponentialLR(0.01, gamma=0.95)
>>> lrs = sched.lrs(100)

__init__(base_lr, gamma)

Methods

bits_at(t)
lr_at(t)
lr_values(n)	lr_at(1) .
lrs(n)	lr_at(1) .