mojolearn.ExponentialLR#

class mojolearn.ExponentialLR(base_lr, gamma)#

Exponential learning-rate schedule: multiply by gamma every step.

Computes torch’s ExponentialLR closed form, lr = base_lr * gamma ** epoch, as a function of the step index. lr_at(t) takes the one-based optimizer step (lr_at(1) is base_lr). Each value is the exact closed form rounded once to float32 (ties to even, values below the smallest normal flushed to +0.0), so a schedule gives the same bits on every machine.

Pass it as lr_schedule= to RMSprop, Adagrad, Lion, Adamax, NAdam and the recurrent estimators.

Parameters:

base_lr (float) – Learning rate of the first step. Must be finite.
gamma (float) – Multiplicative decay factor per step. Must be finite.

Notes

lr_at(t) raises ValueError for t < 1 and OverflowError when the value overflows float32.

Examples

>>> import mojolearn
>>> sched = mojolearn.ExponentialLR(0.01, gamma=0.95)
>>> lrs = sched.lrs(100)

See also

User guide: Neural networks

Methods

`bits_at`(t)
`lr_at`(t)
`lr_values`(n)	lr_at(1) .
`lrs`(n)	lr_at(1) .

lr_values(n)#: lr_at(1) .. lr_at(n) as an array(‘f’), natively.

lrs(n)#: lr_at(1) .. lr_at(n) as a list, from the schedule’s native blocks (lr_values: every schedule here has one).