Schedules

• Cyclic
• Exponential

Module contents

class olympus.optimizers.schedules.LRSchedule(name=None, *, schedule=None, optimizer=None, **kwargs)

Bases: object

Lazy LRSchedule that allows you to first fetch the supported parameters using get_space and then initialize the underlying schedule using init_optimizer

Parameters:

name: str

Name of a registered schedule

schedule: LRSchedule

Custom schedule, mutually exclusive with :param name

Raises:

RegisteredLRSchedulerNotFound

when using a name of an known schedule

MissingArgument:

if name nor schedule were not set

Examples

>>> schedule = LRSchedule('cyclic')
>>> schedule.get_space()
{'base_lr': 'loguniform(1e-5, 1e-2)', 'max_lr': 'loguniform(1e-2, 1)', ... }
>>> schedule.init(optimizer, base_lr=1e-2, ...)

Attributes:

defaults

Return default hyper parameters

lr_scheduler

Methods

epoch(self, epoch[, metrics])	Called after every epoch to update LR
get_current_space(self)	Get currently defined parameter space
get_space(self)	Return the missing hyper parameters required to initialize the LR schedule
init(self[, optimizer, override])	Initialize the LR schedule with the given hyper parameters
step(self, step[, metrics])	Called every step/batch to update LR

get_lr
load_state_dict
state_dict

defaults

Return default hyper parameters

epoch(self, epoch, metrics=None)

Called after every epoch to update LR

get_current_space(self)

Get currently defined parameter space

get_lr(self)

get_space(self)

Return the missing hyper parameters required to initialize the LR schedule

init(self, optimizer=None, override=False, **kwargs)

Initialize the LR schedule with the given hyper parameters

load_state_dict(self, state_dict, strict=True)

lr_scheduler

state_dict(self, destination=None, prefix='', keep_vars=False)

step(self, step, metrics=None)

Called every step/batch to update LR

exception olympus.optimizers.schedules.RegisteredLRSchedulerNotFound

Bases: Exception

exception olympus.optimizers.schedules.UninitializedLRScheduler

Bases: Exception

olympus.optimizers.schedules.known_schedule()

olympus.optimizers.schedules.register_schedule(name, factory, override=False)