Base

class olympus.optimizers.base.OptimizerAdapter(factory, *args, **kwargs)

Bases: olympus.optimizers.base.OptimizerInterface

Wraps an existing Pytorch Optimizer into an Olympus optimizer

Attributes:

param_groups

state

Methods

add_param_group(self, param_group)	Add a param group to the Optimizer s param_groups.
backward(self, loss)	This method comes from FP16 Optimizer, for consistency we add it everywhere
defaults()	Specifies the hyper parameters defaults
get_space()	Specifies the hyper parameters that are supported by this optimizer
load_state_dict(self, state_dict[, strict])	Loads the optimizer state.
state_dict(self[, destination, prefix, …])	Returns the state of the optimizer as a dict.
step(self[, closure])	Performs a single optimization step (parameter update).
zero_grad(self)	Clears the gradients of all optimized torch.Tensor s.

add_param_group(self, param_group)

Add a param group to the Optimizer s param_groups.

This can be useful when fine tuning a pre-trained network as frozen layers can be made trainable and added to the Optimizer as training progresses.

Arguments:

param_group (dict): Specifies what Tensors should be optimized along with group specific optimization options.

backward(self, loss)

This method comes from FP16 Optimizer, for consistency we add it everywhere

static defaults()

Specifies the hyper parameters defaults

static get_space()

Specifies the hyper parameters that are supported by this optimizer

load_state_dict(self, state_dict, strict=True)

Loads the optimizer state.

Arguments:

state_dict (dict): optimizer state. Should be an object returned

from a call to state_dict().

param_groups

state

state_dict(self, destination=None, prefix='', keep_vars=False)

Returns the state of the optimizer as a dict.

It contains two entries:

• state - a dict holding current optimization state. Its content

  differs between optimizer classes.

• param_groups - a dict containing all parameter groups

step(self, closure=None)

Performs a single optimization step (parameter update).

Arguments:

closure (callable): A closure that reevaluates the model and

returns the loss. Optional for most optimizers.

zero_grad(self)

Clears the gradients of all optimized torch.Tensor s.

class olympus.optimizers.base.OptimizerInterface(params)

Bases: torch.optim.optimizer.Optimizer

Base Olympus Optimizer

Methods

add_param_group(self, param_group)	Add a param group to the Optimizer s param_groups.
backward(self, loss)	This method comes from FP16 Optimizer, for consistency we add it everywhere
defaults()	Specifies the hyper parameters defaults
get_space()	Specifies the hyper parameters that are supported by this optimizer
load_state_dict(self, state_dict[, strict])	Loads the optimizer state.
state_dict(self[, destination, prefix, …])	Returns the state of the optimizer as a dict.
step(self[, closure])	Performs a single optimization step (parameter update).
zero_grad(self)	Clears the gradients of all optimized torch.Tensor s.

add_param_group(self, param_group)

Add a param group to the Optimizer s param_groups.

This can be useful when fine tuning a pre-trained network as frozen layers can be made trainable and added to the Optimizer as training progresses.

Arguments:

param_group (dict): Specifies what Tensors should be optimized along with group specific optimization options.

backward(self, loss)

This method comes from FP16 Optimizer, for consistency we add it everywhere

static defaults()

Specifies the hyper parameters defaults

static get_space()

Specifies the hyper parameters that are supported by this optimizer

load_state_dict(self, state_dict, strict=True)

Loads the optimizer state.

Arguments:

state_dict (dict): optimizer state. Should be an object returned

from a call to state_dict().

state_dict(self, destination=None, prefix='', keep_vars=False)

Returns the state of the optimizer as a dict.

It contains two entries:

• state - a dict holding current optimization state. Its content

  differs between optimizer classes.

• param_groups - a dict containing all parameter groups

step(self, closure=None)

Performs a single optimization step (parameter update).

Arguments:

closure (callable): A closure that reevaluates the model and

returns the loss. Optional for most optimizers.

zero_grad(self)

Clears the gradients of all optimized torch.Tensor s.