ksuit.optimizer.param_group_modifiers.base.param_group_modifier_base

Classes

ParamGroupModifierBase

Generic implementation to change properties of optimizer parameter groups.

Module Contents

class ksuit.optimizer.param_group_modifiers.base.param_group_modifier_base.ParamGroupModifierBase

Generic implementation to change properties of optimizer parameter groups.

abstractmethod get_properties(model, name, param)

Returns the modified properties for a given model parameter. This method is called with all items of model.named_parameters() to compose the parameter groups for the whole model.

Parameters:

  • model (torch.nn.Module) – Model from which the parameter originates from. Used to extract properties (e.g., number of layers for a layerwise learning rate decay).

  • name (str) – Name of the parameter as stored inside the model.

  • param (torch.Tensor) – The parameter tensor.

Return type:

dict[str, float]

abstractmethod was_applied_successfully()

Checks if the parameter group modifier was applied successfully.

Return type:

bool