ksuit.optimizer.param_group_modifiers.lr_scale_by_name_modifier¶

Classes¶

LrScaleByNameModifier

Scales the learning rate of a certain parameter.

Module Contents¶

class ksuit.optimizer.param_group_modifiers.lr_scale_by_name_modifier.LrScaleByNameModifier(param_group_modifier_config)¶

Bases: ksuit.optimizer.param_group_modifiers.base.ParamGroupModifierBase

Scales the learning rate of a certain parameter.

Parameters:: param_group_modifier_config (ksuit.schemas.optim.ParamGroupModifierConfig)

scale¶

name¶

param_was_found = False¶

get_properties(model, name, param)¶

This method is called with all items of model.named_parameters() to compose the parameter groups for the whole model. If the desired parameter name is found, it returns a modifier that scales down the learning rate.

Parameters:

model (torch.nn.Module) – Model from which the parameter originates from. Used to extract properties (e.g., number of layers for a layerwise learning rate decay).
name (str) – Name of the parameter as stored inside the model.
param (torch.Tensor) – The parameter tensor.

Return type:

dict[str, float]

was_applied_successfully()¶

Check if the parameter was found within the model.

Return type:: bool