ksuit.schedules.linear_warmup_cosine_decay¶

Classes¶

LinearWarmupCosineDecaySchedule

A cosine annealing scheduler with linear increasing warmup phase."

Module Contents¶

class ksuit.schedules.linear_warmup_cosine_decay.LinearWarmupCosineDecaySchedule(config=None)¶

Bases: ksuit.schedules.base.ScheduleBase

A cosine annealing scheduler with linear increasing warmup phase.”

Example

>>> schedule_config:
>>>   kind: ksuit.schedules.LinearWarmupCosineDecaySchedule
>>>   warmup_percent: 0.05
>>>   end_value: 1.0e-6
>>>   max_value: ${model.optim.lr}

Initialize the scheduler.

Takes either warmup_steps or warmup_percent as argument to determine the length of the warmup phase.

Parameters:: config (ksuit.schemas.scheduler.LinearWarmupCosineDecayScheduleConfig | None) – Configuration for the linear warmup cosine decay schedule.

warmup_steps¶

warmup_percent¶