Module scheduler
Source - CosineAnnealingLR
- Cosine annealing from
initial_lr to min_lr over total_steps. - CosineWarmupLR
- Linear warmup from 0 to
initial_lr over warmup_steps, then cosine
decay from initial_lr to min_lr over the remaining steps. - ExponentialLR
- Multiply the learning rate by
gamma every step. - LinearLR
- Linearly interpolate the learning rate from
start_factor * initial_lr
to end_factor * initial_lr over total_steps steps. - ReduceLROnPlateau
- Reduce the learning rate when a monitored metric plateaus.
- StepLR
- Multiply the learning rate by
gamma every step_size steps.
- LrScheduler
- Trait for learning rate schedulers.