Struct CosineWarmupLR 

pub struct CosineWarmupLR { /* private fields */ }

Linear warmup from 0 to initial_lr over warmup_steps, then cosine decay from initial_lr to min_lr over the remaining steps.

This is the standard scheduler used for training transformers (GPT, BERT, etc.).

warmup phase (step < warmup_steps):
  lr = initial_lr * step / warmup_steps

decay phase (step >= warmup_steps):
  progress = (step - warmup_steps) / (total_steps - warmup_steps)
  lr = min_lr + 0.5 * (initial_lr - min_lr) * (1 + cos(π * progress))

Implementations

impl CosineWarmupLR

pub fn new( initial_lr: f64, warmup_steps: u64, total_steps: u64, min_lr: f64, ) -> CosineWarmupLR

Create a cosine warmup scheduler.

Arguments

• initial_lr: Peak learning rate (reached at end of warmup)
• warmup_steps: Number of linear warmup steps
• total_steps: Total training steps (warmup + decay)
• min_lr: Minimum learning rate at end of training

Trait Implementations

impl LrScheduler for CosineWarmupLR

fn step(&mut self) -> f64

Advance by one step and return the new learning rate.

fn current_lr(&self) -> f64

Get the current learning rate without advancing.

fn current_step(&self) -> u64

Get the current step count.

fn reset(&mut self)

Reset the scheduler to step 0.

fn set_step(&mut self, step: u64)

Set the internal step counter to a specific value (for checkpoint restore).