Struct AdamW 

pub struct AdamW<B>(pub Adam<B>)
where
    B: Backend;

AdamW optimizer (Adam with decoupled weight decay).

This is the standard optimizer for training Transformers. The key difference from Adam: weight decay is applied directly to the parameters, not mixed into the gradient.

Tuple Fields

0: Adam<B>

Implementations

impl<B> AdamW<B>

where B: Backend,

pub fn new(params: Vec<Tensor<B>>, lr: f64, weight_decay: f64) -> AdamW<B>

Create an AdamW optimizer with standard defaults.

Default: lr=1e-3, β1=0.9, β2=0.999, ε=1e-8, weight_decay=0.01

pub fn weight_decay(self, wd: f64) -> AdamW<B>

Set weight decay.

pub fn beta1(self, beta1: f64) -> AdamW<B>

Set β1.

pub fn beta2(self, beta2: f64) -> AdamW<B>

Set β2.

pub fn params(&self) -> &[Tensor<B>]

Access current parameters.

pub fn params_mut(&mut self) -> &mut Vec<Tensor<B>>

Mutable access to current parameters (for checkpoint loading).

pub fn set_lr(&mut self, lr: f64)

Set the learning rate (used by LR schedulers).

Trait Implementations

impl<B> Optimizer<B> for AdamW<B>

where B: Backend,

fn step(&mut self, grads: &GradStore<B>) -> Result<Vec<Tensor<B>>, Error>

Perform one optimization step.

fn learning_rate(&self) -> f64

Return the current learning rate.

fn set_learning_rate(&mut self, lr: f64)

Set a new learning rate (for learning rate scheduling).

impl<B> Stateful for AdamW<B>

where B: Backend,

fn state_dict(&self) -> OptimizerState

Export the optimizer’s internal state as a serializable dictionary.

fn load_state_dict(&mut self, state: &OptimizerState) -> Result<(), Error>

Restore the optimizer’s internal state from a previously saved dictionary.