Struct RMSNorm 

pub struct RMSNorm<B>
where
    B: Backend,
{ /* private fields */ }

RMS Normalization layer (used in LLaMA, Mistral, etc.).

Normalizes by root-mean-square without mean centering: y = x / sqrt(mean(x²) + ε) * γ

Example

let rms = RMSNorm::<CpuBackend>::new(512, 1e-5, DType::F64, &dev)?;
let x = CpuTensor::rand((2, 10, 512), DType::F64, &dev)?;
let y = rms.forward(&x)?;

Implementations

impl<B> RMSNorm<B>

where B: Backend,

pub fn new( normalized_size: usize, eps: f64, dtype: DType, device: &<B as Backend>::Device, ) -> Result<RMSNorm<B>, Error>

Create a new RMSNorm layer.

pub fn normalized_size(&self) -> usize

pub fn eps(&self) -> f64

Trait Implementations

impl<B> Module<B> for RMSNorm<B>

where B: Backend,

fn forward(&self, x: &Tensor<B>) -> Result<Tensor<B>, Error>

Compute the output tensor from the input tensor. This defines the layer’s computation (forward pass).

fn parameters(&self) -> Vec<Tensor<B>>

Return all trainable parameters of this module. The optimizer uses these to update weights during training.

fn named_parameters(&self) -> Vec<(String, Tensor<B>)>

Return all trainable parameters with human-readable names.

fn set_training(&self, _training: bool)

Set training or evaluation mode.

fn is_training(&self) -> bool

Whether the module is in training mode (default: true).

fn train(&self)

Convenience: set training mode.

fn eval(&self)

Convenience: set evaluation mode.

fn num_parameters(&self) -> usize

Total number of scalar parameters in this module.

fn trainable_params_count(&self) -> usize

Number of trainable (variable) parameters.

fn frozen_parameters(&self) -> Vec<Tensor<B>>

Freeze all parameters: returns new parameter tensors with is_variable = false, preventing gradient accumulation.

fn state_dict(&self) -> Vec<(String, Tensor<B>)>

Returns a state_dict-style map of parameter name → tensor.