Struct LayerNorm 

pub struct LayerNorm<B>
where
    B: Backend,
{ /* private fields */ }

Layer Normalization: normalizes over the last N dimensions.

Example

let ln = LayerNorm::<CpuBackend>::new(512, 1e-5, DType::F64, &dev)?;
let x = CpuTensor::rand((2, 10, 512), DType::F64, &dev)?;
let y = ln.forward(&x)?; // same shape, normalized

Implementations

impl<B> LayerNorm<B>

where B: Backend,

pub fn new( normalized_size: usize, eps: f64, dtype: DType, device: &<B as Backend>::Device, ) -> Result<LayerNorm<B>, Error>

Create a new LayerNorm layer.

Arguments

• normalized_size: size of the last dimension to normalize
• eps: numerical stability constant (typically 1e-5)
• dtype: data type for parameters
• device: device to create parameters on

pub fn from_tensors( weight: Tensor<B>, bias: Tensor<B>, eps: f64, ) -> Result<LayerNorm<B>, Error>

Create from existing weight and bias tensors.

pub fn eps(&self) -> f64

pub fn normalized_size(&self) -> usize

Trait Implementations

impl<B> Module<B> for LayerNorm<B>

where B: Backend,

fn forward(&self, x: &Tensor<B>) -> Result<Tensor<B>, Error>

Forward pass: normalize over last dimension, then scale + shift.

For input [batch, seq, d_model]:

1. mean = mean(x, dim=-1, keepdim=true) → [batch, seq, 1]
2. var = var(x, dim=-1, keepdim=true) → [batch, seq, 1]
3. x_norm = (x - mean) / sqrt(var + eps) → [batch, seq, d_model]
4. output = x_norm * γ + β → [batch, seq, d_model]

fn parameters(&self) -> Vec<Tensor<B>>

Return all trainable parameters of this module. The optimizer uses these to update weights during training.

fn named_parameters(&self) -> Vec<(String, Tensor<B>)>

Return all trainable parameters with human-readable names.

fn set_training(&self, _training: bool)

Set training or evaluation mode.

fn is_training(&self) -> bool

Whether the module is in training mode (default: true).

fn train(&self)

Convenience: set training mode.

fn eval(&self)

Convenience: set evaluation mode.

fn num_parameters(&self) -> usize

Total number of scalar parameters in this module.

fn trainable_params_count(&self) -> usize

Number of trainable (variable) parameters.

fn frozen_parameters(&self) -> Vec<Tensor<B>>

Freeze all parameters: returns new parameter tensors with is_variable = false, preventing gradient accumulation.

fn state_dict(&self) -> Vec<(String, Tensor<B>)>

Returns a state_dict-style map of parameter name → tensor.