Struct QuantizedLinear 

pub struct QuantizedLinear<B: Backend> {
    pub in_features: usize,
    pub out_features: usize,
    /* private fields */
}

A quantized linear layer that stores weights in INT8/INT4.

During inference, weights are dequantized on-the-fly for computation. This saves memory (2-8× depending on bit-width) at the cost of a small dequantization overhead.

Example

// Quantize a trained linear layer
let linear = Linear::new(256, 10, true, DType::F32, &dev)?;
// ... train ...
let qlinear = QuantizedLinear::from_linear(&linear, &QuantConfig::int8())?;
let output = qlinear.forward(&input)?;

Fields

in_features: usize

Input features.

out_features: usize

Output features.

Implementations

impl<B: Backend> QuantizedLinear<B>

pub fn from_linear(linear: &Linear<B>, config: &QuantConfig) -> Result<Self>

Create a QuantizedLinear from a trained Linear layer.

pub fn new( weight_q: QuantizedTensor, bias: Option<Tensor<B>>, device: B::Device, ) -> Self

Create from raw quantized data and optional bias tensor.

pub fn weight_quantized(&self) -> &QuantizedTensor

Get the quantized weight data.

pub fn memory_savings_bytes(&self) -> usize

Memory saved compared to FP32 weight storage.

Trait Implementations

impl<B: Backend> Debug for QuantizedLinear<B>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<B: Backend> Module<B> for QuantizedLinear<B>

fn forward(&self, x: &Tensor<B>) -> Result<Tensor<B>>

Compute the output tensor from the input tensor. This defines the layer’s computation (forward pass).

fn parameters(&self) -> Vec<Tensor<B>>

Return all trainable parameters of this module. The optimizer uses these to update weights during training.

fn set_training(&self, _training: bool)

Set training or evaluation mode.

fn is_training(&self) -> bool

Whether the module is in training mode (default: true).

fn train(&self)

Convenience: set training mode.

fn eval(&self)

Convenience: set evaluation mode.

fn num_parameters(&self) -> usize

Total number of scalar parameters in this module.

fn trainable_params_count(&self) -> usize

Number of trainable (variable) parameters.

fn frozen_parameters(&self) -> Vec<Tensor<B>>

Freeze all parameters: returns new parameter tensors with is_variable = false, preventing gradient accumulation.

fn named_parameters(&self) -> Vec<(String, Tensor<B>)>

Return all trainable parameters with human-readable names.

fn state_dict(&self) -> Vec<(String, Tensor<B>)>

Returns a state_dict-style map of parameter name → tensor.