Struct QuantConfig 

pub struct QuantConfig {
    pub bits: QuantBits,
    pub mode: QuantMode,
    pub granularity: QuantGranularity,
}

Full quantization configuration.

Fields

bits: QuantBits

Bit width: Int8 or Int4.

mode: QuantMode

Symmetric or asymmetric quantization.

granularity: QuantGranularity

Per-tensor or per-channel granularity.

Implementations

impl QuantConfig

pub fn int8() -> Self

Create INT8 symmetric per-tensor config (most common).

pub fn int8_per_channel() -> Self

Create INT8 symmetric per-channel config (best accuracy for weights).

pub fn int4() -> Self

Create INT4 symmetric per-tensor config (maximum compression).

pub fn int4_per_channel() -> Self

Create INT4 per-channel config.

pub fn asymmetric(self) -> Self

Set mode to asymmetric.

Trait Implementations

impl Clone for QuantConfig

fn clone(&self) -> QuantConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for QuantConfig

fn default() -> Self

Returns the “default value” for a type.