Model Quantization

Why can phones run large models? What are INT4/INT8?

Quantization GGUF INT4/INT8
10 min read
Chapter 01

What Is Precision?

Computers use different levels of "precision" to represent numbers. Higher precision means more possible values and greater accuracy, but also more storage space. Every parameter in a large model is a number — precision determines how much memory is needed to store them.

FP32 (32-bit floating point): Each number takes 4 bytes and can represent about 4 billion distinct values. This is the standard precision used in traditional training.

FP16 (16-bit floating point): Each number takes 2 bytes and can represent 65,536 distinct values. Sufficient precision at half the size.

INT8 (8-bit integer): Each number takes 1 byte with only 256 possible values. A common choice for quantization.

INT4 (4-bit integer): Each number is just half a byte with only 16 possible values. Maximum compression.

Interactive Demo — Precision Comparison

Click different precision levels to see how the gradient bar's "resolution" changes. Lower precision means coarser color blocks.

256 color levels 4 bytes / param
Precision is like image resolution — FP32 is 4K, INT4 is a thumbnail. Most of the time, a thumbnail is good enough.
Chapter 02

Why Quantize?

Quantization offers three core benefits: smaller size, faster speed, and lower hardware requirements. A 7B parameter model needs 14 GB of memory in FP16, but only 4.1 GB when quantized to Q4_K_M — an ordinary laptop can run it.

Interactive Demo — GGUF Quantization Level Comparison (7B Model)
Quantization is like MP3 compression — it removes details imperceptible to the human ear, making the file 10x smaller while sounding nearly the same.
Chapter 03

How Quantization Works

The core idea of quantization is simple: map continuous floating-point numbers to a limited set of integers. This process requires two key parameters — scale (the scaling factor) and zero_point (the zero-point offset).

The formula is intuitive: q = round(x / scale) + zero_point

To dequantize: x' = (q - zero_point) * scale

Some error is inevitable, but as long as the scale is chosen well, the error remains small.

Interactive Demo — Float to Integer Mapping

Watch how floating-point numbers (orange) get mapped to the INT8 grid (teal). Drag the slider to adjust the scaling factor.

1.00

Original float values    INT8 quantization grid    Notice how nearby values merge into the same integer

Quantization is like rounding heights measured to the centimeter to the nearest 5 cm — 170.3 cm → 170 cm, virtually no difference.
Chapter 04

Quality vs Size Trade-off

Different quantization levels make different trade-offs between model size and output quality. Using the Perplexity metric, we can precisely measure this trade-off. The chart below shows the cost-effectiveness of each quantization level — Q6_K sits on the Pareto frontier as the optimal balance between quality and size.

Interactive Demo — Quality vs Size Scatter Plot

Data from AtomGradient's Apple Silicon LLM inference benchmarks. Y-axis shows perplexity increase percentage — lower is better.

Choosing a quantization level is like picking a seat — first class (FP16) is the most comfortable but most expensive, economy (Q4_K_M) offers the best value.
Chapter 05

Summary

🎯
Precision Is Key
From FP32 to INT4, each step down in precision nearly halves the model size. Understanding precision is the foundation of understanding quantization.
Q4_K_M Is the Sweet Spot
For most use cases, Q4_K_M strikes the best balance between size, speed, and quality, at just 30% the size of FP16.
🏆
Q6_K Is Pareto Optimal
If you want higher quality and have the memory for it, Q6_K achieves significant compression with minimal quality loss.
📱
Making AI Accessible
Quantization enables large models that once required server clusters to run smoothly on laptops and even phones.

"Quantization brings large models from the cloud to your pocket — the same intelligence at one-tenth the size."

See what models your device can run in the AtomGradient Calculator → Next: Inference vs Training →