Introducing AutoRound: Intel’s Advanced Quantization for LLMs and VLMs

What changed

Low-bit LLM Leaderboard Track, rank and evaluate open LLMs and chatbots AutoRound is a weight-only post-training quantization (PTQ) method developed by Intel. It uses signed gradient descent to jointly optimize weight rounding and clipping ranges, enabling accurate low-bit quantization (e.g., INT2 - INT8) with minimal accuracy loss in most scenarios. For example, at INT2, it outperforms popular baselines by up to 2.1x higher in relative accuracy.

Why it matters

A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.

How it compares

Related prior coverage to compare against:

• Fine-tuning LLMs to 1.58bit: extreme quantization made easy
• Stable Diffusion XL on Mac with Advanced Core ML Quantization
• Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA

• Fine-tuning LLMs to 1.58bit: extreme quantization made easy similarity 0.22
• Stable Diffusion XL on Mac with Advanced Core ML Quantization similarity 0.21
• Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA similarity 0.20