BWLA: 1-bit LLM quantization with 3.26× speedup and 70% better results (ACL 2026)
BWLA is a new post-training quantization framework for large language models that for the first time achieves simultaneous 1-bit weight precision and low-bit activations without significant accuracy loss. On the Qwen3-32B model it reaches a perplexity of 11.92 and a 3.26× speedup compared to previous methods.
This article was generated using artificial intelligence from primary sources.
Researchers Zhixiong Zhao, Zukang Xu, and Dawei Yang have introduced BWLA (Breaking the Barrier of W1AX), a post-training quantization framework — reducing model weight precision after training — that for the first time simultaneously achieves 1-bit weights and low-bit activations without serious accuracy degradation. The paper has been accepted at ACL 2026 (main conference).
Why has 1-bit quantization been so difficult?
Previous approaches to binarizing LLMs (reducing weights to values of 0 or 1) stumbled on so-called heavy-tail activations — extreme values appearing in intermediate network layers that demand high numerical precision. Without solving this problem, models would lose accuracy as soon as activations were compressed below 8 bits.
How does BWLA solve this?
BWLA introduces two novel mechanisms. The Orthogonal-Kronecker Transformation (OKT) learns orthogonal mappings to reshape the weight distribution and suppress activation artifacts, eliminating the need for high precision. The Proximal SVD Projection (PSP) then performs low-rank refinements with minimal computational overhead — all without retraining the model.
What do the results show?
On Qwen3-32B, BWLA achieves a perplexity (a measure of language model quality — lower is better) of 11.92, while the previous state of the art stands at 38. Zero-shot tasks improved by more than 70% and inference accelerated 3.26 times. The authors claim this is the first post-training framework that makes W1AX — 1-bit weights with X-bit activations — practically applicable without accuracy trade-offs.
Frequently Asked Questions
- What is LLM quantization and why does it matter?
- Quantization is a technique that reduces model weight precision (e.g., from 32 bits to 1 bit) to shrink memory footprint and accelerate inference. It is critical for running large models on resource-constrained devices.
- How does BWLA address the 'heavy-tail' problem in activations?
- BWLA uses an Orthogonal-Kronecker Transformation (OKT) that learns orthogonal mappings to reshape the weight distribution and suppress activation artifacts, eliminating the need for high activation precision.
- How much better is BWLA than the previous state of the art?
- On Qwen3-32B, BWLA achieves a perplexity of 11.92 versus 38 for prior methods — an improvement of over 70% on zero-shot tasks with a 3.26× inference speedup.
Related news
AdaMeZO: Adam-style LLM fine-tuning without storing gradient moments in GPU memory
KellyBench: AI agents managing a betting bankroll through the Premier League season — all leading models lost money
Latent-GRPO: Stable RL Optimization for Latent Reasoning — 7.86 Points on GSM8K-Aug and 4.27 Points on AIME With 3-4× Shorter Reasoning Chains