arXiv:2607.08733: 'Super Weights' Explain Why Selective Fine-Tuning Fails — Paper Accepted at COLM 2026
Super Weights in LLMs is a paper accepted at COLM 2026 that identifies 'super weights' — a small number of parameters whose modification disproportionately changes a language model's behavior. The paper shows that these weights explain why selective fine-tuning of only certain layers often fails, with direct implications for PEFT and LoRA approaches.