189 articles
Super Weights in LLMs is a paper accepted at COLM 2026 that identifies 'super weights' — a small number of parameters whose modification disproportionately changes a language model's behavior. The paper shows that these weights explain why selective fine-tuning of only certain layers often fails, with direct implications for PEFT and LoRA approaches.
SensorFM is Google's foundation model for health data from wearable devices, trained on more than one trillion minutes of signals from Fitbit and Pixel Watch devices worn by 5 million users in over 100 countries. The model outperforms specialized approaches on 34 of 35 tasks, with +9% AUC on classification and +21% correlation on regression.
Aurora 1.5 is Microsoft's open-source foundation model for the Earth system that outperforms the ECMWF ensemble forecast on 88.9% of evaluated variables and horizons. The new release adds 22 meteorological variables, hourly resolution, and probabilistic ensemble forecasting — achieving around 33% lower track error for Hurricane Helene compared to the original Aurora.
GPT-5.6 is OpenAI's new model family with three variants: Sol (flagship for complex reasoning), Terra (balanced), and Luna (high-volume, cost-efficient). It introduces Programmatic Tool Calling, explicit prompt cache control, persisted reasoning and multi-agent orchestration in beta — available from day one in GitHub Copilot.
Muse Spark 1.1 is a multimodal reasoning model from Meta Superintelligence Labs for agentic tasks, with a context window of one million tokens. The model coordinates sub-agents, navigates flows across multiple apps, and accepts images, video, and PDFs — available through the Meta Model API in public preview and in Meta AI's Thinking mode.
Grok 4.5 is xAI's new model positioned as a flagship for coding and agentic tasks, priced at $2 per million input and $6 per million output tokens. It supports configurable reasoning effort (low/medium/high) and is already available through Perplexity's Agent API, aggressively undercutting comparable frontier models.
Mistral introduced Robostral Navigate, its first model for embodied robotic navigation with 8 billion parameters. Using only a single RGB camera — no LiDAR or depth sensors — it achieves 76.6% success on the R2R-CE benchmark for unseen environments and surpasses multi-sensor competitors by 4.5 percentage points.
OpenAI introduced GPT-Live, a new voice model designed for natural, lifelike conversational AI interactions. Integrated into ChatGPT Voice from day one, it arrives as part of an accelerated voice model development cycle — just two days after GPT-Realtime-2.1. Technical specifications and pricing have not yet been published.
Anthropic and AE Studio announce GRAM (Gradient-Routed Auxiliary Modules) — a method that isolates dual-use knowledge such as virology, cybersecurity, and nuclear physics into removable neural modules during training, enabling a single training run to produce multiple model variants with different capability sets.
Meta Superintelligence Labs has unveiled Muse Image and Muse Video — models that operate as agents, internally invoking code and web-search tools, ranking #2 and #3 on the Arena leaderboard, with mandatory Content Seal watermarking.
Researchers from Tsinghua University and Zhipu AI proposed Direct On-Policy Distillation (Direct-OPD) — a method that transfers reinforcement learning gains from a weaker teacher model to a stronger student without imitating the final policy, achieving a jump from 48.3% to 62.4% on AIME 2024.
Anthropic researchers have identified an emergent internal structure inside Claude called J-space, discovered using a new technique called Jacobian lens (J-lens). Inspired by the neuroscientific Global Workspace Theory, J-space acts as a silent internal reasoning space invisible in the model's output, and can reveal hidden behaviors such as data fabrication, test-scenario recognition, and implanted malicious goals.
Amazon Web Services has published a selective unlearning technique for Nova models based on Reverse Direct Preference Optimization (rDPO). Implemented via LoRA adapters without full retraining, the technique reduces unwanted refusal rates by up to 53.74 percentage points with minimal loss of utility.
On July 6, 2026, OpenAI released two new realtime voice models — GPT-Realtime-2.1 and GPT-Realtime-2.1-mini — available on the existing v1/realtime endpoint. The models bring improved alphanumeric string recognition, better handling of silence and noise, and improved interruption behavior.
Amazon Bedrock has expanded its offering with three open-weight MiniMax models — M2 with a one-million-token context window, M2.1 designed for deep reasoning and coding, and M2.5 with a 230-billion-parameter Mixture-of-Experts architecture designed for agentic workloads.
Researchers from the University of Illinois developed ReContext — an inference technique that recursively replays relevant evidence from a long context window and consistently improves performance across three LLM architectures over eight benchmarks, without any retraining.
Liyan Tang, Fangcong Yin, and Greg Durrett developed VRRL — a reinforcement learning framework that uses trajectory prefix masking and experience replay to force vision-language models to ground self-reflection in real visual input, achieving significantly better performance on out-of-distribution examples.
On July 1, Anthropic restored access to Claude Fable 5 and Claude Mythos 5, nearly three weeks after withdrawing them due to US export controls. The return follows an improved safety classifier and a proposed industry framework for rating jailbreak severity.
AWS GovCloud (US) is receiving six new models on Amazon Bedrock: OpenAI open-weight gpt-oss-120b and gpt-oss-20b, and four NVIDIA Nemotron models with a 1M token context. The infrastructure meets FedRAMP High, DoD IL 2/4/5, ITAR, and CJIS requirements with a zero operator-access design.
GitHub has declared GA two Copilot capabilities: Vision for attaching images and PDFs to chat prompts, and browser tools that give agents in VS Code control over a real browser. Both are available to all plans without admin action.
Anthropic today launches Claude Sonnet 5 — a model that plans multi-step tasks, controls browsers and terminals, and approaches Opus 4.8 performance on key benchmarks, with an introductory price of $2/$10 per million tokens through August 31, 2026, and a 1M token context window.
After 18 days of global suspension, Anthropic announces the redeployment of Fable 5: the U.S. government lifted export controls on June 30, and an improved safety classifier blocks the discovered bypass technique in more than 99 percent of cases.
On June 30, 2026, Google simultaneously announced Gemini Omni Flash for conversational video editing and Nano Banana 2 Lite for rapid image synthesis. The video model is priced at $0.10 per second of output, while the image model costs just $0.034 per thousand images.
Google Research has published TabFM, a foundation model for tabular data that delivers zero-shot predictions in a single forward pass, without hyperparameter tuning or feature engineering. The model achieved top Elo scores on the TabArena benchmark and is available on Hugging Face and GitHub, with a planned integration into Google BigQuery.
Brain2Qwerty v2 is a Meta Research AI system that converts brain signals recorded outside the body — without surgery — into typed text at an average word-level accuracy of 61%, using MEG scanning. This is seven times higher than other non-invasive methods (8%). Training code and datasets have been released as open source.
Claude Opus 4.8 fast mode is now in preview for GitHub Copilot users, delivering significantly faster output token generation while maintaining the model's intelligence level. Simultaneously, Anthropic is retiring fast mode for Opus 4.6 — consolidating fast mode capabilities on the sole remaining model.
DiScoFormer is an Allen Institute for AI (AI2) transformer model that estimates the density function (distribution density) and score function in a single forward pass — previously requiring separate models. It generalizes KDE to high dimensions and adapts to new distributions without retraining.
Tandem RL is a new language model training method that combines RLVR (reinforcement learning with verifiable rewards) with a tandem approach: a stronger model collaborates with a frozen weaker model during chain-of-thought generation. On Qwen3-4B it achieves comparable performance with significantly better readability and robustness when handing off to a smaller model.
MAI-Code-1-Flash is Microsoft's proprietary coding model that became generally available on June 26, 2026 for GitHub Copilot Business and Enterprise plans. Optimized for low latency and high-frequency agentic coding workflows, it is billed on a usage-based model according to the provider's pricing, and administrators must explicitly enable it in organization settings.
A study by Jingyu Liu and colleagues (arXiv:2606.26935) shows that gains from chain-of-thought (CoT) training in LLM agents land in stronger direct action prediction rather than broader reasoning advantage. Later checkpoints revise the action less frequently, while masking supervision over action tokens improves out-of-domain generalization.
A study by Han-yu Wang (arXiv:2606.26502) finds that large reasoning models (LRM) spend more tokens on tasks they ultimately get wrong than on those they solve correctly, the opposite of humans who disengage on harder tasks. The gap is large (Cohen's d 1.47–3.13 on the H-ARC benchmark), and all five tested models showed the reverse pattern from humans.
GPT-5.6 Sol is OpenAI's announced next-generation model, currently in preview status (not generally available), with enhanced capabilities in coding, scientific reasoning, and cybersecurity, along with the most advanced safety stack to date.
Anthropic has equalized API rate limits across all models — Sonnet and Haiku now share the same quotas as Opus at each of the three usage tiers (Start, Build, Scale). At the same time, fast mode for Claude Opus 4.7 is being deprecated, with removal scheduled for July 24.
Researchers showed that standard benchmarks — measuring only one model in one attempt — underestimate the true capabilities of LLMs by as much as 82%. By introducing the Capability Frontier framework, which uses Pareto optimality across 21 models and 16 benchmarks, the same accuracy is achievable at 85% lower cost.
Google accelerated Gemini Nano inference on Pixel 9 and 10 by more than 50% using frozen multi-token prediction — a technique that generates an average of roughly 2 tokens per model pass, saving 130 MB of memory per instance with no change to output results.
A study of 67 frontier models from 21 providers introduces the concept of co-failure ceiling — the upper accuracy bound of an LLM ensemble determined by the rate at which all models fail on the same query. Results show that combining models rarely beats the single strongest model without query-level routing.
Google Research reveals two mechanisms by which a reasoning trace improves retrieval of facts stored in model weights — computational buffer and factual priming — tested on Gemini 2.5 and Qwen3-32B.
OPPO is a reinforcement learning system that trains omni-modal language models to simultaneously understand visual, acoustic, and textual emotion cues, suppressing cross-modal hallucinations and achieving SOTA results on two benchmark datasets.
RaDaR is an open-source reasoning LLM with 32 billion parameters trained for rare disease diagnosis. In a randomized clinical trial it improved physician diagnostic accuracy by 21.44 percentage points versus internet search, with the ability to identify diagnoses in 61% of cases before clinical documentation.
Google Research researchers showed that RL training on beneficial properties such as truthfulness, fairness, and corrigibility improves performance on more than 80% of 50+ independent OOD benchmarks — including domains outside health on which the model was trained.
DiffusionGemma is Google's 26B MoE model that generates text using a diffusion approach — in parallel rather than sequentially. It achieves more than 1,000 tokens per second on a single H100 GPU, up to 4× faster than standard autoregressive models, with a quality trade-off versus Gemma 4.
Google launched Gemini 3.5 Live Translate — a speech-to-speech translation system supporting 70+ languages and more than 2,000 language combinations in real time, with intonation preservation and SynthID watermark protection.
DART is a routing method that decides without any training whether an AI model needs to think deeply or can respond immediately — reducing thinking token consumption by 15–69% while simultaneously improving accuracy by up to +22.5 points on code benchmarks.
Mistral OCR 4 is a new optical character recognition model that tops the OlmOCRBench leaderboard with 85.20 points, supports 170 languages, and delivers paragraph-level bounding boxes — all at a price of $4 per 1,000 pages.
The PyTorch team and SGLang increased the serving throughput of the DeepSeek-V4 Pro model on NVIDIA GB300 architecture from around 2,200 to over 11,200 tokens per second per GPU between April and June 2026 — a fivefold improvement without sacrificing end-user interactivity.
DiffusionGemma is Google's diffusion language model operating in continuous latent space. A study by 13 authors led by Neel Nanda shows that the initial opacity is 28.6× greater than Gemma 4, but an interpretable token bottleneck narrows that gap to just 1.1×.
SSD (Spatially Speculative Decoding) is a new method that simultaneously predicts the horizontal and vertical neighbor of a pixel in autoregressive image generation, achieving up to 13.3× speedup without any loss of visual quality on the DPG-Bench and GenEval benchmarks.
TimeProVe is a framework that accelerates VLM inference over long videos by introducing a two-stage propose-then-verify approach. It reduces calls to expensive models by 75% and total inference cost by 93%, while outperforming the strongest competitor by 7.3 percentage points on the new OpenTSUBench benchmark.
VIMPO is a new reinforcement learning method for LLM reasoning that derives an implicit value function from KL-regularized RL — without a separate critic network. It outperforms GRPO on four mathematical benchmarks including AIME 2024 and AIME 2025, with gains that remain stable even under noisy reward conditions.
SoftSkill is a method described in the paper arXiv:2606.20333 that converts skill documents, such as Markdown SKILL.md files, into compact continuous latent objects that guide model behavior without modifying the base model. Instead of hundreds or thousands of instruction tokens, SoftSkill uses 32 virtual tokens and achieves an improvement of 42.1 percentage points on the LiveMath task compared to operating without the skill.