54 articles
AMD has published a guide for running diffusion models for image generation and editing on Instinct GPUs via SGLang Diffusion on the ROCm stack. SGLang, an inference framework originally popular for large language models, now extends support to image diffusion, strengthening AMD's AI inference offering beyond the NVIDIA ecosystem.
FlyDSL is AMD's Python domain-specific language for writing GPU kernels that, according to a new ROCm guide, matches the performance of manually optimized HIP C++ code while requiring fewer written lines. AMD published a practical guide for porting existing HIP kernels, positioning FlyDSL as a Python-first alternative to NVIDIA's CUDA toolchain.
The AMD ROCm team published a technical guide on manually computing GPU occupancy for the CDNA4 architecture. Key finding: matrix-bound kernels on the MI355X achieve peak throughput at just 12% occupancy — a counterintuitive result for practitioners coming from the CUDA ecosystem.
The NVIDIA Vera CPU with Olympus cores delivers 50% higher IPC than the Grace processor and 3.4 TB/s core-to-core bandwidth. Perplexity reports 1.5x faster job completion on coding agent workflows.
AMD Quark 0.12 enables MXFP4 quantization of the FLUX.1-dev image generation model on Instinct MI350 GPUs via the Diffusers and xDiT frameworks. With torch.compile, MXFP4 ASM achieves a 1.92x speedup over the BF16 eager baseline with CLIP scores identical to the reference.
AMD ROCm Primus Tuning Agent combines a deterministic seed planner and an LLM-guided exploration loop to automatically discover optimal configurations for training LLMs on AMD Instinct GPUs. On Mixtral 8x22B it achieved +27% measured throughput over AMD's published reference in approximately 30 minutes of search.
Industry experts from Multiverse Computing and Toshiba Corporation explain for OECD.ai where quantum technologies are already changing AI today, and where optimistic expectations remain unsupported by evidence. Tensor networks reduce computational requirements by 10-100x; broader quantum ML applications are not realistic within the next decade.
AMD Research published the open benchmarking framework AgentKernelArena on July 3, 2026, measuring how well AI coding agents optimize real GPU kernels. Across 214 tasks in four categories, AMD's own GEAKv3 (Claude Opus 4.6) leads with a 9.04x speedup on HIP kernels, while Claude Code (Opus 4.6) places second at 6.08x. All experiments were run on an AMD Instinct MI300X within ROCm 7.1.1.
The AMD ROCm team published details on July 3, 2026 about the production deployment of Eagle3 speculative decoding on AMD hardware. The combination of Eagle3's multi-layer approach, the vLLM backend, and AMD Quark FP8 quantization achieves 1.69x to 2.00x throughput for Kimi-K2.5 and 1.38x to 1.79x for MiniMax-M2.5 on AMD Instinct MI355X, with no loss in output quality.
AMD has published a GPU-resident object detection pipeline that uses rocDecode, DLPack, PyTorch, and MIGraphX to ensure video frames never leave VRAM — all the way until the final detections are known.
NVIDIA explains how stacked software optimizations on the Blackwell architecture — from NVFP4 precision to speculative decoding — achieve up to 20× greater throughput and a five-fold reduction in token cost for DeepSeek V4 models.
AMD has released the FlyDSL system within the AITER framework (AI Tensor Engine for ROCm) that automatically generates specialized GEMM kernels for the LLM decode phase on AMD GPUs. Results: 1.64× average latency reduction and 1.79× speedup for the most critical M≤8 token scenarios, tested on Instinct MI355X with 256 compute units.
AMD Resource Manager has gained automatic preemption: it monitors GPU utilization per workload and terminates jobs that fall below a configurable threshold (e.g. 10%) after a set idle timer (e.g. 15 minutes). It offers two policies — preempting only under GPU pressure or always — returning resources held by inactive dev environments to the shared pool.
AMD demonstrated W_MXFP4_A_MXFP6 mixed-precision quantization on the Instinct MI355X accelerator, delivering up to 29% higher throughput while maintaining accuracy close to FP8 standard, using the vLLM framework for production inference.
NVIDIA and AWS announced EC2 G7 instances with the RTX PRO 4500 Blackwell GPU delivering 4.6× better AI inference than the previous G6 generation, while the cuVS library becomes the default in Amazon OpenSearch Serverless with 10× faster vector indexing.
OpenAI and Broadcom jointly announced Jalapeño, a custom ASIC chip optimized for LLM inference. A strategic move by which OpenAI enters the custom silicon category — on equal footing with Google TPU, Apple Neural Engine, and AWS Trainium — and reduces its dependence on NVIDIA GPUs.
ATOM is AMD's open-source inference engine for the MI355X GPU that brings two optimizations for DeepSeek-V4: PrefillDelayer eliminates loss during rank coordination, and Two-Batch Overlap accelerates token balancing by overlapping network operations.
AMD ROCm team demonstrated EAGLE3 speculative decoding on 8× Instinct MI325X with the Kimi-K2.5 model, achieving 33% higher output throughput and 58% lower median inter-token latency without accuracy loss on the GSM8K benchmark.
JUPITER is Europe's first exascale supercomputer, located at Forschungszentrum Jülich in Germany. Powered by NVIDIA Grace Hopper Superchips and a Quantum-X800 InfiniBand network, JUPITER mapped 86 billion neurons, broke the quantum record by simulating 50 qubits, and set a world record in climate simulations.
The NVIDIA Vera CPU with the Olympus core is coming to Los Alamos National Laboratory, delivering 7× better performance on URSA workloads and 3× on Monte Carlo heat-transfer simulations compared to the Crossroads x86 processor. Three new supercomputers — Mission, Vision, and Veritas — will be operational in 2027.
NVIDIA announced three new CUDA-X software libraries for AI in science: cuPhoton for astronomy delivers speedups of up to 14,900×, ALCHEMI at Lila Sciences achieves 50× faster materials screening, and DAQIRI accelerates real-time networking for physics detectors.
AMD described on the ROCm blog the optimization of the Matrix3D framework for generating explorable 3D worlds on AMD Instinct GPUs. By replacing CUDA-specific components with Triton kernels and using the gsplat library for 3DGS, rendering on the MI250 GPU was accelerated by 54 percent and on the MI300 by 50 percent, while the rendering kernel itself is 36 percent faster than the CUDA version.
AMD published a comparative analysis of RoCE network traffic patterns during the training of four large LLMs — GPT-4, Llama 3, DeepSeek-V2, and Grok 4.0 — as a practical guide for building AI infrastructure in scale-out clusters with multiple GPU nodes.
AMD introduced Schola, an open-source Unreal Engine plugin that enables Gymnasium-compatible reinforcement learning training through Python frameworks and gRPC. In the example, a collaborative robotic arm xArm6 is trained in Unreal Engine 5.7 with MuJoCo physics, the PPO algorithm, and PyTorch on the AMD ROCm stack for GPU acceleration. The tutorial demonstrates a reach task in which the arm tip moves to randomly placed target locations.
AMD's MLPerf Training v6.0 results show that the Instinct MI355X is within approximately 5% of an equivalent NVIDIA GPU's performance on LLM benchmarks. MI355X is 3.5× faster than last year's MI300X and 13–19% faster than the previous round. AMD introduced MXFP4 (FP4) training recipes and the Primus unified framework for the first time, alongside a multi-node submission of 512 MI300X GPUs across 64 nodes.
NVIDIA announced that its Blackwell platform achieved the best results on all seven MLPerf Training 6.0 benchmarks, cementing dominance in large-model training. GB300 NVL72 delivers up to 1.6× faster training than GB200 NVL72, and the largest submission used 8,192 Blackwell GPUs on the DeepSeek-V3 model with 671 billion parameters. CoreWeave trained DeepSeek-V3 671B in 2.02 minutes on 8,192 GPUs, while Microsoft Azure completed Llama 3.1 405B in 7.07 minutes.
AMD has introduced ATOM, an inference engine for Instinct GPUs that exposes an OpenAI-compatible API and orchestrates KV cache, scheduling, and parallelism. ATOM sits at the top of the ROCm stack, alongside AITER kernels and MoRI RDMA communication, supports TP, DP, and EP parallelism, and is optimized for MoE models such as DeepSeek V2–V4, Mixtral, and Qwen3-MoE. It offers FP8, MXFP4, INT8, and INT4 quantization and MTP speculative decoding with an EAGLE proposer.
Researchers at UC San Diego, backed by Google, are building a data center from 2,000 retired Pixel smartphones. By retaining the motherboards they save around 50% of embodied carbon, and a cluster of 25 to 50 phones already handles peak workloads for 75 parallel computing courses.
![Editorial illustration: FLUX.2 [klein] ships pre-installed on ASUS ProArt laptops](https://images.24-ai.news/black-forest-labs-flux-outpainting-600.webp)
Black Forest Labs, in partnership with ASUS and NVIDIA, brings FLUX.2 [klein] pre-installed on ASUS ProArt consumer laptops. This is the first time a FLUX model has shipped pre-installed on consumer hardware. The 4-billion-parameter model enables image generation and editing in under one second, locally and without dependence on the cloud.
OpenAI announced the construction of a new Stargate data center in the U.S. state of Michigan. This is an expansion of the Stargate compute infrastructure program, which OpenAI frames within the context of the so-called Intelligence Age. Details on capacity and investment are not elaborated in this announcement, so we point to the official source.
AMD announced that Alibaba's open-source reinforcement-learning framework ROLL now runs natively on AMD Instinct GPUs with the ROCm software, with no code changes, custom patches or non-standard builds. The collaboration includes vLLM compatibility, fixes for Ray and support for distributed RL training of large language models.
On the ROCm blog, AMD presented an implementation of speculative decoding on Instinct MI300X GPUs. The technique, in which a smaller draft model predicts tokens and a larger one verifies them, achieved up to 4.3 times higher throughput compared to classic autoregressive generation.
The AMD ROCm blog describes a new 4-wave interleave approach for FP8 GEMM (mixed-precision matrix multiplication) kernels on the Instinct MI355X accelerator with CDNA 4 architecture. The optimization shifts from an 8-wave ping-pong layout to 4 waves with a full VGPR budget of 512 registers, eliminating LDS memory bank conflicts through XOR-based swizzle and hiding memory latency by precisely overlapping MFMA instructions with data loads.
TokenSpeed, an open-source LLM inference engine from LightSeek Foundation with an MIT license, achieved 580 tokens per second on the Qwen3.5-397B-A17B model using eight NVIDIA Blackwell B200 GPUs in a TP8 configuration. The record was achieved for agentic workloads with prefix caching and CPU-GPU execution overlap, with less than 16% performance degradation even at a 1 million token context.
AMD's ROCm blog published a detailed analysis of LLM inference on the Ryzen AI Max+ 395 processor with UMA (Unified Memory Architecture). A system with 128 GB LPDDR5X memory achieves 42 tokens/s on the Qwen3.5 35B MoE model with full GPU offloading, and also supports models up to 122B parameters by combining CPU and GPU memory.
The AMD ROCm team published a tutorial for writing high-performance GEMM kernels in the Gluon programming model on the MI355 GPU. An optimized FP16 kernel achieves 1.489 TFLOPS at 98.75 percent MFMA efficiency, while extensions to BF8 (3.257 TFLOPS) and MXFP4 (5.255 TFLOPS) demonstrate relevance for modern AI workloads. The tutorial includes workgroup remapping and swizzle that reduces L2 cache misses from 5.3 M to 4.1 M.
AMD released ROCm 7.13 on 20 May 2026 — a new version of its open-source AI compute stack that introduces support for the MI350P GPU, virtualisation of up to 8 isolated vGPUs per MI300X accelerator, an open-source ROCprof Trace decoder for transparent performance analysis, and modular TheRock packaging with domain-specific SDKs. The release is validated on Ubuntu 26.04 and RHEL 9.6, and includes VMware ESXi 9.1 support for MI350X and MI355X.
BubbleFence is a new AMD ROCm AI tool announced on May 15, 2026, that solves the fundamental ML problem of semantically splitting video streams into train/validation/test sets without semantic leakage. Instead of classic metadata-based heuristics, BubbleFence uses vision foundation model embeddings (CLIP) and adaptive bubbles with LID weighting for partitioning. Demonstrated on autonomous driving (Zenseact Open Dataset) and Minecraft gameplay scenarios without configuration changes.
AMD ROCm Kimi-K2.5 quantization for MI325X is a new inference acceleration blueprint published May 14, 2026. It combines the AMD Quark quantization toolkit for converting Kimi-K2.5 models to W4A8 and W8A8 precision formats, the FlyDSL inference serving layer, and the AITER optimization stack. The approach positions a non-NVIDIA inference path for Chinese frontier models and demonstrates AMD's strategy to establish the MI325X as a viable alternative to H100/H200 for open-source LLM serving.
AMD Instinct MI355X is a data center GPU that outperforms NVIDIA B200 in published benchmarks across three ComfyUI generative workflows — text-to-video Wan2.2 (1.44×), text-to-image FLUX.1-dev (1.42×), and 3D Hunyuan3D v2.1 (1.20×) — thanks to AOTriton gfx950 kernels, hipBLASLt GEMM tuning, and other ROCm 7.2.0 optimizations.
NVIDIA Fleet Intelligence is a managed service that monitors large fleets of NVIDIA data center GPUs in real time — power, temperature, performance, and ECC errors — with cryptographic GPU authenticity verification through the NVIDIA Remote Attestation Service. The service is free for owners of Vera Rubin, Blackwell, and Hopper GPUs.
TurboQuant is an aggressive KV-cache quantization method at 3-4 bits that the Red Hat AI team systematically compared against the FP8 standard. Results show FP8 retains throughput and accuracy, while 3bit-nc variants lose approximately 20 percentage points on demanding reasoning benchmarks like AIME25.
NVIDIA Spectrum-X Multipath Reliable Connection (MRC) is an RDMA transport protocol that distributes a single connection across multiple network paths and has now been published as an open specification through the Open Compute Project. MRC is already in production at OpenAI, Microsoft's Fairwater data center and Oracle's Abilene data center, and was developed in collaboration with AMD, Broadcom, Intel and Microsoft.
The AMD ROCm team introduced FarSkip-Collective, a modified MoE architecture that eliminates GPU idle time during Expert Parallelism communication. Results: 18% lower TTFT for Llama-4 Scout, up to 1.34× speedup for DeepSeek-V3, and 11% faster Moonlight pre-training.
The team of Dongxin Guo, Jikun Wu, and Siu Ming Yiu presented on May 1, 2026 SAGA — a workflow-atomic scheduler for AI agents on GPU clusters that treats the entire agent workflow as a single schedulable unit instead of individual LLM calls. The system achieves a 1.64× geometric mean reduction in task completion time on a 64-GPU cluster and 99.2% SLO attainment under multi-tenant load. The paper was accepted at HPDC 2026 in Cleveland (July 13–16, 2026).
AMD Primus Projection is a tool that predicts memory requirements and throughput for LLM training on Instinct GPU clusters before a run begins. It uses analytical formulas combined with real GPU benchmarking, and projections fall within ~10% of measured results on MI325X and MI355X accelerators for Llama and Mixtral models.
Google at Cloud Next '26 unveiled two new generations of TPU chips: TPU 8i for AI agent inference and TPU 8t for training the most complex models. The move formalizes the split of Google's TPU line into two specialized branches within the 'agentic era' of computing.
NVIDIA and Google Cloud announced a joint collaboration to accelerate agentic AI and physical AI workloads, combining NVIDIA GPU infrastructure with the Google Cloud platform for robotics, autonomous systems, and agents.
NVIDIA and HuggingFace demonstrated Gemma 4 as a Vision Language Agent that autonomously decides when to use the camera and runs the entire pipeline, including speech-to-text and TTS, locally on an NVIDIA Jetson Orin Nano Super with 8 GB of memory, with no cloud dependency.
At Cloud Next '26, Google introduced the eighth generation of its TPU chips in two specialized variants — TPU 8t for model training and TPU 8i for agentic inference. This is the first generation purpose-built for autonomous AI agents and multi-step reasoning.