AMD Primus Tuning Agent automatically finds the optimal LLM configuration for MI355X clusters

Manually configuring large language model training on multi-node GPU clusters is one of the most expensive engineering tasks in AI infrastructure. The combinatorial space of parameters — parallelisms, schedules, micro-batch sizes, MoE backends — reaches tens of thousands of valid configurations, each full pass through the cluster requiring hours of GPU time. The AMD ROCm team has responded with Primus Tuning Agent, which searches that space automatically.

Primus Tuning Agent: automating LLM training configuration discovery

The agent was validated on a 4-node AMD Instinct MI355X cluster with a total of 64 GPUs, using the Mixtral 8x22B model in FP8 precision, with a global batch size of 512 and a sequence length of 8,192 tokens. The target metric was throughput per GPU — the number of tokens generated per second per card.

The configuration space includes five parallelism dimensions (tensor, pipeline, expert, context, and data), micro-batch sizes from 1 to 8, recomputation granularity (none/selective/full), MoE backends (All-to-All vs. DeepEP), and various pipeline schedules (1F1B, Interleaved, Zero-Bubble variants). The number of topologically valid points is approximately 10 to the fourth power — exhaustive brute-force search is not practical.

How does the agent search the combinatorial configuration space?

Primus Tuning Agent uses a two-phase approach combining determinism with LLM-guided reasoning.

In the first phase, a deterministic seed planner steps through each parameter axis independently — fixing all other values and measuring the effect of changing one parameter. This single-axis sweep establishes reference points and eliminates clearly weak options without expensive cross-axis experiments.

In the second phase, an LLM-guided exploration loop uses a DSPy planner with LiteLLM routing to identify combinations the single-axis sweep missed. The LLM reasons about trade-offs between axes — for example, how the choice of MoE backend affects the optimality of the pipeline schedule — and proposes candidate configurations for measurement. The agent supports OpenAI, Anthropic, and other LLM providers without a separate proxy infrastructure.

An analytical memory pre-filter evaluates each candidate configuration without running GPU kernels. In the case study, the filter rejected 7 out of 30 trial configurations at zero GPU cost, by analytically ruling out those that exceed available memory. This is critical to search scalability.

Results on Mixtral 8x22B and MI355X cluster

The agent found a configuration achieving 4,402 tokens/s per GPU, representing a +27% measured throughput improvement over AMD’s published BF16 reference of 3,475 tok/s/GPU. The projected optimal result is 4,908 tok/s/GPU (+41.2%), with a documented ~10% optimistic bias in projection relative to measured values — which the agent records as a known characteristic, not an anomaly.

The key to the discovered improvement lies in cross-axis synergy: the combination of DeepEP MoE backend, a specific pipeline schedule, and recomputation granularity was visible neither in the single-axis sweep nor in the naive assumption that axes can be optimized independently. The LLM-guided loop identified that synergy by reasoning about trade-offs that a deterministic planner structurally cannot consider.

The entire search took ~30 minutes, driven primarily by single-node profiling data that is then extrapolated to the full-cluster scenario.

Integration and reproducibility

The agent documents each phase of the search: which axes were explored, which memory filters were applied, and how projected results compared to measured ones. Every run is reproducible and auditable, which is important for teams that must justify infrastructure decisions within the organization or repeat the experiment with a different LLM in the exploration loop.

The DSPy framework with the LiteLLM layer means users can use their own LLM endpoint for the exploration loop, keeping search costs controlled and independent of any specific cloud provider. Support for OpenAI, Anthropic, and other providers comes without a separate proxy infrastructure, reducing the operational overhead of setting up the agent.

For teams scaling AMD Instinct workloads, Primus Tuning Agent offers a reproducible path from initial configuration to measurable improvement — without weeks of manual experimentation on expensive cluster time. The projected gain of +41.2% and measured +27% remain consistent with the projection difference documented by the tool itself, making capacity planning results more reliable than benchmark tables alone would allow.