How Schneider Electric Built LLMOps Foundations for 60+ AI Agents with LangSmith

Managing a system of 60+ AI agents serving 160,000 employees across 107 countries is a task for which no ready-made playbook exists. Schneider Electric — the global leader in energy technologies with revenue of approximately 40 billion euros — was forced to build that playbook itself. The result is an LLMOps architecture built on self-hosted LangSmith deployed on AWS EKS, with a clear philosophy: observability, evaluation, and deployment are not optional — they are a prerequisite for a production AI system to function.

Schneider Electric’s internal AI Hub unit comprises 350 specialists and is responsible for building and operationally managing all agent systems within the organization. The company uses the entire LangChain ecosystem, with LangSmith as the central platform for monitoring, debugging, and evaluating models at all levels — from development to production.

Three Pillars of LLMOps Architecture in Production

Schneider Electric organized its LLMOps architecture around three pillars that together form a complete operational framework:

Observability is built on self-hosted LangSmith deployed behind the corporate security perimeter. Each AI product has its own workspace encompassing development, QA, pre-production, and production environments. Production traces are systematically leveraged for evaluation and building regression datasets, making the loop between production and model improvement continuous.

Evaluation operates through three mechanisms. A standardized CLI accelerator for offline evaluation allows teams to run quick checks without ad-hoc scripts. The LLMOps maturity framework tracks the level of instrumentation, evaluation suites, and user feedback for each product individually. About 20% of products have active annotation queues in which domain experts label examples from production — a critical quality assurance step in domains that require specialized expertise.

Deployment follows an isolation model: each AI product runs its own LangSmith Agent Server. The infrastructure uses PostgreSQL, Redis, and LangGraph, and the platform is cloud-agnostic with deployments on both AWS and Azure environments.

Why a Dedicated Agent Server Per Product, Not a Centralized System?

A centralized model sounds appealing — one server for everything, easier management, less duplication. But Schneider Electric chose the opposite approach, and did so deliberately. The “You build it, you run it” philosophy means that each team takes full responsibility for its own product — from development to operational work in production.

The advantages are multiple. There is no single point of failure that would simultaneously bring down all agents. Teams can iterate independently, without coordinating with a central tier that could become a bottleneck. Production changes to one product do not affect the stability of others. In an organization the size of Schneider Electric, where approximately 200 active LangSmith users work daily on different products, this isolation is not a luxury — it is a structural necessity.

CAIO Philippe Rambach emphasizes a dimension that is often underestimated in an enterprise context: “The accuracy challenge, the response quality challenge, the guardrailing challenge — they are very real. When you deploy a solution at scale, you need tools like LangSmith.”

One Jo and Digital Energy: Measurable Results in Production

The flagship production system that embodies all of the architectural principles described above is One Jo — the internal AI assistant serving all 160,000 Schneider Electric employees across 107 countries. One Jo is not merely a chatbot — it continuously feeds improvement datasets that underpin the LLMOps loop for the entire agent ecosystem. Alongside it, the Customer Success Copilot supports more than 250 Customer Success Managers in their daily work.

A second concrete example is the Digital Energy quotation workflow — a system for analyzing request-for-quotation documents in the energy sector. What previously required hours or even days of processing has been reduced to approximately 15 minutes after the introduction of the AI system. That result was not achieved through a single optimization, but through the combination of observability via production traces, continuous evaluation with annotation queues, and robust deployment that guarantees stability without cross-product interference.

For the rest of the industry still searching for justification to invest in LLMOps infrastructure, Schneider Electric offers a concrete example: scale, security, and measurable business outcomes are not contradictions — they are the direct consequence of a systematic approach to managing AI systems in production.