OpenAI: CFO Sarah Friar Proposes an AI Scorecard with Four Metrics to Measure ROI on Corporate AI Investment

Why don’t traditional metrics measure the real value of AI?

Traditional metrics like the number of queries or tokens consumed measure usage, not actual business value. OpenAI’s CFO (Chief Financial Officer) Sarah Friar therefore proposes an “AI scorecard” framework that measures return on investment (ROI) through a concrete business outcome, rather than through the volume of activity.

The scorecard’s four metrics

The framework includes four measures: useful work accomplished, cost per successfully completed task (how much a single successful outcome actually costs the company, rather than a single query), system reliability (dependability), and return on invested compute, i.e., the computing resources spent to complete a task. The cost-per-task metric is especially useful for companies comparing the cost of an AI solution with the cost of human labor for the same work.

Comparison with the current approach

Unlike common reports on the number of queries and tokens consumed, which show only the intensity of usage, the AI scorecard aims to measure whether the work was actually done and at what cost. Sarah Friar presents the framework as a tool companies can use to compare the real cost-effectiveness of AI investment instead of tracking raw resource consumption. The goal is for management to make decisions about further AI investment based on measurable business outcomes, rather than solely on vendor reports about a product’s usage level.