Google Research: Private AI model analytics without accessing user data via Zero-Trust aggregation and TEE

Google Research published details of a private analytics system that addresses the fundamental challenge of evaluating on-device AI models: how to learn the real performance of models on user devices without ever accessing private user data.

What problem does Zero-Trust aggregation solve?

Teams developing AI models that run on-device (on-device AI — models that execute locally on a phone or computer, without sending data to the cloud) face fundamental evaluation questions:

• Are model performance metrics drifting in real-world use?
• Are there hidden biases in specific geographic or demographic contexts?
• What are the actual error rates in the wild (not in a controlled test environment)?

Collecting this data through classical methods requires access to user data. Zero-Trust aggregation resolves this dilemma: aggregate statistics are collected with a mathematical guarantee of no access to individual data.

How does the technical system work?

The system combines two independent layers of protection:

Lattice-based cryptographic protocol (cryptography based on mathematical lattices — a branch of post-quantum cryptography resistant to quantum computer attacks): the user’s device encrypts local data in such a way that the server can sum (aggregate) the ciphertext values, but cannot reconstruct the individual data of any single user. Only the aggregate can be decrypted.

A particular advantage: the protocol supports one-shot sending — the device sends one message and can disconnect. Standard private summation protocols require multi-round communication, which is impractical for mobile devices with intermittent connectivity.

TEE (Trusted Execution Environment): a special isolated processor space that executes code in memory inaccessible to the rest of the system. The TEE provides cryptographic proof (attestation) that the server is genuinely running the intended code — not even Google’s own infrastructure team can modify the processing logic without the attestation revealing it.

Application in Android SafetyCore

The system has been concretely deployed in Android System SafetyCore — a Google system service available on Android 9+ devices. SafetyCore provides private on-device support for Android security features, and Zero-Trust aggregation enables SafetyCore developers to:

• Evaluate the accuracy of harmful content classifiers
• Identify detection errors
• Improve models based on aggregate metrics

All of this without any access to the private content of users whose devices participate in the evaluation.

Why is the two-layer approach more robust than a single-layer one?

The authors emphasize the key advantage of the combined architecture: the cryptographic layer provides protection even if TEE security fails. Each layer independently protects user privacy, making the system more resilient to complex attacks that might compromise one layer but not both simultaneously. The detailed methodology is available in the cryptographic paper published in the ACM DL repository.