Google Open-Sources HEIR, a Compiler Toolchain for Running AI Models on Fully Homomorphic Encrypted Data

Overview

Google is showcasing HEIR, an open-source compiler toolchain that converts pre-trained AI models into versions that can run inference directly on encrypted data, according to a Google security blog post published August 14, 2026, and written by staff software engineer Jeremy Kun. HEIR stands for Homomorphic Encryption Intermediate Representation, and Google describes it as “the latest powerful tool added to our Private Computing Toolkit,” per the blog post.

What We Know

The underlying technology is fully homomorphic encryption (FHE), which the blog post describes as “a rapidly maturing technology that fundamentally alters this trade-off by allowing computations to be performed directly on encrypted data,” so that “servers can process ciphertexts and return encrypted results without exposing any underlying information.” Google frames the problem HEIR is meant to solve as a usability one: “manually converting an existing program to use homomorphic encryption efficiently requires a team of cryptographers,” according to the blog post. HEIR is pitched as a fix for that bottleneck — the company says it “can convert pre-trained AI models that operate on unencrypted data to operate on encrypted inputs,” with an eventual goal of becoming “a one-click solution to enable non-experts to incorporate encrypted inference into production applications,” per the blog post.

On the technical side, HEIR’s GitHub repository describes the project as “an MLIR-based toolchain for homomorphic encryption compilers,” released under an Apache-2.0 license. It supports multiple FHE schemes across different backends: BGV, BFV, and CKKS through the OpenFHE and Lattigo libraries, and CGGI through tfhe-rs and Jaxite, according to the repository. The HEIR project site says the toolchain targets “application developers, compiler engineers, hardware designers, and cryptography researchers,” and supports “front-end languages for ease of development, such as Python and Torch.”

HEIR is not a brand-new effort. Its companion demos repository states that the project “has evolved from its origins as a C++ transpiler five years ago into two new open-source libraries”: HEIR itself, and Jaxite, described as “a fully homomorphic encryption backend targeting TPUs and GPUs, written in JAX.” The repository summarizes HEIR’s approach in three stated principles: “Make it easy, Make it fast, Make it scale.” Google says it first announced its intentions for the project in 2023 and has since built a set of hardware and academic partnerships, per the blog post.

On the hardware side, Google says it has “partnered with companies developing hardware accelerators for homomorphic encryption, including Belfort, Niobium, Cornami, and Optalysys,” and plans “to demonstrate the latency benefits of these accelerators in the near future.” On the research side, HEIR has driven collaborations with Georgia Tech, Carnegie Mellon, UC Santa Barbara, Illinois Institute of Technology, Purdue, the University of Edinburgh, and Tsinghua University, among others, according to the blog post. Google says four peer-reviewed publications have been built on HEIR to date, with more in preparation.

To illustrate the toolchain, Google published four private-inference demo applications, each compiled with HEIR and benchmarked for latency on a single-threaded CPU, with source code available in its GitHub repository, per the blog post:

• A deep learning recommendation model for serving private content recommendations, built jointly with Belfort Labs, LG, and New York University.
• A credit card fraud detector, built with Niobium and hardshell.ai.
• An implementation of the Kitsune anomaly-detection system for encrypted network traffic, built with Niobium, which the blog post says “allows a service provider to detect anomalies without revealing the contents of network packets to the service provider.”
• A hotword detector, built with Belfort Labs, which Google says “could allow an audio-triggered AI agent to recognize hotwords while protecting the privacy of the audio recordings.”

Google positions homomorphic encryption as part of a broader lineage of its privacy engineering work, alongside “differential privacy and private set membership to private information retrieval and secure enclaves on Google Cloud.” It draws a specific distinction from hardware-based privacy approaches: “Like private information retrieval, and in contrast to hardware-based solutions, homomorphic encryption’s strong security and privacy guarantees are purely cryptographic,” according to the blog post.

What We Don’t Know

Google’s announcement does not disclose the specific latency figures measured for the four demo applications, only that the numbers were “presented for a single-threaded CPU” in materials linked from the post. The blog post also does not name the four peer-reviewed publications it says were built on HEIR, and does not give a timeline for when hardware-accelerator latency benchmarks with partners Belfort, Niobium, Cornami, and Optalysys will be published.

Analysis

HEIR’s pitch is squarely aimed at a practical adoption gap in homomorphic encryption: the cryptography has existed for years, but converting an ordinary AI model to run on encrypted inputs has required specialized cryptographic expertise most engineering teams don’t have in-house. By building HEIR on MLIR — the same compiler infrastructure underpinning much of Google’s and the broader industry’s machine-learning tooling — and by adding Python and Torch front-ends, Google is positioning the toolchain to plug into existing AI development workflows rather than asking developers to learn a separate cryptography stack. Whether that translates into broad production adoption will likely hinge on the very numbers Google left out of this announcement: real-world latency overhead against unencrypted inference, and how much of that gap the hardware-accelerator partnerships with Belfort, Niobium, Cornami, and Optalysys can close.