Google releases its LLM (Large Language Model) VaultGemma

By running experiments with varying model sizes and noise-batch ratios, the team established a basic understanding of differential privacy scaling laws, which is a balance between the compute budget, privacy budget, and data budget. In short, more noise leads to lower-quality outputs unless offset with a higher compute budget (FLOPs) or data budget (tokens). The paper details the scaling laws for private LLMs, which could help developers find an ideal noise-batch ratio to make a model more private. Building VaultGemma

This work on differential privacy has led to a new open-weight Google model called VaultGemma. The model uses differential privacy to reduce the possibility of memorization, which could change how Google builds privacy into its future AI agents. For now, though, the company's first differential privacy model is an experiment.

VaultGemma is based on the Gemma 2 foundational model, which is a generation behind Google's latest open model family. The team used the scaling laws derived from its initial testing to train VaultGemma with the optimal differential privacy. This model isn't particularly large in the grand scheme, clocking in at just 1 billion parameters. However, Google Research says VaultGemma performs similarly to non-private models of a similar size.

https://arstechnica.com/ai/2025/09/google-releases-vaultgemma-its-first-privacy-preserving-llm/