# Not to Break, but to Attest: Adversarial Probes for Privacy-Preserving LLM Verification

Crypto.world

> When a model is deployed, it can be quietly modified — through fine-tuning, adapters, model editing, backdoors, or parameter changes — in ways that leave ordinary outputs and public benchmarks…

**arXiv**: 2608.27954

**Published**: 2026-08-28

**Authors**: Cameron Wilding, Mina Shaker, Fatemeh Ganji

## AI summary

## Overview

- **Research area:** Cryptography and security for AI governance — specifically, privacy-preserving post-deployment verification of large language model (LLM) integrity, combining adversarial-style probing with zero-knowledge succinct non-interactive arguments of knowledge (zk-SNARKs).
- **Technical level:** Intermediate. The paper assumes familiarity with LLM inference interfaces (logits, tokenizers, embedding layers) and with the basic vocabulary of zk-SNARKs (statements, witnesses, arithmetic circuits). No deep cryptographic derivation is required to follow the argument.
- **Scope:** The paper proposes a governance-oriented audit framework in which synthetic, non-semantic probes are selected to maximize logit drift between an approved model and a modified deployment, and a Groth16 proof attests that the deployed model matches a committed baseline on that hidden probe set without revealing probes or logits.

## What This Paper Is About

When a model is deployed, it can be quietly modified — through fine-tuning, adapters, model editing, backdoors, or parameter changes — in ways that leave ordinary outputs and public benchmarks looking normal while changing behavior exactly where oversight matters. The authors argue that verifying a deployed model is still "the model that was approved" requires inputs chosen specifically to amplify small hidden changes, rather than average-case or semantic evaluation. Their goal is a mechanism that detects such drift while protecting the model provider's proprietary weights and the secrecy of the audit probes, using a zk-SNARK to produce portable, publicly verifiable evidence.

## Key Contributions

1. **A probe-driven governance framework.** The paper reframes adversarial-example sensitivity from a threat signal into an audit instrument: probes are designed not to break a model but to make selective hidden changes (hidden capability insertion, backdoors, narrow safety weakening, targeted behavioral bias) produce disproportionately large logit drift.
2. **Multiple probe families compared along access and cost axes.** The authors develop token-based probes (black-box access plus tokenizer and vocabulary), embedding-based spike/sparse probes (gray-box access to the embedding interface and embedding dimension), and stress-style probes split into MLP-targeted and attention-targeted constructions motivated by internal pathways. None require full white-box access to weights or architecture.
3. **A hidden-probe audit that is harder to game.** Rather than semantic prompts or a fixed public challenge set, the audit commits to the approved model's responses on a secret, high-sensitivity probe set, so a provider cannot pre-learn the challenge, cache answers, or tune against a known set.
4. **Binding the audit to Groth16 with measured practicality.** The hidden-probe relation is compiled into an arithmetic circuit and proven with Groth16, with practicality quantified through constraint counts, proving and verification time, proof size, scaling in the number of probes, and end-to-end audit latency.

## Main Findings

- **Token-based probes win on sensitivity despite the weakest access requirement.** The authors report that token-based probes "consistently deliver the strongest mean sensitivity across models and GPU platforms," even though they operate in a black-box setting requiring only the input interface, tokenizer, and vocabulary.
- **Proving time scales gently from 1 to 50 probes.** Proving time increases from 1.02 to 1.78 seconds as the probe set scales from 1 to 50.
- **Verification cost stays flat.** Verification remains near 0.84 seconds across that scaling range.
- **Proof size is constant.** Proof size does not grow with the probe set, consistent with Groth16's three-group-element proofs and a verification check involving three pairings in total.
- **LoRA-only tampering was too subtle for the probe comparison.** In preliminary experiments, a LoRA-only modified model produced only subtle logit drift on non-semantic probes, so it was judged an insufficiently representative test case; the main experiments instead use the original model with LoRA plus Gaussian perturbation as the primary changed model, with Gaussian-only perturbation retained as a control.
- **Stress probes are evaluated against matched perturbations.** MLP-targeted probes are tested against late-MLP perturbations and attention-targeted probes against early-attention perturbations, reflecting the pathway each family is designed to stress.
- **Telemetry verifies execution, not model behavior.** The paper positions GPU telemetry (e.g., kernel-execution validation, FLOPs accounting, utilization inference) as complementary evidence about hardware engagement that cannot establish behavioral consistency with an approved baseline.

## Methodology in Plain English

The authors start from a simple observation: adversarial inputs are usually studied because they reveal where a model is fragile. That same fragility is useful if your question is "has this model been secretly changed?" A tampered model may look normal on everyday prompts, but on inputs that excite sensitive internal directions it will diverge from the approved version.

So they generate large numbers of candidate probes in several families. Token-based probes are non-semantic token sequences — random sequences, rare tokens, delimiters, repeats, and length variants — needing only the tokenizer and vocabulary. Embedding-based probes are sparse "spike" vectors that activate only a few embedding dimensions at once. Stress probes deliberately overstimulate internal pathways: repeated sparse directions aimed at MLP feature transformations, and repeated anchors at controlled spacings aimed at attention routing.

Each candidate probe is run through both the approved model and a modified version, and the instability score is the L2 norm of the difference between their next-token logit vectors, measured at a fixed position (the logits after the last input token). Probes with large scores are the useful ones. Selection happens in two stages: first pick the best-performing family based on mean or median score, separation in the score distribution, top-K sensitivity, and run-to-run consistency; then keep the top-K probes within that family. K is treated as a real trade-off — more probes means a stronger signal but larger witnesses, more constraints, and slower proving.

The modification types are chosen to model stealthy, targeted adversaries rather than random noise. Gaussian-only perturbation serves as an unstructured control; a LoRA-only adapter was found too localized to be a good test case; the main changed model combines LoRA with Gaussian perturbation so the change is structured but broad enough for non-semantic probes to detect.

Finally, the approved model's logits on the selected probe set are hashed into a public commitment. At audit time the verifier regenerates and issues the hidden probes; the prover runs the deployed model and uses Groth16 to prove that the resulting logits reproduce the same commitment. The public statement contains the commitment and circuit parameters; the witness contains the current logits. Neither the probes nor the logits nor the weights appear in the proof transcript. The authors are explicit that the guarantee is narrow: not functional equivalence on all inputs, but that a prover cannot convince the verifier the deployed model matches the committed baseline on the hidden probe set unless that relation actually holds.

## Why This Matters

**Impact on research.** The paper carves out a middle ground between three existing literatures that each solve only part of the problem: probing and interpretability work (LogitLens4LLMs, Tuned Lens, TaskTracker) that analyzes model state but does not build governance probes; adversarial and stress-testing work (GPS, DeP, Adversarial Moral Stress Testing, DeepContext) that amplifies latent failures for robustness or alignment purposes; and backdoor/watermark work (BadEdit, ConfGuard, SEAL, poisoning studies) that targets specific attack manifestations or ownership claims. It also distinguishes itself from zk verification systems like IMMACULATE, zkGPT, and TeleSparse, which require model-specific circuits and greater proving effort. To the authors' knowledge, the combination of adversarial-style synthetic probes with zk-SNARK verification for AI governance has not been studied in this form.

**Real-world applications:**

- **Contractual model auditing.** A model owner and a deployment provider can agree on interface-level guarantees without exposing weights, then run repeated privacy-preserving audits to confirm the served checkpoint still matches the approved one.
- **Supply-chain and repository integrity.** Detecting targeted edits introduced through compromised repositories, adapters, or post-training modifications that leave public benchmarks intact.
- **Safety-relevant drift checks.** Catching narrow safety weakening or selective behavioral bias in one domain that standard evaluation and routine prompts do not surface.
- **Third-party attestation.** Producing a compact, portable proof that a regulator or auditor can verify without access to the model, the probes, or the responses.

**Industry relevance.** The access requirements are deliberately minimal — a tokenizer and vocabulary for the strongest-performing family, no weights or architecture — which maps onto how model owners and GPU providers actually negotiate. The Groth16 cost profile (proving from 1.02 to 1.78 seconds for 1 to 50 probes, verification near 0.84 seconds, constant proof size) suggests repeated audit rounds are operationally feasible rather than a theoretical exercise, which is the practical barrier most cryptographic verification proposals face.

## Future Directions

- **Extending coverage beyond the tested modification classes.** The framework is calibrated against proxy perturbation models; whether the same probes remain sensitive to the full space of real post-deployment attacks (editing, adapter swaps, supply-chain compromise) is an open question the paper's design invites.
- **Choosing K principledly.** The authors note that K trades signal strength against witness size, constraint count, and proving time, but the provided content does not report an optimal operating point or a rule for selecting one.
- **Closing the gap between probe-level evidence and semantic guarantees.** The paper explicitly disclaims full functional equivalence; narrowing the distance between "consistent on the hidden probe set" and broader behavioral or safety specifications remains future work.
- **Combining with telemetry and TEE-based attestation.** The paper frames GPU telemetry, trusted execution environments with remote attestation, and secure multiparty inference as complementary rather than competing, leaving the design of a layered evidence stack open.

## Target Audience

Researchers and practitioners working at the intersection of AI governance, model integrity, and applied cryptography — particularly those building post-deployment audit mechanisms, red-teaming or tamper-detection tooling, or privacy-preserving verification pipelines. It is also useful for policy and compliance audiences who need to understand what a cryptographic attestation of model behavior can and cannot claim, and for ML engineers who want a concrete picture of what access (tokenizer, embedding interface, logits) such an audit actually demands. Readers seeking detailed empirical tables should note that the experimental section is not included in the available content: the specific model architectures, the number of probe candidates generated, dataset sizes, and per-model sensitivity figures beyond the headline token-probe result are not reported here.

## Authors’ abstract

Post-deployment changes to large language models can alter behavior while leaving routine outputs largely unchanged, creating a challenge for AI governance when model weights are proprietary. We present a privacy-preserving zk-SNARK-based audit framework that searches for probes designed in the spirit of adversarial examples to amplify logit drift between an approved model and a modified deployment. Our framework explores complementary probe families under different access…

[The full abstract on arXiv](https://arxiv.org/abs/2608.27954)

[Read the original paper](https://arxiv.org/abs/2608.27954)

---

Canonical: https://crypto.world/paper/not-to-break-but-to-attest-adversarial-probes-for-privacy-preserving-2608.27954
