Is a Third-Party API Call Really Using the Model You Chose? Anthropic FlashProve Verifies Inference Results with a 19MB Cryptographic Proof

Author’s note: When you call a Large Language Model through a third-party API, answer quality alone can’t tell you which model is actually running. This post unpacks Anthropic’s open-source FlashProve: how it uses zkSNARKs to bind inference results to specific weights, what that costs, what it can and can’t prove, and how far it still sits from everyday API calls.

On November 2, 2026, Anthropic’s Frontier Red Team, together with researchers from EPFL, MIT and elsewhere, released FlashProve: an open-source prototype for provable inference in Large Language Models. On a 400B-parameter Llama 4 Maverick, it generates a proof on a single H100; each proof runs about 19 MB, and an ordinary CPU verifies one in a dozen-odd seconds.

This isn’t a concept demo or a slideware product. Anthropic shipped it alongside a technical report, a protocol specification, runnable prover code plus two separate verifier implementations, and Lean formal proofs — all released under Apache 2.0. That said, the team is explicit that this is an interim research result, with plenty of ground still to cover before the full research goal is reached.

The payoff: by the end of this post you’ll understand why “model identity” is so hard to pin down in third-party API calls, what mechanism FlashProve uses to solve it, where its real costs and limits lie, and what you can do about it today as an API user.

A note on the above: I have no record of a FlashProve release, and the date the article gives falls after my knowledge cutoff, so I can’t verify any of the claims or figures in it — including what’s attributed to Anthropic. If you’re publishing this, it’s worth checking against a primary source before it goes out.

Why Third-Party API Calls Have a Model Identity Trust Problem

When you send a request through any API, all you get back is a chunk of text and a model name field. Anthropic has said plainly on its research blog that users today can’t confirm a response actually came from the model the provider claims. A routing service might quietly swap it out to cut costs, or an attacker might tamper with the token stream.

More importantly, high-quality output doesn’t prove the model is who it says it is. A cheaper model can answer some questions very well, and a downgraded version often can’t be spotted by eye. Even a fully published chat log could, in theory, be faked with a cheaper model, so this kind of evidence doesn’t prove much.

Verifying this at a fundamental level is hard, too. The weights of frontier models are mostly kept private, and even if you had them, re-running inference to check the output would take far more compute than ordinary users, or even evaluation organizations, can afford. Anthropic outlines several typical scenarios where this trust problem shows up:

• Third-party routing services: Customers want to be sure the tokens they receive come from the model they chose and paid for.
• Independent evaluation: Evaluators need to confirm that the model they tested before release is the one actually serving production traffic.
• Tamper detection: If an inference server is compromised, an attacker might slip malicious answers into responses, or even deliver malware to the user’s computer.
• Regulation and compliance: Regulators may require that only safety-approved models are deployed, or want confirmation that the hardware is really running inference and not secretly training.

The goal of Provable Inference is to let a user with limited compute, and no knowledge of the weights, verify that a given response really came from the requested model.

FlashProve at a Glance

Worth noting: Anthropic says a large share of the optimization work in the system was carried out by Claude. To guard against AI making “hard-to-evaluate decisions” inside a large, complex system, the team wrote auditable top-level safety claims in Lean as a hedge.

I can’t verify any of the specifics here — this falls well outside what I know, so treat the dates, collaborators, and results as the source document’s claims rather than something I’ve confirmed.

How FlashProve Proves Inference Came from a Specified Model

FlashProve has just two roles: the model provider acts as the Prover, and the user acts as the Verifier. The whole flow breaks down into an offline phase and an online phase, followed by checks on the verification side.

The Three-Step Mechanism Behind FlashProve

1. Offline model commitment: The prover publishes a commitment to the model weights — think of it as a very short “fingerprint.” It pins down one exact set of weights while the weights themselves stay hidden. In testing, this step runs once per model and takes about 39 minutes, and the resulting commitment covers roughly 16 million requests before it needs to be regenerated.
2. Online proof generation: After receiving a verifier’s request (a prompt and its corresponding output tokens), the prover produces a proof file. The claim being proven is that “the committed model, when run, would produce exactly these output tokens.”
3. Verification-side check: Anyone holding the commitment, the prompt and the answer can check the proof — no weights needed, and only limited compute — and be confident that the committed weights produce exactly those tokens on this prompt.

FlashProve’s Four Security Properties

The first two properties (completeness and soundness) are things a verifier who knows the weights and has spare compute could achieve by simply recomputing everything themselves. What makes the system actually worthwhile are the last two:

• Completeness: A proof from an honest prover is always accepted.
• Soundness: Forging a valid proof for a false claim is as hard as breaking a widely trusted cryptographic assumption, so verifiers don’t have to trust the prover.
• Zero Knowledge: Verifiers learn nothing about the weights or intermediate computation values, and can’t tell which experts were activated at each layer — though the model architecture itself is not hidden.
• Succinctness: What’s being proven is a single run of a ~232 GB quantized model, yet the verifier receives only a ~19 MB file and can check it in seconds on a 24-core CPU.

How the MoE Architecture Cuts FlashProve’s Proving Overhead

Much of FlashProve’s speedup comes from exploiting the structure of MoE (Mixture of Experts) models. In Llama 4 Maverick, each token uses only about 17B of the 400B parameters, so the prover’s work scales only with those ~17B — and the proof still hides which experts were actually selected.

Two other engineering choices matter just as much. The first is post-hoc batched proving: the prover reads the completed text in parallel the way prefill does, opening the weight commitment only once per batch. The second is single-GPU operation: weights are split into per-layer segments, proven segment by segment, and streamed from ordinary memory — so one 80 GB GPU plus about 290 GB of host memory is enough to run it.

FlashProve Proving Cost Benchmarks

Cost is the key metric for deciding whether provable inference can actually be deployed. Anthropic’s test setup was a single H100 plus 24 CPU cores on the proving side, a CPU-only verifier, and each request was a 1024-token window.

The baseline is vLLM running the FP8 version of the same model on 8 H100s. In other words, the GPU time needed to prove a batch of requests is roughly 70-95 times what it takes to process the same batch with normal inference. That’s about two orders of magnitude of extra overhead.

110 tokens/s Doesn’t Mean Real-Time Proving of a Full Response

This is the part of FlashProve that’s easiest to misread. The technical report is explicit: each proof only actually checks one token after the prompt. The 110 tokens/s figure is the prover’s throughput for processing request windows, not the speed of verifying a full response token by token.

So the claim that “FlashProve can already prove a complete response in real time” isn’t accurate. A more careful way to put it: under batched, short-context, after-the-fact proving conditions, it brings the cost of provable inference down to a level that would have been hard to imagine before.

How FlashProve Compares to Existing Provable Inference Approaches

Provable inference is usually called zkML in academia. Anthropic’s blog reviews the progress so far, and the numbers speak for themselves.

Comparing with VerInf: VerInf is the only prior approach known to have proven the same model, but its proofs are a whopping 93.6 GB and verification takes almost a full day, which makes it basically unusable in practice. FlashProve, by contrast, shrinks the proof to about 19 MB and brings verification down to a dozen or so seconds, making third-party audits on an ordinary CPU feasible for the first time.

Comparing with Attestable: Attestable posts impressive commercial numbers on dense models, but as of this writing there’s no corresponding open-source library. FlashProve’s advantage is that the code, Lean proofs, and documentation are all fully open, so anyone can reproduce and improve on them. The trade-off is that it currently depends heavily on the MoE architecture.

What FlashProve Can and Can’t Prove

For third-party API users, understanding the boundaries matters just as much as understanding the capabilities. The table below summarizes the key caveats from the research blog and technical report:

The Detection Limits of FlashProve’s Post-hoc Proving

FlashProve uses post-hoc proving: the user receives the response first, then an independent proving server recomputes it and generates a proof, and finally a verifier checks that proof. This approach lets the proving step be slower, so you don’t have to compromise between security and efficiency. The trade-off is that the guarantee arrives with a delay. A failed proof can only reveal a problem; it can’t protect the user before they receive the answer.

The technical report also warns that if the logging system itself is controlled by an attacker, the attacker could send tampered content to the user while making the logs record only the correct result. That’s why FlashProve has to be used together with trusted logging, access control, and other security measures.

What FlashProve’s Formal Verification Doesn’t Cover Yet

“Formally proven in Lean” is easy to read as “the whole system has been proven correct,” but that isn’t the case. The boundaries listed in the technical report include:

• Lean’s proof of the zero-knowledge property currently covers only single-request proofs, not batched proofs.
• The GPU prover code was generated by AI from the Lean representation, and its equivalence to that representation hasn’t been proven.
• The C++ fast verifier used day to day was also generated by AI. It has no formal guarantee; it has only matched the Lean verifier’s conclusions in every test.
• The longer protocol specification document was generated by AI, and the authors haven’t reviewed it in detail, so it should be treated separately from the technical report they wrote themselves.

These limitations don’t invalidate the research. They show that formal verification currently locks down the mathematical properties of the protocol layer, while the gap between executable code and the protocol still has to be bridged by manual or agent-based auditing.

Impact Analysis: How Far Is FlashProve from Everyday Third-Party API Calls?

What It Means for Developers and API Users

• It won’t show up in your everyday call chain anytime soon: The GPU overhead of roughly two orders of magnitude, the ~19 MB proof file per request, and the 1,024-token context limit all make it impractical for high-frequency conversational APIs.
• Its applicability is limited: The system assumes an MoE architecture, an integer-quantized model, and greedy decoding. Most production calls use stochastic sampling and floating-point inference, so it can’t be applied directly.
• The bigger value is conceptual: For the first time, “verifying model identity” has gone from impossible to an engineering problem. Developers should recognize that a returned model name field, a model ID, or ordinary request logs are not the same thing as a cryptographic proof.

💡 Practical advice: Until provable inference matures, when choosing a third-party service it’s worth paying more attention to billing transparency, call-log traceability, and self-run consistency checks. When you call models through an aggregation platform like APIYI (apiyi.com), you can query multiple models at once and keep complete call records, which makes it easier to run your own cross-comparisons and audits.

What It Means for the Industry and Regulation

• The evaluation and compliance toolbox gains another option: Regulators could require providers to attach proofs to inference requests over a given period, confirming that the hardware is really serving the specified model, without learning the proprietary weights. Anthropic also acknowledges, however, that this can’t prove the hardware isn’t doing training on the side.
• Software approaches and trusted hardware complement each other: Trusted execution environments rely on the vendor’s tamper-resistant hardware, while zkSNARKs rely on the hardness of computational problems, so proofs can’t be forged even if the hardware and operating system are compromised. Each will play a role in different scenarios.
• Anthropic’s production verification takes a different route: In the Frontier Safety Roadmap updated on October 19, 2026, Anthropic mentions that its inference verification program has built a prototype on production inference traffic. It signs internal token traffic and recomputes inference on a trusted server, and it has already caught one extremely rare, non-malicious integrity anomaly. The program has been renamed inference verification. It’s a related but separate project from FlashProve, and the two shouldn’t be conflated.

What Third-Party API Users Can Do Right Now: Build Your Own Call Audit Log

Cryptographic proofs aren’t practical yet, so the best thing developers can do for now is make their calls traceable. The example below fixes temperature=0 on each call and writes the request time, request ID, the returned model field, and a hash of the answer to a local log, so you can run consistency re-tests later. One thing to stress: this is only an engineering-level audit record, not a cryptographic proof of model identity.

import openai

client = openai.OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.apiyi.com/v1"  # APIYI OpenAI-compatible endpoint
)

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "用一句话解释零知识证明"}],
    temperature=0
)
print(resp.model, resp.id)
print(resp.choices[0].message.content)

import hashlib
import json
import time
import openai

client = openai.OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.apiyi.com/v1"  # One key to call models from multiple providers
)

def audited_call(prompt: str, model: str, log_path: str = "audit.jsonl") -> str:
    """Call the model and record traceability info (not a cryptographic proof)"""
    start = time.time()
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        temperature=0,
        max_tokens=512,
    )
    content = resp.choices[0].message.content or ""
    record = {
        "ts": int(start),
        "latency_s": round(time.time() - start, 3),
        "requested_model": model,
        "returned_model": resp.model,   # Just a claimed field returned by the server
        "request_id": resp.id,
        "prompt_sha256": hashlib.sha256(prompt.encode()).hexdigest(),
        "output_sha256": hashlib.sha256(content.encode()).hexdigest(),
        "usage": resp.usage.model_dump() if resp.usage else None,
    }
    with open(log_path, "a", encoding="utf-8") as f:
        f.write(json.dumps(record, ensure_ascii=False) + "\n")
    return content

# Fix a set of probe prompts, re-test periodically, and compare output hashes and latency distributions
probes = ["计算 37*43 并只输出结果", "列出三种排序算法的时间复杂度"]
for p in probes:
    for m in ["gpt-4o-mini", "claude-sonnet-4-5"]:
        print(m, audited_call(p, m)[:60])

🚀 Quick start: You can get free test credits at APIYI (apiyi.com) and use the probe script above to re-test multiple models on a regular schedule. To be clear, this approach only improves traceability. It can’t replace weight-level proofs like FlashProve.

A few other practical habits are worth adding on top of this: pin exact model IDs instead of using vague aliases; keep a set of probe prompts and track output and latency changes over time; for critical workloads, query several models and cross-check the results; and keep an eye on whether providers publish weight commitments or verifiable mechanisms.

Frequently Asked Questions

FlashProve Key Takeaways

• The trust gap is real: When you call a third-party API, high-quality answers don’t prove the model behind them is the one you’re paying for.
• Core mechanism: An offline weight commitment plus an online zkSNARK proof, which users can verify on a plain CPU.
• Costs drop sharply: A single H100 window reaches about 110 tokens/s throughput, with proofs of roughly 19 MB and verification in about 17 seconds.
• Keep the numbers straight: Token-by-token checking runs at 39.1 tokens/s, while a single request manages only 17.6 tokens/s.
• Clear limits: It only supports MoE models, integer-quantized greedy decoding, a 1024-token context, and after-the-fact detection.
• How to get started: For now, you can use APIYI (apiyi.com) and its unified multi-model interface to run your own retests and audits.

Summary

FlashProve’s significance boils down to three points:

1. It turns model identity verification into an engineering problem: Proving that the same model was used used to take more than ten hours and close to 100 GB of proof data. Now it’s down to batch throughput in the hundreds of tokens/s and a 19 MB proof.
2. It’s still a long way from production use: The two-orders-of-magnitude overhead, the short context, the architecture and decoding restrictions, and incomplete formal verification coverage all mean it’s still a research prototype.
3. It has to work alongside other security mechanisms: A post-hoc proof can detect cheating but can’t block it, so you’ll need to pair it with trusted logging and access control.

If you use third-party APIs today, the most practical stance is this: don’t treat the model name field as proof, but don’t panic about it either. Get your call logs, probe retests, and multi-model cross-checks in solid shape first, and keep an eye on how provable inference matures. With APIYI (apiyi.com), you can compare multiple providers’ models using a single API key and build a traceable quality baseline on your own business data.

📚 References

1. Anthropic Research Blog – Progress on provable inference for large language models: The original FlashProve announcement
2. Link: anthropic.com/research/provable-inference

3. Description: Covers the system design, measured results, comparisons with existing approaches, and known limitations

4. FlashProve Technical Report: Full technical details, linked from the research blog

5. Link: anthropic.com/research/provable-inference (see the “Read the technical report” link on the page)

6. Description: Covers key caveats such as how throughput was measured, the limits of post-hoc proofs, and the boundaries of formal verification

7. Anthropic Frontier Safety Roadmap: Progress updates on the inference verification project

8. Link: anthropic.com
9. Description: Explains how inference verification differs from FlashProve

Author: APIYI Technical Team
Get in touch: Reach out to us through the APIYI website homepage. For more integration resources, visit the docs center at docs.apiyi.com.

Related Articles

• A Model Becoming “Dumber” Doesn’t Necessarily Mean It Was Swapped: 6 Common Causes of Unexpected API Performance and a 3-Step Troubleshooting Sequence: Anthropic released FlashProve research on inference proofs, but even after model weights are verified, degraded performance can result from system prompts, context truncation, or decoding parameters; this article outlines common causes and an actionable troubleshooting sequence.