Revyzor Technical Whitepaper · August 2026
Revyzor is not an application, not a model, and not an API wrapper. It is a plug-in for the inference engines that run open-weights models in production. This paper explains why that is the only place governed retention can be built, and why we chose the engines' own documented extension points rather than a fork or a proxy.
When a large language model reads a document, it builds an internal working state: the accumulated representation of everything it has read so far. In every modern serving engine this state is held in GPU memory for exactly as long as the request is being served, and then discarded.
Everything that matters for retention happens in that state. It is the difference between a model that has read a 300-page agreement and one that is about to read it again. It is what a follow-up question resumes from. And it is the only faithful record of what the model actually considered when it produced an answer.
Crucially, nothing above the serving layer can touch it. Applications built on an API, whether a frontier-model API in a US cloud or a self-hosted endpoint, see tokens in and tokens out. The working state never crosses the API boundary. A legal AI application, an agent framework, or an orchestration layer can log prompts and responses, but it cannot retain, sign, or reload the state the model reasoned in. Only software running inside the engine can.
If the product is retention of the model's working state, the product must live where the working state lives: inside the serving engine, on the GPU.
The good news is that the engines anticipated this. vLLM, the de facto standard open-source serving engine, exposes a documented connector interface (KVConnectorBase_V1) precisely so that external systems can participate in the lifecycle of the working state: observe it as it is built, store it, and supply it back to the engine.
This is not an obscure hook. It is the same interface used by NVIDIA's own offload stack (KVBM, part of Dynamo), by LMCache, and by FlexKV. It is how the ecosystem has decided this class of software should attach. Revyzor implements that interface as a standard connector: one configuration entry, no engine patches, no custom build.
| Engine | How Revyzor attaches | Status |
|---|---|---|
| vLLM | Standard connector via the documented KV connector interface; enabled by configuration | Integrated and benchmarked |
| SGLang | Integration at the equivalent cache-management layer | Integrated and benchmarked |
| NVIDIA NIM | Runs inside NIM's vLLM-based serving path | Integrated and validated |
The consequence for a buyer: adopting Revyzor does not mean adopting a new serving stack. It means adding a plug-in to the stack your platform team has already standardised on.
There were two other ways to build this, and both fail in predictable ways.
Forking the engine gives total freedom for one release cycle. But vLLM ships breaking internal changes constantly; the internal layout of the working state itself has changed shape between minor versions. A fork diverges immediately and dies quietly: every upstream security fix, performance improvement, and new model architecture has to be re-ported by hand. Enterprises are rightly unwilling to run a stale fork of their inference engine in production.
Building against the documented extension point instead means Revyzor upgrades with the engine. Tracking those internal changes release after release, and shipping a connector that keeps working through them, is a core part of what customers pay us for.
The other route is a hosted service: send us your documents, we return retained sessions. That fails the customers we built this for. In regulated deployments the entire point is that client documents, and the model's working state derived from them, never leave the customer's infrastructure. A retention product that requires shipping the most sensitive artefact in the system to a third party is a contradiction. Revyzor therefore ships as software the customer runs, in their jurisdiction, under their keys.
Even if the architecture allowed retention from outside the engine, the physics would not. The working state for a long document on a production model is tens of gigabytes of GPU memory. Moving that off the GPU to be processed elsewhere would stall serving and saturate interconnects.
Revyzor's compression and signing therefore run on the GPU, hardware-accelerated, inside the serving process. The state is compressed where it lives, in milliseconds per layer, and only the compact result ever moves. The same applies in reverse: a retained session is reloaded rather than re-read, at roughly 7x less GPU time than re-reading the document.
| Measurement | Result | Context |
|---|---|---|
| Lossless compression ratio | 1.45× | Full-precision (bf16) working state, 31B-parameter model, per-layer checks on every layer |
| Answers under compression | 500 of 500 unchanged | Controlled study, five legal benchmark tasks; every answer generated from the compressed-and-reconstructed state matched the uncompressed run |
| Session reload | 56.2 ms | Separate experiment: 5,097-token session on an L40S with a 7B model, restored from local NVMe (33.5 ms disk, 7.9 host-to-device, 15.1 decompress) against 417 ms to re-read the document. Excludes serving-engine scheduler overhead. A session still resident in the engine's own prefix cache is restored in 39.5 ms and costs nothing. |
| Per-layer round trip | 11–22 ms | Compress and reconstruct in GPU memory, inline with serving |
Compression, answer-parity and per-layer figures are from our August 2026 controlled study on LegalBench tasks with a 31B open-weights model at full precision on a single H200 GPU. The session reload figure is from a separate restore experiment on an L40S with a 7B model and is not comparable to the rows above. Full methodology available under NDA.
The serving ecosystem is excellent at what it was built for. vLLM and SGLang are throughput engines: they exist to serve as many tokens per second as possible. The offload stacks around them (KVBM, LMCache, FlexKV) are placement systems: they decide where working state sits across GPU, CPU memory, and NVMe to improve cache hit rates.
None of them do three things regulated deployments need:
This is why Revyzor composes with the offload stacks rather than competing with them. A placement system decides where the bytes go; Revyzor makes those bytes smaller (roughly 30% fewer bytes on every offload transfer), signed, and governed. The value multiplies at every tier of the memory hierarchy the placement system manages.
The serving layer is where the working state lives, where the performance budget allows it to be handled, and where the ecosystem has agreed extensions belong. That is why we built there.