KV Cache Compression for vLLM & SGLang & NIM

Compressed. Attested. Portable.

Compressed

1.444x lossless: the state is preserved exactly and the model’s output does not change. GPU-native, hardware-accelerated.

Attested

Cryptographic credentials bind every saved state to the model that produced it. Signed audit trail.

Portable

Saved state reloads across your NVIDIA estate, any provider, any framework. No vendor lock-in.

Your model. Your hardware. Your benchmarks.

Revyzor KV cache compression roundtrip flow

The math

Lossless by default. No application code changes.

1.444x
Lossless: exact
State preserved exactly. The model’s output does not change.
56.2 ms
Session restore
5,097 tokens from NVMe, against 417 ms to re-read the document.
128K
NIAH context
Every needle found, matches baseline

A higher-ratio mode is available for workloads that do not require exactness. It is opt-in, never the default, and we do not recommend it where the output has to be defensible.

50-question contract comprehension · Side-by-side comparison

100% baseline match. Zero differences.

ConfigAccuracyCompressionBaseline Match
Baseline (uncompressed)95.5%1.0xN/A
Revyzor95.5%1.444x lossless100%

50-question legal contract · Qwen 2.5 7B · vLLM 0.19 · every answer compared side-by-side

Validated on vLLM (A100 80GB) and SGLang (A100 80GB) · Also validated: Qwen 3B/14B · Gemma 12B

Needle-in-a-haystack · 64K context

Every needle found. Identical to baseline.

MethodNIAH Score64K Context
BaselineAll needles1x
RevyzorAll needles3.4x compression

Lossless fidelity

Not “close enough.” Byte-for-byte identical.

0 bytes
Difference between the original model state and the reconstructed state after compression. Checked byte-for-byte at capture, on contexts up to 128K tokens.
1.0000
Cosine similarity between original and reconstructed state. Downstream accuracy is preserved by construction, not by benchmark luck.

One flag. Full roundtrip.

python -m vllm.entrypoints.openai.api_server \
  --model your-model \
  --kv-transfer-config '{
    "kv_connector": "RevyzorConnector",
    "kv_connector_module_path": "revyzor_vllm_connector",
    "kv_role": "kv_both"
  }'
No model changes|No retraining|No code changes|Your NVIDIA GPUs

Revyzor integrates via vLLM’s KVConnectorBase_V1 interface: the same documented extension point NVIDIA’s own KVBM (Dynamo), LMCache, and FlexKV use. It composes with cache tiering rather than competing with it: every tier holds losslessly compressed, signed state, and every offload transfer moves ~30% fewer bytes.

Cache Certification

Every saved state is cryptographically certified.

DID

Model identity. Weights, config, adapters: one cryptographic identifier.

SHA-256

KV cache hash. Cryptographically signed: change one byte and the signature fails verification.

Ed25519

Signed credential. Model, org, timestamp, validity; independently checkable.

Revoke

Compromised model? Revoke its DID. All cached states invalidated instantly.

Regulators ask “show us what this AI was working with.” The credential is the answer. Learn more →

Session restore · Measured

Restoring an evicted session costs 56.2 ms.

Qwen 2.5 7B bf16, a 5,097-token contract, 292.3 MB of raw state, local NVMe on an L40S. Ten iterations, first dropped. Every arm asserted bit-identical before it was timed.

What happensMeanNote
Re-read the document from cold416.6 msCaching off
Evicted session, cache pool flooded441.5 msThe realistic miss
Revyzor restore, compressed56.2 ms33.5 disk + 7.9 H2D + 15.3 decode
Same restore, stored uncompressed60.2 msCompression is a net latency win
Still resident in the engine’s prefix cache39.5 msFaster than us, and free

The last row is on this page deliberately. If your sessions stay resident in GPU memory, the serving engine already handles them and you do not need us. Revyzor is for state that has been evicted, has to survive a restart, or has to be produced again months later.

See it working in real time.

Two live demos. Two platforms. Same lossless compression. Cosine 1.0000.

Early access

Pilot on your workload.

We run Revyzor on your model, your data, your hardware.

You see the compression ratio and accuracy before committing to anything.

Exact lossless compression for AI in regulated environments.

State reloads exactly as captured, across server restarts. Byte-for-byte checked at capture. Cosine similarity 1.0000. Validated across 4 model families, 3 GPU architectures, 2 serving frameworks, 1K to 128K tokens.

Banking·Healthcare·Legal·Compliance

Provider agnostic

Deploy anywhere. Save everywhere.

31%

Of the cache freed on every GPU, bit for bit identical. The same hardware carries a longer context, or more sessions at once.

7.4x

Faster to restore an evicted session from storage than to re-read the document, measured on a 5,097-token contract.

One connector. Every major inference platform.

Cerebrium·InferX·Replicate·RunPod·Regolo·Modal·Together AI·Fireworks AI

Cerebrium and InferX live · Others integration-ready

Get early access.

Join the waitlist for Revyzor. We'll notify you when pilot slots open.

Or email directly: sales@revyzor.com