CacheAligner: KV Cache Prefix Stabilization for LLM Providers

Every call to an LLM provider (OpenAI, Anthropic, vLLM) reprocesses the prompt prefix — the system message, instructions, and context header — to compute its KV (key-value) cache. If the same prefix appears in multiple requests, the provider can reuse the cached KV state instead of recomputing it. This is called prefix caching or KV cache reuse.

The catch: the prefix must be byte-identical across requests. Any variation — a different timestamp, a reordered section, even an extra space — invalidates the cache. CacheAligner solves this for compressed context by wrapping every compressed output in a deterministic, byte-identical XML preamble.

The Problem: Compressed Output Is Not Cache-Friendly

When you compress context with SuperCompress, the output varies per request — different inputs produce different kept lines. This is the whole point of query-aware compression. But it means the upstream provider can't cache the KV state for the compressed content because every request looks different.

Even with a shared system prompt and fixed instructions, the compressed context block changes on every call, so the KV cache hit only covers the system prompt — typically a few hundred tokens. The bulk of the prompt (the compressed context) is never cached.

How CacheAligner Works

CacheAligner wraps the compressed text in a stable <supercompress> XML envelope:

<supercompress version="1" type="compressed_context">
The following text has been optimized by SuperCompress. It preserves
the information most relevant to answering the user's question while
removing low-value tokens. Use this context to answer the user.

--- compressed context ---

[compressed text goes here — this is the only part that varies]

--- end compressed context ---
</supercompress>

Answer the question: ${query}

The preamble (<supercompress> through --- compressed context ---) is ~250 characters (~60 tokens) of byte-identical, cacheable prefix. Every request starts with the exact same tokens. Only the variable compressed content and query change.

How providers use the cached prefix

Using CacheAligner in the API

CacheAligner is opt-in via the cache_prefix parameter:

curl -X POST https://supercompress.dev/api/v1/compress \
  -H "X-API-Key: sc_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "long text to compress...",
    "query": "What matters?",
    "cache_prefix": true
  }'

The response includes cache_prefix_applied: true and the compressed_text is wrapped in the deterministic preamble.

Response

{
  "compressed_text": "<supercompress version=\"1\"...>\n...compressed output...\nAnswer the question: What matters?",
  "cache_prefix_applied": true,
  "original_tokens": 2048,
  "kept_tokens": 320,
  "kv_savings_pct": 84.4,
  ...
}

CacheAligner + System Prompts

For maximum cache hit rates, combine CacheAligner with a stable system prompt:

System: You are a helpful assistant specialized in analyzing technical context.

CacheAligner-wrapped compressed context

Question: What is the root cause?

The system prompt + CacheAligner preamble form a longer stable prefix. With OpenAI's 1,024-token minimum for automatic caching, a 500-token system prompt + 60-token CacheAligner preamble = 560 stable tokens. Add 464+ tokens of compressed content that happens to be identical across similar queries (e.g., repeated headers or shared boilerplate), and the entire prefix is cached.

Browser-Side CacheAligner

CacheAligner is also available in the browser engine:

const E = window.SuperCompressEngine;
const result = E.compressAdaptive(context, query, model);
const { wrapped, preambleTokens } = E.cacheWrap(result.compressed_text, query);

console.log(preambleTokens);  // ~60 tokens of cacheable prefix
console.log(wrapped);         // Fully wrapped output, ready to send to any LLM

Provider-Specific Best Practices

OpenAI

OpenAI's automatic prefix caching caches any prefix ≥1,024 tokens. To maximize hits: place a long, stable system prompt before the CacheAligner wrapper. Use the prompt_cache_key parameter to route similar requests to the same cache pool.

Anthropic (Claude)

Anthropic requires explicit cache_control markers on content blocks. Place your system message and the CacheAligner preamble in a content block with "cache_control": {"type": "ephemeral"}. This ensures the cached block survives across requests within the 5-minute cache window.

vLLM (Self-Hosted)

vLLM's Automatic Prefix Caching (APC) caches KV blocks at the PagedAttention block level. CacheAligner's byte-identical preamble creates exact block matches, so vLLM can reuse cached blocks without any special configuration. Enable APC with --enable-prefix-caching when starting the vLLM server.

Why "CacheAligner"?

The name comes from the core insight: provider-side KV cache alignment depends on the exact byte sequence of the prompt prefix. CacheAligner aligns the compressed output to a stable template so the provider's caching infrastructure can recognize and reuse it — no guessing, no heuristics, just a deterministic wrapper that every provider treats the same way.

Combined with CCR

CacheAligner works alongside CCR (Cache, Compress, Retrieve). Enable both for reversible compression with provider-side caching:

curl -X POST https://supercompress.dev/api/v1/compress \
  -H "X-API-Key: sc_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "long text...",
    "query": "What matters?",
    "ccr": true,
    "cache_prefix": true
  }'

The compressed output will have interspersed [SC-Retrieve: hash] markers for reversible retrieval, wrapped in the CacheAligner preamble for KV cache hits.

What's Next

CacheAligner is available now in the SuperCompress API (v0.7+). Set cache_prefix=true on any compression request to start caching. Future versions will add:

• Automatic preamble length scaling for provider-specific cache thresholds
• Provider-aware middleware that emits proper cache_control markers upstream
• Cache hit rate reporting through the API response