SuperCompress documentation

Everything you need to reduce LLM token costs with query-aware context compression. Open source (MIT), runs on CPU, ~5K parameters, no GPU required.

What is SuperCompress?

SuperCompress is a query-aware context compiler for LLM inference. It sits between your application and the language model, preprocessing the prompt to remove tokens that aren't relevant to the current user question. This reduces input token counts by ~65% on average, cutting API costs and latency while preserving the evidence the model needs to answer correctly.

Unlike truncation (which blindly drops the tail) or summarization (which rewrites content and risks losing detail), SuperCompress scores every segment of the prompt against the user's actual question and keeps only what matters for that query.

Key facts

Metric	Value
Policy size	~5,200 parameters
Latency	~60ms per compression (CPU)
Token savings (compiler mode)	82.5% avg on bundled presets
Oracle recall	100% — every answer-critical line preserved
Model download	None — weights included in pip package
GPU required	No — runs on CPU before inference
License	MIT — free for commercial use

When to use it

SuperCompress is designed for production AI features where context grows over time and token costs scale linearly with usage:

• Chatbots & conversational AI — long chat histories where each turn appends to the context window
• RAG pipelines — retrieved chunks that overshoot the context budget
• AI search — indexing results, snippets, and document extracts sent with every query
• Support copilots — customer transcripts, ticket history, knowledge base articles
• Agent loops — tool call traces, intermediate outputs, log entries accumulating across turns
• Memory systems — compressed episodic context for persistent agent memory

How it works (briefly)

In compiler mode (the default), SuperCompress:

1. Segments the input context into semantic blocks (headings, code blocks, log entries, prose sections)
2. Scores each block against the user's query using a learned policy
3. Removes duplicate content, noise, boilerplate, and low-value blocks
4. Preserves context dependencies (nearby headings, imports, trace context)
5. Reports tokens saved, important context kept %, and compression risk level

The full pipeline takes ~60ms on a single CPU core — no GPU, no model download, no extra LLM calls.