Architecture

How SuperCompress works under the hood — the learned policy, feature extraction, training approach, and compiler-mode pipeline that runs in ~60ms on CPU.

The compression pipeline

Long context + user question
        ↓
  Tokenizer + feature extraction (CPU)
        ↓
  Semantic block segmentation
        ↓
  Block scoring (learned policy)
        ↓
  Duplicate/noise removal
        ↓
  Dependency-preserving packer
        ↓
  Verifier (estimates important context kept)
        ↓
  Compressed evidence prompt → LLM

The entire pipeline from input to compressed output runs in ~60ms on a single CPU core. No GPU, no model download, no extra LLM calls. The policy is small enough to fit in L1 cache.

The learned policy (~5,200 parameters)

The core of SuperCompress is a tiny MLP (multi-layer perceptron) called EvictionPolicyNetwork:

Property	Value
Parameters	~5,200
Architecture	MLP with 1 hidden layer (64 units)
Input features	12-dimensional per-token feature vector
Activation	ReLU (hidden), Sigmoid (output)
Output	Eviction score per token — lower score = more likely to be kept
Inference	Pure NumPy / ONNX / browser JS — no PyTorch required at inference
Checkpoint size	~25 KB (included in pip package)

This is 25,000x smaller than alternative approaches like Headroom's ModernBERT (165M params). The small size means zero model download time, zero warmup, and instant cold starts.

Feature extraction

Each token is represented by a 12-dimensional feature vector:

• Position encoding — normalized position within the document (recent tokens are more likely to be kept)
• N-gram similarity — token-level overlap with the query (how relevant is this token to the question?)
• Line length — normalized line length (very short or very long lines get different treatment)
• Indent depth — code/list nesting level (deeper nesting = more structured content)
• Block boundary — whether this token starts a new section or heading
• Token type — code keyword, natural language, punctuation, number, etc.
• Entropy — character-level entropy (boilerplate tends to have lower entropy)

Features are computed on the fly in Python/NumPy. No tokenizer or model download needed — feature extraction uses lightweight string operations and precomputed tables.

Training

The policy was trained on thousands of question-answering examples spanning code, documentation, logs, prose, and structured data. For each example:

1. Start with the full context and question
2. Run the LLM on the full context to get a reference answer
3. Remove one line at a time and re-run the LLM
4. If the answer changes significantly, mark that line as "important"
5. If the answer stays the same, mark that line as "removable"
6. Train the policy to predict importance from the 12-dimensional features

The training objective is a pairwise ranking loss: the policy should assign lower eviction scores (higher importance) to lines that affect the answer. Training takes ~60 seconds on a laptop CPU.

supercompress-train --fast
# Outputs checkpoints/default.pt (~25 KB)

Compiler mode

Compiler mode is the production path (and the default on the hosted API). Unlike the fixed-ratio mode, it does not ask users for a budget. Instead it:

1. Segments context into semantic blocks — headings, code definitions, log entries, chat turns, prose sections, tables, config blocks
2. Downranks repeated tool output, boilerplate, generated noise, duplicate blocks, and irrelevant context
3. Preserves nearby headings, imports, trace/log context, balanced markdown fences, and content that depends on context
4. Verifies the result — estimating important-kept percentage and assigning a risk level (low / medium / high)
5. Reports what was kept and dropped, with reasons per block

The verifier makes compiler mode safe for production. If the verifier detects high risk (too much important context may have been removed), it signals the user to adjust their approach.

Fixed-ratio mode

The legacy fixed-ratio mode uses a budget (e.g., keep 35% of tokens) and evicts the lowest-scored tokens up to the budget limit. This mode exists for research comparisons and backward compatibility. In production, use compiler mode.

	Compiler mode	Fixed-ratio mode
Budget required?	No — maximizes removal safely	Yes — pass budget_ratio
Savings	82.5% avg on bundled presets	Fixed at budget_ratio (e.g., 65%)
Risk reporting	Yes — risk level + important-kept %	No — just keeps X% of tokens
Block breakdown	Yes — kept/dropped blocks with reasons	No
Best for	Production API calls	Research benchmarks

Browser port

The same policy runs in the browser via compress-engine.js. After exporting the model weights to JSON:

python scripts/export_model_json.py
# → Creates web/assets/data/model.json

The browser engine uses the identical NumPy-derived numerical computation as the Python version. The feature extraction, scoring, and block segmentation are reimplemented in JavaScript with the same logic, producing the same results.

The browser demo at supercompress.dev/playground runs this engine. No API key needed — compression happens entirely in your browser.