Integration guide
Compress Firecrawl results before they hit your LLM.
Firecrawl is useful because it turns pages into model-ready content. The problem is that model-ready content can still be huge. SuperCompress runs between Firecrawl and the model, keeping query-relevant evidence while removing scraped noise.
Where it fits
| Step | Without compression | With SuperCompress |
|---|---|---|
| Scrape | Firecrawl returns markdown for full pages or crawls. | Same Firecrawl result. |
| Prompt assembly | Entire scrape dump enters the prompt. | Context is compressed around the current query. |
| Model call | LLM pays attention to boilerplate, nav, repeated sections, and irrelevant page chunks. | LLM sees less input, with the answer-critical evidence prioritized. |
Hosted API example
curl -X POST https://supercompress.dev/compress \
-H "X-API-Key: sc_live_..." \
-H "Content-Type: application/json" \
-d '{
"query": "What pricing limits matter for a startup using this tool?",
"context": "PASTE_FIRECRAWL_MARKDOWN_HERE"
}'Python pipeline sketch
from firecrawl import FirecrawlApp
from supercompress import Compressor
firecrawl = FirecrawlApp(api_key="fc-...")
page = firecrawl.scrape_url("https://example.com/pricing", formats=["markdown"])
query = "What pricing limits matter for a startup using this tool?"
compressed = Compressor().compress(page["markdown"], query)
# Send compressed.compressed_text to OpenAI, Claude, Gemini, or your agent.
print(compressed.stats)Best first use cases
- AI search over scraped documentation
- Competitive research agents that crawl multiple pricing pages
- Sales/research workflows that scrape company websites
- Coding agents that ingest docs before writing integration code
FAQ
Why compress Firecrawl output?
Scraped pages often include navigation, repeated boilerplate, unrelated sections, and long markdown blocks. Compressing around the user query reduces token waste before the LLM call.
Does this replace Firecrawl?
No. Firecrawl extracts page content. SuperCompress runs after extraction and before inference to reduce the context sent to the model.
Try it today
Compress your first scrape result.
Get a free hosted API key, paste a Firecrawl markdown result into the demo, and measure what drops before the model call.