Search optimization guide
Token compression for AI search
LLM-enhanced search sends product data, filters, and user context with every query. Compression keeps only the search-relevant product attributes, cutting costs and improving relevance.
Compressed search pipeline
from supercompress import Compressor
comp = Compressor()
def search_products(query, product_catalog):
# Compress the product catalog against the search query
result = comp.compress(product_catalog, query)
# Only products matching the query intent remain
return llm.generate(
f"Search results for '{query}':\n{result.compressed_text}"
)Frequently asked questions
Does compression slow down search?
Compression adds ~60ms, but the LLM generates faster with less context. Net impact is neutral or faster.
Can I use it with Elasticsearch or Algolia?
Yes. Use SuperCompress as a re-ranking step after your search results are returned.
Try it yourself
Paste your long prompt into the playground, ask a question, and see what SuperCompress keeps and removes. Free, no signup needed.