Prompt optimization for GPT models

Why GPT prompts need optimization

GPT-4o at $2.50/1M input tokens means every 1,000 tokens of unnecessary context costs $0.0025. That sounds small until you scale. At 1M requests per month with 4,000 tokens each, you are spending $10,000/month on input tokens alone. Removing 65% saves $6,500/month.

Beyond cost, optimized prompts also improve GPT output quality. Models are sensitive to noise in the context window. Removing irrelevant information helps GPT focus on what matters.

Optimization strategies for GPT

1. Query-aware compression - SuperCompress scores every line of context against the user query and keeps only the lines relevant to the answer. This is the highest-impact optimization.
2. Remove redundant instructions - If your system prompt already says "Answer based on the context", do not repeat it in every user message.
3. Structure context with XML tags - Wrap context sections in <context> tags. GPT models understand structure and can parse relevant sections more efficiently.
4. Trim examples - Few-shot examples help but each example costs tokens. Keep only the most representative examples.

Before and after comparison

# BEFORE: 4,200 tokens, most irrelevant to the question
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Answer the question."},
        {"role": "user", "content": full_document},
    ]
)

# AFTER: 1,470 tokens, only relevant context preserved
from supercompress import Compressor
comp = Compressor()
result = comp.compress(full_document, question)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Answer the question."},
        {"role": "user", "content": result.compressed_text},
    ]
)

Frequently asked questions

Does optimization work differently for GPT-4 vs GPT-3.5?

No. Compression is model-agnostic. It works the same way for all GPT models.

Will compression change GPT's response style?

No. Only the input context is compressed. The system prompt, instructions, and model behavior are unchanged.