Prompt Codebooks: Discrete Compositional Optimization for Language Model Instruction Refinement

Researchers introduce Prompt Codebooks (PCO), a new framework for automatic prompt optimization that breaks down instructions into reusable, atomic components rather than treating prompts as fixed strings. The method achieves up to 30% performance gains over baseline approaches while reducing prompt lengths by 14x, enabling more efficient and adaptive language model instruction refinement.

Prompt Codebooks represents a meaningful shift in how researchers approach automatic prompt optimization for large language models. Rather than treating prompts as monolithic text blocks optimized through broad edits, PCO decomposes instructions into discrete, reusable units organized in a learnable codebook. This architectural innovation enables instance-specific prompt routing—different inputs receive tailored instruction compositions rather than identical prompts, addressing a fundamental limitation of existing global optimization methods.

The framework's technical approach combines an LLM-based encoder that routes inputs to relevant codebook entries, a generator that composes selected instincts into complete prompts, and a critic that provides structured feedback decomposed into per-variable gradients. This design enables joint optimization of routing, composition, and instruction semantics under a unified objective. The method emerged from recognition that prompt optimization methods produce brittle updates and prevent knowledge reuse across tasks.

The empirical results demonstrate substantial improvements. PCO surpasses prior state-of-the-art methods like GEPA by 3.34 points on HotpotQA and 1.11 points in aggregate performance, while using only 16 discrete instincts. More significantly, it achieves 14.1x prompt compression versus MIPROv2 and 3.0x versus GEPA, directly addressing deployment efficiency concerns. Testing on Qwen3-8B and LLaMA-3.1-8B models validates generalizability across architectures.

For the AI research community, this work has implications for model efficiency and interpretability. Shorter, compositionally-structured prompts may be easier to understand, audit, and transfer across domains. As organizations deploy increasingly sophisticated agentic systems, methods that reduce prompt overhead while improving performance could significantly impact real-world LLM applications and inference costs.

• →PCO recasts prompt optimization as discrete learning over finite vocabularies of reusable instruction units rather than monolithic text optimization.
• →Instance-specific routing enables different inputs to receive tailored instruction compositions, structurally improving upon instance-blind baseline methods.
• →Performance gains reach +30.36 points over zero-shot and +3.34 over prior best method GEPA on HotpotQA benchmarks.
• →Prompt length reduction of up to 14.1x versus MIPROv2 demonstrates significant deployment efficiency improvements.
• →The framework's decomposition enables structured feedback and per-variable gradients, improving interpretability of optimization objectives.