COLD-Steer: Steering Large Language Models via In-Context One-step Learning Dynamics

Researchers introduce COLD-Steer, a training-free framework that enables efficient control of large language model behavior at inference time using just a few examples. The method approximates gradient descent effects without parameter updates, achieving 95% steering effectiveness while using 50 times fewer samples than existing approaches.

• →COLD-Steer enables inference-time control of LLM behavior without requiring model retraining or parameter updates.
• →The framework achieves up to 95% steering effectiveness while using 50 times fewer training samples than baseline methods.
• →Two complementary approaches are used: unit kernel approximation and finite-difference approximation requiring only two forward passes.
• →The method addresses the trade-off between sample efficiency and signal extraction quality in current steering approaches.
• →Applications include pluralistic alignment tasks and accommodating diverse human preferences without extensive demonstration data.