Chain of Thought

Definition

Chain of Thought (CoT) is a prompting technique that instructs an LLM to produce explicit step-by-step reasoning before arriving at a final answer. Rather than jumping directly to a conclusion, the model is guided to articulate its reasoning process—breaking down a complex problem into intermediate steps, evaluating each step, and building toward an answer. This technique significantly improves accuracy on multi-step reasoning tasks, mathematical problems, and complex decision-making scenarios where direct answer generation produces errors.

Engineering Context

Chain of thought improves performance on multi-step reasoning tasks significantly, but at the cost of more output tokens and therefore higher latency and cost. In production, use CoT selectively: for high-stakes decisions where reasoning transparency matters and can be logged, but not for simple extraction tasks where it adds waste. Structured CoT—using explicit XML tags or JSON fields for each reasoning step—improves reliability over free-form CoT, as it forces the model into a consistent format and makes reasoning steps parseable. Zero-shot CoT (using "Let's think step by step") works for many tasks, but few-shot CoT with domain-specific examples yields better results for specialized applications.

Related Terms