LLM reasoning strategies are prompting methods that control how a language model goes from the question to the answer. They change the prompt and the way that the outputs are combined. The model weights stay the same.
A thought is 1 intermediate reasoning step that the LLM writes. Each strategy arranges its thoughts in a different shape.
Comparison
| Strategy | Shape of thoughts | Key novelty |
|---|---|---|
| Input-Output (IO) | Input goes straight to output | The basic prompt, the model answers at once |
| Chain-of-Thought (CoT) | 1 chain | Intermediate thoughts inside 1 chain |
| Self-Consistency (CoT-SC) | Many independent chains | Sample many chains, then select the chain with the best score or the majority answer |
| Tree of Thoughts (ToT) | Tree | Branch out from any thought, score each thought, and backtrack from weak thoughts |
| Graph of Thoughts (GoT) | Graph | Aggregate thoughts into a new thought, and loop over a thought to refine it |
Each row adds 1 new move to the row above it.
Remembering tip
Line, Lines, Tree, Graph.
The new move at each level is: steps, voting, backtracking, merging.
Chain-of-Thought (CoT)
CoT asks the model to explain each reasoning step before it gives the final answer. A typical prompt is "Let's reason this out step by step." The basics of the method are in Chain-of-Thought Prompting.
Why CoT is useful:
- LLMs often jump straight to an answer and show no reasoning
- CoT solves logic, math and multi-hop questions more reliably
- the reasoning becomes transparent and traceable, from question to reasoning to conclusion
Example: "John's father's brother is Mark. Who is John's uncle?" The model reasons through each family relationship before it gives the answer Mark.
Without CoT, a model that must choose the faster of 2 travel routes can pick a route at once and pick the wrong one. With CoT, it adds up the travel time of each route first, then it compares the totals.
Challenges of CoT:
- the responses are longer
- the compute cost is higher
- an error in 1 step can carry through to the final answer
CoT is often used with RAG, so that the model reasons over the retrieved facts.
Self-Consistency (CoT-SC)
Self-consistency samples many reasoning paths for the same question. It then selects the majority answer or the best answer.
How it helps:
- 1 path can contain a wrong step, but a vote over many paths is less likely to follow that error
- it reduces one-shot guessing and encourages exploration
Tree and graph methods
The 2 structured methods are:
- Tree of Thoughts (ToT): explore several branches and prune the weaker ones
- Graph of Thoughts (GoT): let thoughts join and loop, as well as branch
These strategies make the model reason. When the model also acts on its reasoning with tools, the system becomes Agentic AI.