LLM reasoning strategies are prompting methods that control how a language model goes from the question to the answer. They change the prompt and the way that the outputs are combined. The model weights stay the same.

A thought is 1 intermediate reasoning step that the LLM writes. Each strategy arranges its thoughts in a different shape.

Comparison

StrategyShape of thoughtsKey novelty
Input-Output (IO)Input goes straight to outputThe basic prompt, the model answers at once
Chain-of-Thought (CoT)1 chainIntermediate thoughts inside 1 chain
Self-Consistency (CoT-SC)Many independent chainsSample many chains, then select the chain with the best score or the majority answer
Tree of Thoughts (ToT)TreeBranch out from any thought, score each thought, and backtrack from weak thoughts
Graph of Thoughts (GoT)GraphAggregate thoughts into a new thought, and loop over a thought to refine it

Each row adds 1 new move to the row above it.

Remembering tip

Line, Lines, Tree, Graph.
The new move at each level is: steps, voting, backtracking, merging.

Chain-of-Thought (CoT)

CoT asks the model to explain each reasoning step before it gives the final answer. A typical prompt is "Let's reason this out step by step." The basics of the method are in Chain-of-Thought Prompting.

Why CoT is useful:

  • LLMs often jump straight to an answer and show no reasoning
  • CoT solves logic, math and multi-hop questions more reliably
  • the reasoning becomes transparent and traceable, from question to reasoning to conclusion

Example: "John's father's brother is Mark. Who is John's uncle?" The model reasons through each family relationship before it gives the answer Mark.

Without CoT, a model that must choose the faster of 2 travel routes can pick a route at once and pick the wrong one. With CoT, it adds up the travel time of each route first, then it compares the totals.

Challenges of CoT:

  • the responses are longer
  • the compute cost is higher
  • an error in 1 step can carry through to the final answer

CoT is often used with RAG, so that the model reasons over the retrieved facts.

Self-Consistency (CoT-SC)

Self-consistency samples many reasoning paths for the same question. It then selects the majority answer or the best answer.

How it helps:

  • 1 path can contain a wrong step, but a vote over many paths is less likely to follow that error
  • it reduces one-shot guessing and encourages exploration

Tree and graph methods

The 2 structured methods are:

These strategies make the model reason. When the model also acts on its reasoning with tools, the system becomes Agentic AI.