Graph of Thoughts (GoT) models the output of an LLM as a graph. Each LLM thought is a vertex (node), and each edge shows a dependency between 2 thoughts. A node can have more than 1 incoming edge, so GoT can combine thoughts.

Comparison with ToT

OperationToTGoT
Branch out from a thoughtYesYes
Backtrack from a weak thoughtYesYes
Aggregate several thoughts into 1 new thoughtNoYes
Refine a thought by looping over itNoYes

Remembering tip

GoT = ToT + merge + loop.
A tree can only split. A graph can also join and circle back.

Structure

GoT has 3 steps:

graph TD
    A[Create the graph] --> B[Identify key concepts and relationships]
    B --> C[Formulate prompts]

Create the graph

Build a graph for the problem or query. To do this, find the key concepts and define the relationships between them.

Example: for the question about the meaning of the word "earthquake", the rationale becomes nodes such as earth, quake, ground and shake. Edges such as comes from and means join these nodes.

Identify key concepts and relationships

Key concepts are the nodes of the graph:

  • a node can be a crucial piece of information, a possible solution, or a step in a logical process
  • to find the nodes, you need a deep understanding of the problem and of what the solution needs

Relationships are the edges of the graph:

  • an edge can be causal, sequential, hierarchical, or any other logical connection
  • the edges let the model move from 1 concept to the next

Example: for the question "Who is the director of the movie Avatar?", the nodes are the movie, its cast, its producer and its director. The edges are relations such as Director, Starring and Producer. This kind of prompt is called a Graph-CoT prompt.

Formulate prompts

How the prompts are designed:

  • the prompts follow the structure of the graph
  • the prompt guides the model to traverse the graph and explore the nodes and their connections, so the model does not answer in 1 straight line
  • the model can consider several parts of the problem at the same time and give a better reasoned response

Code

These are the 4 classes that we import from the graph_of_thoughts library:

from graph_of_thoughts.operations.thought import Thought
from graph_of_thoughts.language_models import AbstractLanguageModel
from graph_of_thoughts.prompter import Prompter
from graph_of_thoughts.parser import Parser
ClassRole
Thought1 thought, which is 1 node of the graph
AbstractLanguageModelThe base class for the LLM that writes the thoughts
PrompterBuilds the prompt for each operation
ParserReads the LLM response back into thoughts

The library keeps the reasoning plan as a GraphOfOperations:

from graph_of_thoughts.operations.operations import Operation
 
class GraphOfOperations:
    """
    Represents the Graph of Operations, which prescribes the execution plan of thought operations.
    """
 
    def __init__(self) -> None:
        """
        Initializes a new Graph of Operations instance with empty operations, roots, and leaves.
        The roots are the entry points in the graph with no predecessors.
        The leaves are the exit points in the graph with no successors.
        """
        self.operations: List[Operation] = []
        self.roots: List[Operation] = []
        self.leaves: List[Operation] = []

What each list holds:

  • operations: every operation in the graph
  • roots: the entry points, which have no predecessors
  • leaves: the exit points, which have no successors

When to use

GoT is useful for:

  • complex problem solving: problems that need many solution paths and many pieces of information joined together
  • workflow automation: a workflow written as a graph can be automated, for example in data analysis, strategy formulation and project management
  • personalised decision-making: the reasoning paths change based on the intermediate results
  • enhanced reasoning: the model combines information from different paths and refines its solutions through feedback loops

Limitations

Limitations of GoT:

  • computational cost: a graph of reasoning is expensive for complex problems and large models
  • implementation complexity: many reasoning paths are harder to manage than a simple prompt
  • redundancy: several paths can repeat the same computation and slow down the processing
  • specialised tools: graph construction and management need special tools and frameworks, which are not always available

GoT completes the reasoning methods. The next stage is a model that also acts on its reasoning, see Agentic AI.