Graph of Thoughts (GoT) models the output of an LLM as a graph. Each LLM thought is a vertex (node), and each edge shows a dependency between 2 thoughts. A node can have more than 1 incoming edge, so GoT can combine thoughts.
Comparison with ToT
| Operation | ToT | GoT |
|---|---|---|
| Branch out from a thought | Yes | Yes |
| Backtrack from a weak thought | Yes | Yes |
| Aggregate several thoughts into 1 new thought | No | Yes |
| Refine a thought by looping over it | No | Yes |
Remembering tip
GoT = ToT + merge + loop.
A tree can only split. A graph can also join and circle back.
Structure
GoT has 3 steps:
graph TD A[Create the graph] --> B[Identify key concepts and relationships] B --> C[Formulate prompts]
Create the graph
Build a graph for the problem or query. To do this, find the key concepts and define the relationships between them.
Example: for the question about the meaning of the word "earthquake", the rationale becomes nodes such as earth, quake, ground and shake. Edges such as comes from and means join these nodes.
Identify key concepts and relationships
Key concepts are the nodes of the graph:
- a node can be a crucial piece of information, a possible solution, or a step in a logical process
- to find the nodes, you need a deep understanding of the problem and of what the solution needs
Relationships are the edges of the graph:
- an edge can be causal, sequential, hierarchical, or any other logical connection
- the edges let the model move from 1 concept to the next
Example: for the question "Who is the director of the movie Avatar?", the nodes are the movie, its cast, its producer and its director. The edges are relations such as Director, Starring and Producer. This kind of prompt is called a Graph-CoT prompt.
Formulate prompts
How the prompts are designed:
- the prompts follow the structure of the graph
- the prompt guides the model to traverse the graph and explore the nodes and their connections, so the model does not answer in 1 straight line
- the model can consider several parts of the problem at the same time and give a better reasoned response
Code
These are the 4 classes that we import from the graph_of_thoughts library:
from graph_of_thoughts.operations.thought import Thought
from graph_of_thoughts.language_models import AbstractLanguageModel
from graph_of_thoughts.prompter import Prompter
from graph_of_thoughts.parser import Parser| Class | Role |
|---|---|
Thought | 1 thought, which is 1 node of the graph |
AbstractLanguageModel | The base class for the LLM that writes the thoughts |
Prompter | Builds the prompt for each operation |
Parser | Reads the LLM response back into thoughts |
The library keeps the reasoning plan as a GraphOfOperations:
from graph_of_thoughts.operations.operations import Operation
class GraphOfOperations:
"""
Represents the Graph of Operations, which prescribes the execution plan of thought operations.
"""
def __init__(self) -> None:
"""
Initializes a new Graph of Operations instance with empty operations, roots, and leaves.
The roots are the entry points in the graph with no predecessors.
The leaves are the exit points in the graph with no successors.
"""
self.operations: List[Operation] = []
self.roots: List[Operation] = []
self.leaves: List[Operation] = []What each list holds:
operations: every operation in the graphroots: the entry points, which have no predecessorsleaves: the exit points, which have no successors
When to use
GoT is useful for:
- complex problem solving: problems that need many solution paths and many pieces of information joined together
- workflow automation: a workflow written as a graph can be automated, for example in data analysis, strategy formulation and project management
- personalised decision-making: the reasoning paths change based on the intermediate results
- enhanced reasoning: the model combines information from different paths and refines its solutions through feedback loops
Limitations
Limitations of GoT:
- computational cost: a graph of reasoning is expensive for complex problems and large models
- implementation complexity: many reasoning paths are harder to manage than a simple prompt
- redundancy: several paths can repeat the same computation and slow down the processing
- specialised tools: graph construction and management need special tools and frameworks, which are not always available
GoT completes the reasoning methods. The next stage is a model that also acts on its reasoning, see Agentic AI.