Agent Planning — Deep Dive

Prompt template for generative agent daily planning balancing immediate and long-term believability.

planning_prompt = (
    "{Intro of an agent X}. Here is X's plan today in broad strokes: "
    "1) Relationships between agents and observations of one agent by another "
    "are all taken into consideration for planning and reacting. "
    "Environment information is present in a tree structure."
)

Imagine planning a road trip: one way is to just drive and ask for directions at every turn, while the other is to study a map and write down the route before leaving. This chapter explains the second style—building an AI agent that creates a plan first. It is meant to help the agent avoid wandering down wrong paths by giving it a clear sequence of steps to follow.

The plan-first agent works in two phases. First, a “planner” generates a list of subgoals, like “search for flights, then book hotel.” Then, separate “executors” carry out each step one at a time—for instance, calling a search tool or a booking API. After each step, the agent checks the result and, if needed, re-plans the remaining steps. This design appears in the Plan-and-Execute architecture from the source, where the planner uses a large language model (LLM) and the executors can use smaller, faster models. Another example is ReWOO, which lets the agent reason about the whole task before making any observations, cutting down on repeated LLM calls.

The trickiest point is that even a careful plan can break when something unexpected happens, because LLMs struggle to adjust their plans on the fly. To guard against this, KnowAgent introduces an “action knowledge base” and a “knowledgeable self-learning strategy” that constrain which actions the agent can take next, preventing it from hallucinating a wrong move. If the planner makes a mistake, the agent might execute a step that leads nowhere—like driving to a closed hotel. Without planning, the agent would react to each new piece of information without a map, often getting stuck in dead ends or ignoring long‑term goals entirely.

In the Plan-and-Execute agent architecture, the ordered mechanism begins with a planner component that uses an LLM to generate a multi-step plan covering the entire task. This plan is then handed to one or more executor components, each responsible for taking a single step from the plan and invoking the appropriate tool (such as a database query or API call). After all steps are completed, the system invokes a re-planning prompt, which lets the agent decide whether to finish with a response or generate a follow-up plan if the first plan did not have the desired effect. On failure—for instance, if an executor’s tool call returns an error—the re-planning loop is re-entered, potentially producing a new plan that accounts for the failure.

The invariant the design preserves is that cost savings and better overall performance are maintained by not calling the large planner LLM on every single tool invocation. Instead, the planner’s LLM is invoked only at planning and re-planning points, while the executors may use lighter-weight LLMs or no LLM at all. This directly addresses the downside of traditional ReAct agents (thought-act-observation loops), which require an LLM call for each action and only plan one sub-problem at a time. The planning guarantees that the agent’s reasoning is forced to cover the whole task, reducing sub‑optimal trajectories.

The key trade‑off is choosing the planning architecture over the obvious alternative: a pure ReAct-style agent that interleaves thought, act, and observation at every step. ReAct is simpler to implement but suffers from high latency and cost because each tool invocation demands a fresh LLM call, and the model never “sees” the entire task at once. Plan-and-Execute rejects that per‑step LLM call pattern, accepting instead the risk that the initial plan may be flawed or that a tool failure may not be fully corrected during re‑planning. The cost it avoids is the multiplicative expense of many LLM calls per task and the compounding of local decisions that ignore the global objective.

A concrete failure mode occurs when an executor’s tool call returns an unexpected result—for example, a database query that was assumed to return data returns an empty set. The executor completes the step with that incorrect output, and the re‑planning prompt may not detect the discrepancy because it relies on the same original plan context. An operator would see the agent enter a loop: the re‑planning prompt repeatedly generates similar follow-up plans that do not alter the flawed step, or the agent produces a final response that logically contradicts the missing data. The signal is a log of multiple identical re‑planning cycles followed by an obviously wrong answer, with no intermediate check that the executor’s output matched the plan’s expectation.

Failure 1: Planning Hallucination

• Trigger — The agent lacks built-in action knowledge, leading it to generate an invalid or unrealistic plan.
• Guard — The action knowledge base and knowledgeable self-learning strategy from KnowAgent constrain the action path during planning, mitigating hallucination.
• Posture — Fail-soft: the guard filters or corrects the plan, allowing the agent to continue with a valid trajectory rather than aborting.
• Operator signal — The agent invokes the action knowledge base and applies the knowledgeable self-learning strategy; no explicit error log is emitted, but the operator sees a constrained plan instead of a hallucinated one.
• Recovery — The guard prevents the hallucinated plan from being executed; no retry is needed because the constraint operates during plan generation.

Failure 2: Inability to Adjust Plan on Unexpected Errors

• Trigger — An unexpected error occurs during execution, and the LLM struggles to adjust its plan (as stated in the chapter).
• Guard — No guard is identified in the source: the chapter only notes this as a limitation, with no exception handling, retry, fallback, or validation component presented.
• Posture — Fail-hard: the agent cannot adapt, so the run aborts or produces an incorrect result.
• Operator signal — Silent absence of any correction or re‑planning; the agent may output a confusing response or remain stuck.
• Recovery — Manual intervention required: the operator must restart the task or provide an explicit corrective prompt.

Failure 3: Long-Term Planning Breakdown

• Trigger — The task is complex and requires planning over many steps; long‑term planning and task decomposition remain challenging.
• Guard — The re‑planning prompt from the Plan‑and‑Execute architecture: after execution of one step, the agent is called again with a re‑planning prompt to decide whether to finish or generate a follow‑up plan.
• Posture — Fail-soft: the agent attempts to re‑plan rather than aborting, degrading performance but continuing.
• Operator signal — The re‑planning prompt is invoked; the operator sees that the agent is generating a new plan mid‑execution.
• Recovery — The agent generates a follow‑up plan based on the re‑planning prompt; no fixed retry count is specified, but the loop repeats until success or manual stop.

Failure 4: Agent Stuck in Loop

• Trigger — The trajectory contains a sequence of consecutive identical actions that lead to the same observation (hallucination in the trajectory).
• Guard — The heuristic function from the self‑reflection mechanism: it determines when the trajectory is inefficient or contains hallucination and should be stopped.
• Posture — Fail-hard: the heuristic function stops the current trajectory to prevent endless looping.
• Operator signal — The heuristic function triggers; the operator sees that the trajectory is terminated and self‑reflection is about to begin.
• Recovery — Self-reflection is performed using two‑shot examples (failed trajectory, ideal reflection), and up to three reflections are added to the agent’s working memory as context for future queries.

Failure 5: Task Decomposition Failure

• Trigger — The planner fails to decompose the task into valid, achievable sub‑tasks due to task complexity (the source notes that task decomposition remains challenging).
• Guard — No guard is explicitly provided in the source: the planner is mentioned as part of Plan‑and‑Execute, but no fallback or validation for decomposition failure is described.
• Posture — Fail-hard: without a valid decomposition, the agent cannot proceed and the run effectively halts.
• Operator signal — The agent either outputs an incomplete or incorrect multi‑step plan, or fails to generate any plan; the operator observes a missing or broken execution sequence.
• Recovery — Manual step required: the operator must replan the decomposition manually or restart with a simpler prompt.

Parallel task decomposition: breaking a topic into three independent subtasks (joke, story, poem) and aggregating results.

@entrypoint()
def parallel_workflow(topic: str):
    joke_fut = call_llm_1(topic)
    story_fut = call_llm_2(topic)
    poem_fut = call_llm_3(topic)
    return aggregator(
        topic, joke_fut.result(), story_fut.result(), poem_fut.result()
    ).result()

for step in parallel_workflow.stream("cats", stream_mode="updates"):
    print(step)
    print("\n")

Think of a chef preparing a complex meal: they break the recipe into steps—chop vegetables, simmer sauce, plate—before touching a knife. This task decomposition helps a language model handle big goals by dividing them into smaller, manageable pieces it can work through one at a time. It is for turning a vague request into a reliable sequence of actions.

The model works through each step separately, using a thinking process much like writing out each leg of a journey before moving. This method, known as chain-of-thought reasoning, lets the model consider what to do next without rushing. Explicit intermediate steps improve reliability because they give the model a clear path to follow. But if an unexpected error occurs—like a missing ingredient—the model struggles to adjust its plan. According to the source, models are less robust than humans who learn from trial and error, and planning over a lengthy history remains challenging.

The trickiest point is why the model fails to adapt: it cannot easily revisit or change its earlier decisions once made. The source specifically notes that “planning over a lengthy history and effectively exploring the solution space remain challenging.” Without this subsystem, the model would get stuck on a multi‑step goal, producing incomplete or nonsensical outputs—like a traveler who never starts because they cannot decide the first turn. The concrete failure a beginner would feel is a confusing, half‑finished answer that feels like the model lost its way halfway through.

The subsystem initiates by employing the action knowledge base to constrain the planning trajectories, ensuring that the agent generates reasonable action paths before any execution. Next, the language agent follows these constrained paths to produce executable actions, enabling step-by-step task progression. Upon failure—such as when planning hallucination occurs, where the model deviates from feasible pathways—the knowledgeable self-learning strategy is invoked to adjust the model and mitigate errors, allowing for iterative correction without full re-planning.

The invariant preserved is that action paths are consistently constrained to reasonable trajectories, as enforced by the action knowledge base. This guarantee ensures that the generated plans remain aligned with executable actions, preventing unbounded exploration of incorrect paths and maintaining reliability in task completion.

The key trade-off involves using explicit action knowledge to guide planning versus relying entirely on the LLM's intrinsic capabilities. The obvious alternative rejected is free-form generation without action knowledge, which leads to planning hallucination—a state where the model fabricates illogical or unexecutable actions. By avoiding this approach, the design eliminates the cost of frequent manual intervention to correct implausible trajectories, shifting the burden to structured constraints that inherently reduce error likelihood.

A concrete failure mode is planning hallucination, where the agent produces action paths that are logically inconsistent or infeasible. An operator would observe that, in environments like ALFWorld, the agent attempts actions that fail to interact correctly with the environment (e.g., trying to open a door without being near it), resulting in unexecuted steps or task interruptions. This manifests as a prompt-dependent error, often signaled by repeated failures to achieve subtasks despite adequate token budgets.

Failure 1: Hallucinated Action Sequence

• Trigger — The language model generates a sequence of consecutive identical actions that lead to the same observation in the environment.
• Guard — The heuristic function h_t determines when the trajectory contains hallucination and should be stopped. The agent optionally resets the environment to start a new trial based on self‑reflection results.
• Posture — Fail‑soft. The environment is reset and a new trial begins; the overall run degrades but continues.
• Operator signal — Observation of repeated identical actions (the heuristic detects the pattern). The exact log or metric is not specified, but the operator would see the same observation repeated.
• Recovery — The agent resets the environment and starts a new trial. The self‑reflection mechanism (using two‑shot examples of failed trajectories) is added to the agent’s working memory (up to three reflections) to guide future changes.

Failure 2: Inefficient Planning Trajectory

• Trigger — The trajectory takes too long without reaching success.
• Guard — The same heuristic function h_t assesses efficiency and stops the trajectory when it is deemed inefficient.
• Posture — Fail‑soft. The environment is reset and a new trial is initiated.
• Operator signal — Silent absence of success after a timeout; the heuristic triggers a reset without an explicit error message. The operator would observe no progress and then a restart.
• Recovery — Environment reset and start of a new trial. Self‑reflection is invoked (again, two‑shot examples, up to three reflections in working memory) to adjust the plan.

Failure 3: Planner Produces an Incomplete Plan

• Trigger — The planner (LLM) generates a multi‑step plan that is missing critical steps or leads to a dead‑end where no tool invocation can complete the task.
• Guard — In the Plan‑And‑Execute architecture, after execution the agent is called again with a re‑planning prompt, which lets it decide whether to finish or generate a follow‑up plan if the first plan did not have the desired effect.
• Posture — Fail‑soft. The agent re‑plans, allowing the run to continue (degraded but still progressing).
• Operator signal — Observation that the agent repeatedly triggers re‑planning without advancing; or final response is still incomplete.
• Recovery — The re‑planning prompt causes the planner to produce a new plan. No retry count or backoff is specified; the agent iterates until it decides to finish.

Failure 4: Tool Execution Error (e.g., API Failure)

• Trigger — During the execution step, a tool invocation fails (e.g., a database query error or API timeout).
• Guard — No explicit guard, retry, fallback, or validation is shown in the source for this failure. The model’s Observation step simply receives the error message.
• Posture — Fail‑soft or fail‑hard depending on the model’s subsequent action; the source does not specify a guaranteed fallback. The run degrades if the model retries via another thought‑action cycle, otherwise it stalls.
• Operator signal — Error message appears as part of the observation (e.g., “API call failed”).
• Recovery — No automated recovery is described. The operator would need to manually intervene or restart the run. The model may attempt a different action in the ReAct loop, but no retry count or backoff is given.

Failure 5: Acceptance of Unsafe Subtask (e.g., Illicit Synthesis)

• Trigger — The model, when decomposing a task, generates a subtask that involves a known chemical weapon or other prohibited action. In the source, 36% of such requests were accepted.
• Guard — No explicit guard is shown in the source. Rejection sometimes occurs after a Web search (5 out of 7 rejections) or based on prompt only (2 out of 7). There is no systematic safety filter identifier.
• Posture — Fail‑open. The agent proceeds to attempt the subtask (e.g., consult documentation to execute the synthesis).
• Operator signal — The agent output shows it is trying to acquire a synthesis solution and consulting documentation for the illicit procedure.
• Recovery — Only manual oversight can interrupt the unsafe trajectory. No automated recovery is provided.

Failure 6: Context Window Overflow Due to Long Reasoning Trace

• Trigger — The repeated thought‑action‑observation cycles accumulate too many tokens, exceeding the LLM’s context window limit.
• Guard — No guard is shown in the source. The Chain of Hindsight (CoH) regularization masks 0%–5% of past tokens during training to avoid copying, but this is not a runtime guard against overflow.
• Posture — Fail‑hard. The model stops generating or produces incoherent output once the context limit is reached.
• Operator signal — Truncated response or an error message from the LLM call (e.g., “context length exceeded”).
• Recovery — The operator must manually restart with a shorter plan or increase the context window (if possible). No automated retry is described.

Plan-And-Execute: Orchestrator generates sections plan, workers execute each section, synthesizer assembles final report.

from typing import List

class Section(BaseModel):
    name: str = Field(description="Name for this section of the report.")
    description: str = Field(description="Brief overview of the main topics to be covered.")

class Sections(BaseModel):
    sections: List[Section] = Field(description="Sections of the report.")

planner = llm.with_structured_output(Sections)

@task
def orchestrator(topic: str):
    report_sections = planner.invoke([
        SystemMessage(content="Generate a plan for the report."),
        HumanMessage(content=f"Here is the report topic: {topic}"),
    ])
    return report_sections.sections

@task
def llm_call(section: Section):
    result = llm.invoke([
        SystemMessage(content="Write a report section."),
        HumanMessage(content=f"Here is the section name: {section.name} and description: {section.description}"),
    ])
    return result.content

@task
def synthesizer(completed_sections: list[str]):
    final_report = "\n\n---\n\n".join(completed_sections)
    return final_report

Think of a project manager who writes a detailed to-do list for a team, then checks in to revise the list whenever a step fails or new info arrives. This subsystem is built to break a large, messy task into smaller, manageable steps, and to keep the plan flexible when things go wrong instead of stubbornly sticking to the original script.

The system has two roles: a planner and an executor. The planner first writes a multi-step plan for the whole task, like the manager sketching the day’s steps. Then the executor carries out each step one by one, often using separate, simpler tools or models so the big planner doesn’t have to be consulted after every single action. When a step fails or the situation changes, the planner gets called back to re-plan—it can adjust the remaining steps based on the latest results and any reflections on what went wrong. This real mechanism, called the “planner” and “executor” in the source, replaces the old pattern where the large language model had to decide every tiny action itself.

The trickiest part is why splitting planner from executor makes the whole system more robust. Without this split, the large model would only see one sub‑problem at a time (like a boss who can’t see the full blueprint) and would cost a full model call per action. By letting the planner step in only for (re‑)planning, the system forces it to think about the entire task at once, and it can use lighter models for the executor steps—saving time and money. The source calls this a “re‑planning prompt” that lets the planner decide whether to finish or generate a follow‑up plan. If the planner were not separate, unexpected errors would freeze the agent: it would keep repeating one‑step decisions without ever zooming out to see the bigger picture, leading to broken, expensive attempts that never learn from trial and error.

The Plan-And-Execute subsystem, as implemented in LangGraph and informed by the Plan-and-Solve prompting paradigm, separates a task into two distinct roles: the planner generates a multi-step plan, and the executor(s) carry out each step by invoking tools. The ordered mechanism begins with the planner receiving the user query and producing a sequence of sub‑tasks. The executor then processes each step in order, calling the appropriate tool for that step. Once all steps are executed, the system invokes a re‑planning prompt that allows the planner to assess the outcome. If the result matches expectations, the agent produces a final response; otherwise, the planner generates a follow‑up plan to recover or adjust. On failure—for example, if a tool call returns an error—the executor returns an observation to the planner, which triggers a re‑plan cycle rather than continuing blindly.

The design preserves the invariant that the large, general‑purpose language model is only consulted during planning and re‑planning phases, never for each individual tool invocation. This guarantees that the expensive planner is used sparingly, while the executor can be a lighter‑weight model or even non‑LLM code for each action. The system thus avoids the per‑step LLM call overhead of the traditional ReAct agent, where every action requires a full “thought, act, observation” loop calling the LLM. The invariant is explicitly named in the source as “avoid having to call the large planner LLM for each tool invocation.”

The key trade‑off is between serial, step‑by‑step reasoning (as in ReAct) and explicit, ahead‑of‑time planning. The obvious alternative it rejects is the ReAct agent, which calls the same LLM for every sub‑problem. That approach forces the model to reason about only one step at a time, leading to sub‑optimal trajectories because the LLM is “not forced to ‘reason’ about the whole task.” By rejecting this design, Plan‑And‑Execute avoids the cost of an LLM call per tool invocation and also reduces the risk of shortsighted decisions. The cost of this rejection is the need for a separate re‑planning step and the possibility that the initial plan may be wrong, but the re‑plan loop mitigates that.

A concrete failure mode occurs when the executor attempts a tool call that fails—for example, a database query times out or an API returns a 500 error. The executor logs the error as the observation and passes it to the re‑planning prompt. The operator would see a signal such as a log entry showing the failed tool call (e.g., Search("current score") returned error: timeout) followed by a re‑plan step where the planner decides whether to retry, modify the step, or abort. This observable sequence of “failed action → re‑plan” is the direct signal that the executor encountered an unexpected condition, and the planner’s intervention is the recovery mechanism built into the system.

Inefficient Plan Trajectory

• Trigger — The planner generates a multi-step plan that takes too long without achieving success, causing the trajectory to exceed a reasonable length.
• Guard — The heuristic function (h_t) is computed after each action; if it determines the trajectory is inefficient, it stops the current trial.
• Posture — Fail-soft: the environment is reset and a new trial begins, allowing the agent to continue trying with a fresh plan.
• Operator signal — A log entry indicating “Inefficient planning” or a metric showing the trajectory exceeded the efficiency threshold.
• Recovery — The environment is reset to start a new trial; the agent’s working memory may retain self-reflection from prior trials to guide the next plan.

Hallucination Cycle

• Trigger — The executor repeats a sequence of consecutive identical actions that produce the same observation, indicating the agent is stuck.
• Guard — The same heuristic function (h_t) detects hallucination (consecutive identical actions leading to the same observation) and aborts the trajectory.
• Posture — Fail-soft: the environment is reset and a new trial begins, discarding the hallucinated loop.
• Operator signal — A log entry capturing “Hallucination detected” or a metric counting consecutive identical observations.
• Recovery — The environment is reset; the agent may incorporate self-reflection from the failed trajectory to avoid repeating the same pattern.

Executor Tool Failure

• Trigger — A step in the plan requires the executor to call a tool (e.g., an API or database query), but the tool returns an error, times out, or is unavailable.
• Guard — The source does not show a guard for this failure; no exception handler, retry mechanism, or fallback is specified for tool execution errors in the Plan-And-Execute description.
• Posture — Fail-hard: without a guard the executor cannot complete the step, and the execution loop halts with no automatic degradation.
• Operator signal — An error response from the tool call (e.g., an HTTP error code or timeout exception) that is not captured by any handler.
• Recovery — None is defined; the agent does not automatically retry or fall back. Manual intervention or a re‑planning prompt that only triggers after execution completes would not run because execution never completed.

Planner Generates Infeasible Step

• Trigger — The planner produces a step that cannot be executed (e.g., requires a tool not in the agent’s tool list, or expects an input the environment cannot provide).
• Guard — The source does not show a guard for this failure. The executor will attempt to invoke the step and fail, but there is no validation check before execution.
• Posture — Fail-hard: the executor cannot honor the step, and the execution sequence breaks.
• Operator signal — An error from the executor (e.g., “Tool not found” or “Invalid input”) that is unhandled.
• Recovery — None specified; the agent does not automatically replan because it never receives a completed execution status. Manual restart or a change to the planner’s instructions is required.

Re‑planning Loop

• Trigger — After each execution of a plan, the re‑planning prompt decides to generate a follow‑up plan indefinitely, never reaching a condition that causes it to return a final response.
• Guard — The source does not show a guard for this failure. The heuristic function (h_t) applies within a trajectory (per action), but the re‑planning loop spans multiple execution cycles and is not limited by that heuristic.
• Posture — Fail-soft: the agent continues generating and executing plans with no internal stop, degrading performance over time.
• Operator signal — A growing sequence of “re‑planning” logs with no final answer, or a metric showing repeated plan‑execute cycles without completion.
• Recovery — Manual intervention is needed to halt the process; no automatic retry count or backoff is defined for the re‑planning step.

Agent creation with summarization, memory, and skills middleware for interleaving reasoning and action.

from deepagents.backends import StateBackend
from deepagents.middleware import MemoryMiddleware, SkillsMiddleware, SummarizationMiddleware

backend = StateBackend()
model = "anthropic:claude-sonnet-4-6"

agent = create_agent(
    model=model,
    tools=[search],
    middleware=[
        SummarizationMiddleware(model=model, backend=backend),
        MemoryMiddleware(backend=backend, sources=["./AGENTS.md"]),
        SkillsMiddleware(backend=backend, sources=["./skills/"]),
    ],
)

Imagine a chef who doesn’t scribble a fixed menu beforehand but instead tastes the dish at each step, then decides what to add next. That is what this method does: it helps an AI system tackle long, complex tasks without getting lost because it lets the system think and act in tiny, back‑and‑forth steps rather than committing to one big plan upfront.

Here is how it actually works. The system keeps a running memory stream of everything it has done and observed. When it needs to decide its next move, it retrieves past observations that are most relevant and recent—like a chef remembering what spice he just added. Then it reflects on those memories to form a higher‑level summary of what worked and what didn’t. Armed with that summary and the current situation, it plans its next action. This loop—retrieve, reflect, plan, act—repeats constantly, so the system can change course the moment an unexpected result appears.

The trickiest part is knowing when to stop and reflect. The system uses a hidden heuristic that checks whether the last action led to a good outcome. If the heuristic signals a bad outcome, the system can decide to reset the environment and start a fresh trial, guided by the lessons it just learned. Without that safety valve, the chef would keep adding salt to a dish that is already salty, never tasting to adjust—the AI would blindly repeat mistakes, get stuck on a broken plan, and fail to recover from even a small surprise.

In the "Reasoning Meets Acting" subsystem, the ordered mechanism begins with the agent retrieving past observations from its memory stream using Maximum Inner Product Search (MIPS) , which scores stored experiences by relevance and recency. After retrieval, the agent performs self-reflection to form higher‑level summaries and refine past decisions. It then generates a subgoal and interleaves reasoning with acting via the ReAct template: first a Thought: trace, then an Action: (e.g., a tool call), followed by an Observation: from the environment. This cycle repeats. On failure—such as an observation that conflicts with the expected outcome—the agent invokes reflection again (as in the Reflexion framework, which uses a heuristic and reward model) to adjust its next thought or even reset the environment for a new trial, ensuring the plan remains adaptive.

The design preserves the invariant of self-reflection for iterative improvement. This guarantee, named explicitly in the source, ensures that the agent’s decisions are continuously refined by correcting past mistakes and integrating new observations. Because every action is grounded in the memory stream and reflected upon, the agent maintains a coherent trajectory without drifting from task objectives. The invariant prevents the agent from committing to a fixed upfront plan that cannot adapt, instead enforcing a feedback loop where each step is validated against both current context and historical experience.

The key trade‑off is adaptability versus per‑step cost. This design rejects the alternative of separating planning from execution (e.g., the plan‑and‑execute style agents or the LLM+P approach that outsources planning to an external PDDL planner). By requiring an LLM call for every thought and action, the ReAct‑style agent incurs higher latency and expense compared to executing sub‑tasks without LLM involvement. However, this cost is accepted because interleaving reasoning with action avoids the far more expensive failure of having to replan an entire multi‑step sequence when an unexpected error occurs. The rejection of a strict upfront plan thus saves the computational overhead of re‑generating a complete plan from scratch, a cost that would be paid repeatedly in dynamic environments.

A concrete failure mode is planning hallucination, where the agent generates unrealistic or invalid action sequences that deviate from the task. For example, the agent might repeatedly output Thought: I need to search for X followed by Action: Search(Y) even after receiving a Observation: No results. The operator would see a log of repeated, identical Thought: and Action: entries without progress. This signal indicates that the retrieval or reflection components failed to ground the agent’s reasoning in the current observation, a symptom that KnowAgent mitigates by constraining action paths with an explicit action knowledge base. Without such mitigation, the loop persists, wasting LLM calls and delaying task completion.

Planning Hallucination

• Trigger — The language model generates an action sequence that is infeasible, contradictory, or based on fabricated tool outputs during the planning step of the generative agent loop.
• Guard — The heuristic function explicitly detects when the trajectory is inefficient or contains hallucination – defined in the source as “encountering a sequence of consecutive identical actions that lead to the same observation” – and decides to stop the trial.
• Posture — fail-hard: the heuristic stops the current run, preventing further wasted iterations.
• Operator signal — The trajectory is cut short; the operator sees a truncated plan and an abrupt termination without a final answer. The heuristic’s stopping criterion is the only logged decision point.
• Recovery — After the heuristic fires, the agent performs self-reflection using two-shot examples of failed trajectories. The reflection is added to the agent’s working memory (up to three reflections), which influences the next planning query. The agent then retries with a new trial from the start.

Tool Execution Failure

• Trigger — The agent calls an external tool (e.g., a search API, database query, or code execution) and the tool returns an error, timeout, or unparseable response.
• Guard — None explicitly described in the source. The ReAct pattern captures only an Observation field, but no dedicated exception handler, retry mechanism, or validation step is named for tool errors.
• Posture — fail-soft: the agent treats the error as an observation and continues the Thought–Action–Observation loop, potentially reusing the same action or hallucinating a recovery.
• Operator signal — The Observation step contains an error message (e.g., “API returned 500” or “tool not found”), which the operator can see in the logged trajectory. No metric or log line specifically flags the failure.
• Recovery — None intrinsic. The agent may attempt the same action again (risking hallucination) or move to a different tool based on its reasoning. The heuristic function could eventually catch repeated identical actions and stop the trajectory, but that is an indirect, late-stage recovery.

Memory Retrieval Failure

• Trigger — The agent’s memory stream fails to retrieve relevant past observations because the relevance or recency scoring mechanism misses the correct memory, or the reflection store is empty.
• Guard — None explicitly named in the source. The description states the agent “retrieves past observations based on relevance and recency” but no exact guard variable, validation check, or fallback function is provided.
• Posture — fail-soft: the agent proceeds with incomplete context, degrading the quality of subsequent reflections and plans.
• Operator signal — Silent absence: the operator sees plausible but suboptimal planning without any logged error, making the root cause invisible. No metric records retrieval misses.
• Recovery — None specified. The agent will use whatever memories are retrieved; there is no explicit retry or alternative retrieval strategy documented in the source.

Reflection Inaccuracy

• Trigger — When forming higher-level summaries from the memory stream, the language model overgeneralizes, fabricates, or misattributes conclusions from past observations.
• Guard — None dedicated to reflection in the source. The heuristic function targets planning hallucination and inefficient trajectories, not the correctness of reflection summaries. Self-reflection is a corrective step that occurs after a failed trajectory, not during reflection creation.
• Posture — fail-soft: the agent uses the flawed summary as input for subsequent planning, leading to persistently poor decisions until a trajectory failure triggers the heuristic.
• Operator signal — The operator may notice that plans are inconsistent with the agent’s recent history (e.g., ignoring a critical failure mentioned in an earlier observation), but no direct log or metric flags the inaccurate reflection.
• Recovery — Indirectly, a subsequent failure of the planner (caught by heuristic function) will invoke self-reflection, which can correct the mistaken summary in the next round. No direct recovery path exists for the reflection step itself.

Inability to Adjust Plans to Unexpected Errors

• Trigger — The environment changes in an unanticipated way (e.g., an API deprecation, a resource exhaustion, or contradictory new information) that the agent’s memory stream and current environment snapshot do not explain.
• Guard — The heuristic function may detect inefficiency if the agent repeats actions that no longer work, but it has no explicit mechanism to recognize unexpected errors as requiring a plan adjustment. Self-reflection is applied only after a trajectory is stopped.
• Posture — fail-soft: the agent continues its current plan, possibly with repeated suboptimal actions, until the heuristic triggers or the plan runs to completion with incorrect results.
• Operator signal — The operator observes a prolonged trajectory with no progress, or the agent fails to incorporate clear error signals from observations. No dedicated error field exists for “unexpected change.”
• Recovery — Only when the heuristic stops the trajectory does self-reflection add a hint to working memory, which may guide a future plan. No online adaptation or explicit fallback plan is described in the source.

Using summarization middleware to compress and reflect on past observations, generating higher-level inferences that guide future reasoning.

from deepagents.backends import StateBackend
from deepagents.middleware import FilesystemMiddleware, MemoryMiddleware, SkillsMiddleware, SummarizationMiddleware

backend = StateBackend()
model = "anthropic:claude-sonnet-4-6"

agent = create_agent(
    model=model,
    tools=[search],
    middleware=[
        FilesystemMiddleware(backend=backend),
        SummarizationMiddleware(model=model, backend=backend),
        MemoryMiddleware(backend=backend, sources=["./AGENTS.md"]),
        SkillsMiddleware(backend=backend, sources=["./skills/"]),
    ],
)

Imagine an explorer who, instead of rushing down a single path, stops to jot down what went wrong at each dead end and then uses those notes to pick a smarter route forward. That is what this subsystem does: it lets a large language model learn from its own missteps instead of stubbornly repeating the same bad plan. Without it, the model would keep bulldozing ahead even after hitting an error, like a hiker who ignores a cliff and keeps walking.

In practice, the system collects the model’s one hundred most recent observations—tiny records of what happened—and turns them into a short list of three big-picture questions. For example, “Which part of my last answer caused confusion?” The model then answers those questions, distilling the raw memories into a higher‑level inference that directly steers its next actions. This is the reflection mechanism named in the source: it synthesizes past mistakes into guidance for future behavior, much like the explorer writing a lesson like “avoid paths with loose rocks.”

The trickiest detail is that the model must decide which three questions matter most from a flood of recent data. If it picks trivial ones, the reflection becomes useless—the agent might keep making the same error. The source notes that LLMs struggle to adjust plans when they face unexpected errors; the reflection mechanism forces the model to pause and deliberately prioritize, using a limited context window (the last hundred observations) to force focus. Without this subsystem, the model would commit to one line of reasoning too quickly, collapsing into the very failure a beginner would feel: the agent endlessly repeating a mistake because it never bothered to ask why.

The subsystem is built on the self-reflection mechanism described in the agent architecture overview. The ordered mechanism begins with the agent using short-term memory (in-context learning) to decompose tasks and plan subgoals. Next, it invokes Self‑Reflection: the agent synthesizes past memories into higher-level inferences and then guides its future behavior by self-criticizing and refining its actions. On failure—for example, when an action produces an unexpected error—the reflection step re-engages: the agent evaluates the mistake, draws higher-level lessons from the memory of that failure, and adjusts subsequent planning to avoid repeating the error. This loop continues until the task is solved or a termination condition is met.

The design preserves the invariant of preference-based log odds optimization, as defined in the PRefLexOR framework. By optimizing the log odds between preferred and non‑preferred responses via rejection sampling, the system ensures that reasoning steps are continuously steered toward more desirable outcomes. Additionally, masking reasoning steps during training focuses the model on discovery, guaranteeing that feedback from failures is used to improve rather than reinforce static paths. This invariant guarantees that the agent’s reasoning depth increases over time, even when starting from small (3‑billion‑parameter) models.

The key trade-off is between reasoning depth and inference speed. The technique rejects the obvious alternative of committing to a single line of reasoning early and executing it without revision. That approach would be simpler and faster but brittle, because the agent cannot recover from unexpected errors. By instead recursively refining intermediate steps and using self-reflection to learn from mistakes, the system gains robustness and cross‑domain adaptability. The cost it avoids is the repeated failure of a rigid plan—a failure that would require human intervention or full restart. The price paid is increased computational overhead from iterative optimization and multiple passes over the same problem.

A concrete failure mode occurs when masking reasoning steps is misapplied, causing the model to lose crucial context and generate incoherent high-level questions. An operator would observe that the model’s self‑reflection outputs become repetitive or nonsensical—for example, the agent repeatedly generates the same three questions across different contexts, or the answers contradict prior successful steps. The signal is a sudden drop in task completion rate alongside an increase in the number of reflection cycles that produce no change in the agent’s behavior. This indicates that the preference signal has saturated or that the masking is too aggressive, preventing the model from learning useful new paths.

Failure 1: Question Generation Hallucination

• Trigger — Ambiguous or contradictory observations cause the model to invent high‑level questions that are not grounded in the actual data, leading to irrelevant reflections.
• Guard — No explicit guard is shown in the source for hallucination in question generation. The heuristic function (described later) detects hallucination only in action sequences (identical consecutive actions), not in the reflection step itself.
• Posture — fail‑soft — the system continues to use the flawed questions and answers, degrading the quality of subsequent planning without aborting.
• Operator signal — The operator would observe downstream repeated identical actions or inefficient trajectories; no direct log of question hallucination exists.
• Recovery — No automatic recovery is triggered by this failure. The system relies on the eventual detection of inefficiency by the heuristic function to reset, but that is a separate mechanism.

Failure 2: Insufficient Context Window

• Trigger — A task requires reasoning over more than the 100 most recent observations (e.g., long‑term dependencies, episodic memory). The reflection mechanism discards earlier critical information.
• Guard — No guard is provided in the source. The subsystem does not include a mechanism to expand, summarize, or retrieve older observations.
• Posture — fail‑soft — the agent continues planning with incomplete context, which increases the probability of incorrect inferences and sub‑optimal actions.
• Operator signal — The operator would see a gradual rise in repeated errors or failure to adapt to recurring patterns, but no explicit warning.
• Recovery — No automatic recovery; manual intervention to adjust the context window or augment memory is required.

Failure 3: Inefficient Planning due to Over‑Broad Reflection

• Trigger — The generated high‑level questions are too numerous or vague, causing the agent to pursue many tangents and take excessive steps without progress.
• Guard — The heuristic function (exact identifier: heuristic function) determines when the trajectory is inefficient (takes too long without success) and optionally triggers a reset.
• Posture — fail‑soft — the heuristic function signals that the current trial should be stopped, and the environment is reset to start a new trial; the run continues.
• Operator signal — The operator would observe a reset event; the source states the agent “optionally may decide to reset the environment” based on self‑reflection results.
• Recovery — The environment is reset and a new trial begins immediately; no backoff or retry count is specified.

Failure 4: Failure to Generate Meaningful Questions

• Trigger — The model lacks domain expertise or the observations are too sparse, causing it to produce generic or trivial questions that do not advance reasoning.
• Guard — The self‑reflection mechanism uses two‑shot examples (pairs of failed trajectory and ideal reflection) to guide the model toward useful questions. These examples are included in the agent’s working memory.
• Posture — fail‑soft — the system proceeds with weak reflections, but the two‑shot examples provide a corrective guide for future reasoning.
• Operator signal — The operator would notice that the agent’s actions do not improve over multiple trials, even when the environment resets.
• Recovery — The self‑reflection mechanism incorporates up to three such reflections into working memory, and the two‑shot examples are used to adjust subsequent question generation.

Failure 5: Overfitting to Recent Observations

• Trigger — Rapid changes in the most recent observations (e.g., a sudden outlier) dominate the reflection, causing the agent to ignore longer‑term stable patterns.
• Guard — No explicit guard is shown in the source. The heuristic function checks for consecutive identical actions (hallucination) but does not balance recency versus history.
• Posture — fail‑soft — the agent’s plan becomes unstable and oscillates, but execution continues.
• Operator signal — Frequent and abrupt changes in the agent’s decisions, visible in action logs.
• Recovery — No automated recovery; the agent may eventually reset via the heuristic function if inefficiency is detected, but that is not guaranteed.

Failure 6: Anchoring Bias in Answering High‑Level Questions

• Trigger — The model commits to an early incorrect high‑level inference and then answers the self‑generated questions in a way that reinforces that bias, ignoring contradictory observations.
• Guard — No guard is provided in the source for this failure mode. The source discusses anchoring bias as an observed error in medical reasoning (MedQA evaluation) but does not present a guard within the reflection subsystem.
• Posture — fail‑soft — the agent continues with the biased inference, potentially compounding errors over successive reflections.
• Operator signal — The operator would see a persistent error pattern across multiple trials, with the agent consistently choosing the same incorrect line of reasoning.
• Recovery — No automatic recovery is defined; manual review and correction of the agent’s reasoning path are required.

AutoGPT's system message imposes a 4000‑word short‑term memory limit, forcing the agent to offload information immediately to compensate for finite context length.

system_message = """
You are {{ai-name}}, {{user-provided AI bot description}}.
Your decisions must always be made independently without seeking user assistance.
Play to your strengths as an LLM and pursue simple strategies with no legal complications.
...
Constraints:
1. ~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files.
2. If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.
3. No user assistance
...
"""

Imagine a chef following a recipe on a small notepad that can only hold a few lines at a time. This subsystem exists to help a language model adapt when the kitchen suddenly changes—ingredients run out or the oven breaks—but the notepad’s tiny size makes it nearly impossible to revise the plan. The chef writes down the first steps, then works through them, but if something unexpected happens, the notepad already overflows with past notes, leaving no room to write a new strategy or recall earlier instructions. Because the model has only that cramped notepad (called “finite context length”), it cannot keep enough history—past instructions, tool responses, or environment feedback—to recognize that the original plan no longer works. Instead, the chef confidently keeps following the outdated steps, stirring an empty bowl.

The trickiest part is why the model cannot correct itself: even when the chef notices the mistake, the notepad is so full that there is no space to explore alternative actions or replay the earlier context needed for a new plan. The source explains that this limited context restricts how much “historical information” the system can include, so the model cannot effectively search the solution space or learn from trial-and-error like a human would. Without this ability to revise on the fly, the subsystem fails in a concrete way: the agent executes actions based on a plan that is now invalid, wasting time and resources while the user waits for a result that never arrives.

In this subsystem, the ordered mechanism begins with a planner, which prompts an LLM to generate a multi-step plan covering the entire task. Executors then accept the user query and one step of the plan, invoking one or more tools to complete that sub-task. After all steps are attempted, the agent is called again with a re‑planning prompt, letting it decide whether to finish with a response or generate a follow‑up plan if the first plan did not have the desired effect. On failure—such as an unexpected error mid‑execution—the agent does not react immediately; instead, it continues executing the remaining steps until the re‑planning prompt is reached. The source explicitly describes this loop: “Once execution is completed, the agent is called again with a re‑planning prompt.”

The design preserves the invariant that the planner’s output is treated as the authoritative sequence until the entire plan has been executed; re‑planning occurs only at that boundary, not proactively mid‑step. This guarantee is named the “plan-and-execute” loop in the source. It ensures that sub‑tasks are executed serially without interruption, and that the larger LLM is consulted only for the initial plan and for post‑execution re‑planning. The cost of this invariant is that the system cannot adjust to environment changes until the plan completes, as the source notes: “LLMs struggle to adjust plans when faced with unexpected errors, making them less robust compared to humans who learn from trial and error.”

The key trade‑off is between adaptability and cost. The obvious alternative is a ReAct‑style agent, which interleaves reasoning and action after every tool invocation, using a “Thought: … Action: … Observation: …” loop. The source explicitly rejects this approach because “it requires an LLM call for each tool invocation” and the LLM “only plans for 1 sub‑problem at a time,” leading to sub­optimal trajectories and higher latency. The plan‑and‑execute design avoids that cost by batching sub‑tasks without intermediate LLM calls, using a lighter‑weight executor or no LLM per step. The rejection avoids the expense of calling the large planner LLM repeatedly, but at the cost of reduced responsiveness when the environment changes unexpectedly.

A concrete failure mode occurs when a database goes offline after the plan is generated. The executor continues to invoke Search() on the database for the next step, even though the query will return an error. The operator would see a repeated pattern in the logs: “Action: Search("...")” followed by “Observation: error” for several consecutive steps, with the agent never interrupting the sequence. The signal is that the agent produces the same plan‑step actions and error observations without ever entering a re‑planning state until all steps are attempted. This matches the source’s description: the system “may continue confidently with a plan that is no longer valid” because the finite context length and lack of mid‑step reasoning prevent it from detecting the change.

Repeated Action Hallucination

• Trigger – The agent generates a sequence of consecutive identical actions that lead to the same observation in the environment.
• Guard – The heuristic function (heuristic function / h_t) determines that this pattern constitutes hallucination and should be stopped.
• Posture – fail-hard: the trajectory is aborted when the heuristic halts further action.
• Operator signal – No explicit log line; the agent simply ceases to produce new actions after the heuristic trigger. The operator would infer a stop from the missing next observation.
• Recovery – The agent may decide to reset the environment to start a new trial, depending on the self-reflection results.

Inefficient Planning Loop

• Trigger – The trajectory takes too long without making progress toward the goal.
• Guard – The same heuristic function (h_t) determines that the planning is inefficient and halts the run.
• Posture – fail-hard: the entire run is stopped.
• Operator signal – Absence of completion or any subsequent action; the heuristic does not output a message in the source.
• Recovery – As with hallucination, a new trial may be started after self-reflection.

Plan Invalidated by Unexpected Environment Change

• Trigger – The environment changes after the plan is formed, rendering the original plan invalid.
• Guard – self-reflection (up to three reflections stored in working memory) is used to guide future changes in the plan based on past failed trajectories.
• Posture – fail-soft: the agent continues, but uses reflection to adjust its plan rather than aborting.
• Operator signal – The self-reflection entries are added to working memory; the operator could observe a change in the agent’s reasoning if logs are inspected.
• Recovery – The agent re-queries the LLM with the reflection context to produce a revised plan.

Context Window Overflow

• Trigger – The finite context length of the LLM is exceeded, causing loss of historical information (past instructions, API call context, responses).
• Guard – No guard is shown in the source. The text only states that this limitation “causes trouble” and prevents effective solution-space exploration.
• Posture – fail-soft: the agent continues confidently with an incomplete context, degrading performance.
• Operator signal – Silent absence of earlier history; the agent may repeat actions or produce contradictory decisions without any error indication.
• Recovery – No built‑in recovery is described. A manual step (e.g., truncating or summarizing context) would be required, but that is not in the source.

Lack of Built‑In Action Knowledge Leading to Planning Hallucination

• Trigger – The LLM is given no explicit action knowledge, so it plans trajectories with invalid or nonexistent action steps.
• Guard – KnowAgent’s action knowledge base constrains the action path during planning, preventing the generation of invalid steps.
• Posture – fail-soft: the knowledge base restricts the set of possible actions, so the agent degrades to a narrower but valid plan rather than aborting.
• Operator signal – The agent’s action choices are limited; the operator may see that certain possible actions are missing from the generated plan.
• Recovery – The agent follows the constrained path; no retry is needed because the guard prevents the hallucination at generation time.