Agent Frameworks Compared

Most framework comparisons are vendor-doc collages: someone reads three quickstarts, tabulates the marketing bullets, and calls it a bake-off. This one was written the expensive way. The same agent system — a planner that fans out three parallel researchers, then a synthesizer that merges whatever came back — was built three times, once per framework, behind one shared contract:

async def run_research(topic: str, emit: Callable[[dict], None] | None = None) -> dict

Same task, same LLM (DeepSeek through an OpenAI-compatible endpoint), same retry and timing helpers, same pytest suite asserting that the agents actually overlap in time (tracker.peak == 3). Three implementations, one signature, one test. Everything that differs below is the orchestration layer — the framework — and nothing else.

That constraint is what makes the rest of this page worth reading. When the app is held fixed, "which framework is better" stops being a taste question and turns into three concrete ones: how many lines does it cost you, what does it do when a branch dies, and what does it refuse to do at all.

The verdict

For a time-boxed prototype: LlamaIndex Workflows. Leanest orchestration of the three (~87 lines against ~107 for the OpenAI Agents SDK and ~119 for LangGraph), typed events that make the design explainable on a whiteboard, a one-argument concurrency cap, and native reach into retrieval the moment the prototype needs it. Mantra: post the envelope, count the heads.

Pick LangGraph when the requirements say "durable." Pause/resume, human-in-the-loop approval, or state that must survive a process restart. Its extra ~30 lines buy a checkpointer and a resumable interrupt() that neither of the other two has at all. Mantra: every superstep is a save point.

Pick the OpenAI Agents SDK — or raw asyncio — when ceremony is the enemy. An agent is a six-line declaration, fan-out is asyncio.gather, and the framework mostly stays out of the way. Mantra: agents are name tags; asyncio does the work.

Skip LangChain proper for agents. It is not a fourth option in this race; LangGraph is what its own maintainers point you to. See the clarification below. Mantra: LangGraph is LangChain's answer, not its rival.

If you remember one sentence from this page: the durability requirement is the only thing that actually forces your hand. Everything else is a taste and line-count argument, and taste arguments are cheap to reverse.

The scorecard

How to read this table

The speedup row is not a ranking. Those numbers are wall-clock time against the sum of the individual agent spans (wall_ms vs serial_ms) from real runs against a live model. Three researchers running perfectly in parallel cap out near 3×; all three land between 2.0× and 2.6×, and the spread is dominated by how long DeepSeek happened to take on a given afternoon, not by framework overhead. Read the row as a single finding — all three parallelize fine — and be suspicious of anyone who reads it as a leaderboard.

The LOC row is the ceremony tax. It counts only the code you write to make the orchestration happen: state or event definitions, agent or node declarations, graph wiring, and the run_research wrapper. Thirty lines is not a real cost on its own — but it is a proxy for how much of the framework's model you have to hold in your head while debugging, and that cost is real at minute 45 of a 60-minute build.

The footgun row is the one that bites at 2am. Each framework has exactly one failure mode that is invisible until it isn't: the SDK gives you nothing durable and you find out when the process dies; LangGraph strands a superstep when a branch raises; LlamaIndex waits forever for an event a crashed step never sent. Every one of those is a silent failure — no stack trace at the place where you made the mistake. That is why the deep dives below tell them as stories rather than listing them as bullets.

One system, three shapes

The whole comparison rests on one small system, rebuilt three times. It is worth understanding precisely, because every framework claim below is a claim about how that system's shape changed.

The pipeline: a planner (the "pro" model tier) breaks a topic into exactly three independent research queries. Each query goes to a researcher (the cheaper, faster "flash" tier) running concurrently with the others. A synthesizer waits for all three, then writes a three-sentence executive summary from whatever survived.

Held constant across all three implementations: the same DeepSeek models, the same retry_transient exponential-backoff helper, the same Timeline class recording when each agent started and finished, the same OpenAI fallback path when DeepSeek returns a 429, and the same parametrized pytest that stubs the model layer and asserts peak concurrency reached exactly 3.

Same thirty lines, three accents

Strip away the timing and retry scaffolding and the interesting part of each implementation is tiny. Fan-out — the moment one query becomes three concurrent agents — is where the three frameworks disagree most visibly.

OpenAI Agents SDK — fan-out is a gather call.

semaphore = asyncio.Semaphore(MAX_CONCURRENCY)

async def research(query: str) -> str:
    async with semaphore, timeline.track(query):
        result = await run_agent(researcher, f"Research: {query}...", emit)
        return result.final_output

results = await asyncio.gather(*(research(q) for q in queries), return_exceptions=True)

LangGraph — fan-out is a list of Send objects.

def fan_out(state: State) -> list[Send]:
    """One Send per query → one concurrent researcher node each."""
    return [Send("researcher", {"query": q}) for q in state["research_queries"]]

graph.add_conditional_edges("supervisor", fan_out, ["researcher"])

LlamaIndex Workflows — fan-out is emitting events.

for query in queries:
    ctx.send_event(ResearchQueryEvent(query=query))
return None  # events were dispatched by hand

Three lines, three worldviews. In the SDK, concurrency is something you do with the standard library and the framework never learns about it. In LangGraph, concurrency is something you declare to the graph, which then owns scheduling, merging, and checkpointing it. In LlamaIndex, concurrency is a consequence of messages in flight: you post envelopes and a worker pool decides how many get opened at once.

Everything else in this page — the failure semantics, the footguns, the durability story — falls out of that one difference.

OpenAI Agents SDK — least ceremony, least framework

Mental model: asyncio with business cards

An Agent is a name, a model, and an instruction string. That's the whole abstraction. It does not own an event loop, a state store, or a scheduler; it is a business card you hand to a runner. The framework's job is to turn that card plus an input string into a model call and give you back .final_output.

Which means: if you already know asyncio, you already know this framework's concurrency model, because it is asyncio. There is nothing to learn about fan-out here, and nothing the framework can do for you that gather doesn't already do.

planner = Agent(
    name="Planner",
    model=PRO_MODEL,
    instructions=(
        "Break the topic into exactly 3 independent research queries. "
        "Return 3 lines, no numbering, no preamble."
    ),
)

researcher = Agent(
    name="Researcher",
    model=FLASH_MODEL,
    instructions="You are a fast research agent. Answer concisely with facts only...",
)

Fan-out anatomy

The real fan-out from the implementation, semaphore and all:

semaphore = asyncio.Semaphore(MAX_CONCURRENCY)

async def research(query: str) -> str:
    # Tracking starts after the semaphore, so a queued agent shows as a
    # later bar in the gantt instead of a falsely long one.
    async with semaphore, timeline.track(query):
        result = await run_agent(
            researcher, f"Research: {query}. Return 2-3 concise bullet points.", emit
        )
        return result.final_output


results = await asyncio.gather(*(research(q) for q in queries), return_exceptions=True)

Two details in there are worth stealing regardless of framework. The semaphore is acquired outside the timing context, so an agent that spent four seconds queued doesn't report a four-second run. And return_exceptions=True is what converts "one branch died" from a run-ending event into a data point.

Delegation: agents as tools

Parallel fan-out is one shape; sequential delegation is the other. The SDK handles it by letting an agent become a tool on another agent, so a manager decides at runtime when to call it:

manager = Agent(
    name="Manager",
    model=PRO_MODEL,
    instructions=(
        "You orchestrate a team. Delegate code generation to the builder tool, "
        "then send the output to the reviewer tool..."
    ),
    tools=[
        builder.as_tool(tool_name="builder", tool_description="Generate code"),
        reviewer.as_tool(tool_name="reviewer", tool_description="Review code"),
    ],
)

This is genuinely elegant: the routing logic lives in a prompt rather than in a graph, so adding a third specialist is one line plus one sentence. It is also the trade-off in miniature — the control flow is now inside the model, where you cannot single-step it.

Failure semantics, as a story

Three researchers go out. One of them hits a provider error that survives all three retry attempts and raises. Because gather was called with return_exceptions=True, that exception comes back as a value in the results list rather than propagating. The run continues.

Then you zip the queries back against the results and partition:

findings, failures = [], []
for query, result in zip(queries, results):
    if isinstance(result, BaseException):
        failures.append({"query": query, "error": str(result)})
    else:
        findings.append({"query": query, "result": result})

The synthesizer runs on findings alone, and the response carries the failures list so the UI can say "2 of 3 agents reported." A dead branch costs you one finding, not the run. That is the whole failure story, and you wrote all of it — which is precisely the point: no framework semantics to learn, no framework semantics to be surprised by.

The provider tax, as a story

You point this SDK at DeepSeek instead of OpenAI. Nothing works, and the reasons arrive one at a time.

First, the client: the SDK has a global default, so you replace it. Then the API surface — the SDK prefers OpenAI's Responses API, which non-OpenAI providers generally do not serve, so you force Chat Completions. Then tracing: left alone, the SDK cheerfully exports traces to platform.openai.com using a key you may not even have configured.

set_default_openai_client(AsyncOpenAI(base_url=DEEPSEEK_BASE_URL, api_key=deepseek_key()))
set_default_openai_api("chat_completions")  # provider doesn't serve the Responses API
set_tracing_disabled(True)                  # else traces export to platform.openai.com

Three global mutations at import time. They are three lines and you write them once, so this is a papercut rather than a wound — but notice the shape of it: these are process-global settings, not per-agent configuration. In a codebase where one service talks to two providers, that shape starts to matter.

What's missing

No checkpointing. No pause/resume. No state merge. No built-in concurrency cap — the Semaphore(4) above exists because the direct DeepSeek API rate-limits harder than a gateway would, and nothing in the framework was going to tell you that. Retries, backoff, failure accounting, and progress events are all code you own.

For a stateless fan-out that is a feature: less framework means fewer abstractions between you and the bug. For anything that must survive a restart, it is a wall, and the only way through is to build the durability yourself — at which point you have started writing LangGraph.

Pick it when: the task is stateless fan-out or prompt-routed delegation, you're on OpenAI (or willing to pay the setup tax once), and you want the smallest possible distance between a blank file and a running demo.

Walk away when: the requirement includes an approval gate, a resumable long-running job, or an audit trail of intermediate state.

Key Takeaway The OpenAI Agents SDK is asyncio with agent-shaped business cards. It gives you declarations and a runner; concurrency, limits, retries, and durability stay yours. That is the cheapest possible framework — and the ceiling is exactly where "must survive a restart" begins.

LangGraph — the only one with a durability story

Mental model: a workflow engine with a save-game file

LangGraph is a state machine that persists. You declare a state schema, write nodes that return partial state, and connect them with edges. The runtime executes in supersteps: all nodes scheduled in a step run, their partial states are merged through reducers, and the merged result is written to a checkpointer before the next step begins.

That checkpoint-per-superstep is the entire reason to choose this framework. A graph can stop mid-run — deliberately, or because the process died — and start again from the last save point with its state intact. None of the alternatives here can do that at all.

The cost is that you now think in state, not in variables. Every value a later node needs must be in the state schema, and every value two branches both write must have a reducer saying how to combine them.

The state schema and its reducers

class State(TypedDict, total=False):
    topic: str
    research_queries: list[str]
    findings: Annotated[list[dict], operator.add]   # branches merge via +
    failures: Annotated[list[dict], operator.add]
    synthesis: str
    approved: bool

Read Annotated[list[dict], operator.add] as: when several branches return this key in the same superstep, concatenate the lists instead of fighting over who wins. Without the annotation, concurrent writes to the same key are a conflict; with it, fan-in is automatic and you never write a join.

This is why nodes return partials. A researcher branch returns only {"findings": [...]} — one dict, one key — and the runtime merges three of those into a six-item list without any code from you.

Fan-out with the Send API

def fan_out(state: State) -> list[Send]:
    """One Send per query → one concurrent researcher node each."""
    return [Send("researcher", {"query": q}) for q in state["research_queries"]]

graph.add_conditional_edges("supervisor", fan_out, ["researcher"])
graph.add_edge("researcher", "synthesizer")

Each Send is "run this node with this payload." Three Sends means three concurrent invocations of one node definition, each seeing a different input — map-reduce, expressed as edges. Note that Send payloads are not the full state: the researcher receives {"query": q}, does its work, and returns a partial that gets merged back.

The interrupt story

Here is the capability nothing else in this comparison has. A node calls interrupt():

def human_review(state: State) -> State:
    """Pause the graph. The checkpointer makes the pause survive the process."""
    answer = interrupt(f"Approve synthesis?\n{state['synthesis']}\n(y/n): ")
    return {"approved": str(answer).lower().startswith("y")}

At that call, the graph parks. Its full state is in the checkpointer, keyed by a thread id. Minutes or days later — from a different process, triggered by a button in a web UI — you resume it:

async for _ in app.astream({"topic": topic}, config):
    pass

async for _ in app.astream(Command(resume="y"), config):
    pass

state = await app.aget_state(config)   # inspect everything it accumulated

Swap MemorySaver for AsyncSqliteSaver and that pause survives a restart: the process can die between the two astream calls and the graph picks up where it stopped. aget_state makes the whole accumulated state inspectable after the fact, which is also the best debugging story of the three.

That is what the extra ~30 lines buy. If your requirements never say "approve," "resume," or "audit," you are paying for a save-game feature you will never load.

Footgun one: an exception strands the superstep

You write the researcher node the obvious way — call the model, return the finding — and it works. Then one branch hits a real error and raises. The superstep does not complete. The merge never happens. The synthesizer never runs. One bad branch takes the whole graph down, which is exactly the failure mode return_exceptions=True prevented in the SDK version.

The fix is a discipline: inside a graph, failure is data, never an exception.

async def researcher(task: ResearchTask, config) -> State:
    """One parallel branch. Failures become state, not exceptions, so a dead
    branch still completes its superstep and the graph reaches the synthesizer."""
    timeline: Timeline = config["configurable"]["timeline"]
    query = task["query"]
    try:
        async with timeline.track(query):
            resp = await llm_call(
                "flash",
                lambda llm: llm.ainvoke(f"Research: {query}. Return 2-3 concise bullet points."),
            )
        return {"findings": [{"query": query, "result": resp.content}]}
    except Exception as exc:
        return {"failures": [{"query": query, "error": str(exc)}]}

Note that failures has its own operator.add reducer for exactly this reason. Every node in a fan-out should end in that except clause. Once you internalize "convert failure to state," LangGraph stops surprising you.

Footgun two: state goes through a serializer

The second one is subtler and the error message is worse. Your Timeline object needs to be reachable from inside every researcher branch, so the obvious move is to put it in state. Don't. State passes through the checkpointer's serializer, and a live object — a logger, an emit callback, an open timer — is not serializable.

Runtime objects ride in config, which is per-invocation and never persisted:

config = {
    "configurable": {
        "thread_id": f"research-{uuid.uuid4().hex[:8]}",
        # Config is runtime-only, so the Timeline object never hits the
        # checkpointer's serializer — that's why it rides here, not in State.
        "timeline": timeline,
        "emit": emit,
    }
}

The rule to carry: state is what you'd want to see in a database row; config is what only makes sense inside this process. The thread_id in there is also what makes resume possible — it's the key the checkpointer files your graph under.

The weight

It is the heaviest install of the three. The DeepSeek integration (langchain_deepseek) dragged in transformers as a transitive dependency — a multi-hundred-megabyte ML library, for a project that never runs a local model. On a laptop it's a slow uv sync; in a container image it's a real line item.

Pick it when: the words "approval," "resume," "audit," or "long-running" appear in the requirements. That is the moment the extra lines and the extra weight pay for themselves, and no competitor is close.

Walk away when: the job is a stateless fan-out that finishes in twelve seconds. You'd be buying a save-game file for a game with no levels.

Key Takeaway LangGraph is the only one of the three that can pause, crash, and resume. The price is thinking in serializable state with explicit reducers, and two disciplines: convert every branch failure into state, and keep live runtime objects in config, never in state.

LlamaIndex Workflows — typed events as the contract

Grounded deep dive on the engine itself: LlamaIndex Workflows.

Mental model: a typed pub/sub mailroom

A workflow is a class of @step methods, and steps never call each other. A step receives one event type and returns one or more event types; the runtime reads those return annotations and uses them as the routing table. Write a step that returns ResearchResultEvent, and any step that accepts a ResearchResultEvent is now downstream of it. You never draw an edge.

Think of it as a mailroom with typed envelopes. Steps post mail and receive mail; the type on the envelope is the address. The consequence is that the type signatures are the architecture diagram — read the annotations top to bottom and you have the whole flow, which is the single best property of this framework when you have to explain a design out loud.

Events are the wire format

class ResearchQueryEvent(Event):
    query: str


class ResearchResultEvent(Event):
    query: str
    result: str = ""
    error: str = ""

Two small classes, and they carry the entire contract between steps. Notice error is a field, not an exception path — that design decision is load bearing, and the footgun section below is why.

The steps, end to end

class ParallelResearchWorkflow(Workflow):
    @step
    async def planner(self, ctx: Context, ev: StartEvent) -> ResearchQueryEvent | None:
        """Fan out: one event per query, each picked up by a researcher worker."""
        queries = [...]                      # from the pro-tier model
        await ctx.store.set("num_queries", len(queries))
        for query in queries:
            ctx.send_event(ResearchQueryEvent(query=query))
        return None  # events were dispatched by hand

    @step(num_workers=NUM_WORKERS)
    async def researcher(self, ctx: Context, ev: ResearchQueryEvent) -> ResearchResultEvent:
        """num_workers caps how many of these run at once. A failure still emits
        its event — otherwise collect_events waits out the whole workflow timeout."""
        try:
            async with self.timeline.track(ev.query):
                resp = await llm_call("flash", lambda llm: llm.acomplete(f"Research: {ev.query}..."))
            return ResearchResultEvent(query=ev.query, result=resp.text)
        except Exception as exc:
            return ResearchResultEvent(query=ev.query, error=str(exc))

    @step
    async def synthesizer(self, ctx: Context, ev: ResearchResultEvent) -> StopEvent | None:
        """Fan in: returns None until every ResearchResultEvent has arrived."""
        num_queries = await ctx.store.get("num_queries")
        events = ctx.collect_events(ev, [ResearchResultEvent] * num_queries)
        if events is None:
            return None
        ...

The concurrency cap is one argument

@step(num_workers=3) is the whole story. Compare it to the SDK version, where the same guarantee cost you a module-level Semaphore, a MAX_CONCURRENCY constant, and an async with in the right place — and where getting the placement wrong silently corrupts your timing measurements. Here the cap is declarative, attached to the step it governs, and visible in the same line as the step's name.

This is the clearest example of what the ~20-line LOC advantage is made of: not one dramatic saving, but half a dozen small things the framework already knows how to do.

The fan-in barrier: counting heads before the bus leaves

collect_events is the piece worth understanding precisely, because it looks like a function call and behaves like a synchronization primitive.

events = ctx.collect_events(ev, [ResearchResultEvent] * num_queries)
if events is None:
    return None   # that None IS the fan-in barrier

The synthesizer step is invoked once per arriving event. On the first two calls, collect_events stashes the event and returns None, so the step returns None and nothing happens. On the third, it returns the full list and the step proceeds. It is a chaperone counting heads before the bus leaves: called every time someone boards, does nothing until the count matches.

Scratch state lives in ctx.store — that's where num_queries came from, written by the planner and read by the synthesizer. It's the workflow's key-value side table for anything that isn't event payload.

The footgun, as a story

You add a retriever to the researcher step. It throws on a malformed query. Your try/except isn't there yet, so the step raises instead of returning its event.

Now count the heads: two ResearchResultEvents arrived, the barrier wants three, and the third will never come. collect_events returns None forever. The synthesizer never fires, the workflow never stops, and it sits there until the 180-second timeout expires — and then fails with a timeout error that says nothing about the step that actually broke.

This is the sharpest failure mode of the three precisely because it is silent and misattributed. The fix is a discipline you adopt once and never revisit: every step returns its event, even on failure, with the error carried inside the event. That is why ResearchResultEvent has an error field, and why the synthesizer partitions rather than assumes:

findings = [{"query": e.query, "result": e.result} for e in events if not e.error]
failures = [{"query": e.query, "error": e.error} for e in events if e.error]

Every framework here demands the same lesson in a different dialect: in parallel orchestration, failure must become data. LangGraph strands a superstep; LlamaIndex deadlocks a barrier; the SDK needed return_exceptions=True. Same lesson, three punishments.

Where it wins beyond line count

It is LlamaIndex. Bolting a retriever onto a step is native rather than an integration — the index, the query engine, and the node postprocessors are all first-party. If there is any chance retrieval ends up inside the agent loop, this is the framework where that change is additive rather than architectural.

Where it's weaker

Checkpointing exists in the ecosystem but is not first-class the way LangGraph's is; there is no equivalent of "park on an interrupt, resume in another process next week." And progress reporting here rode a side channel — the Timeline object passed into the workflow's constructor — rather than a built-in stream, so wiring per-agent events to a UI is your code.

Pick it when: you want typed, inspectable orchestration in the least code, and especially if retrieval is anywhere in the prototype's future.

Walk away when: the requirement is durable pause/resume, or you need framework-native streaming of intermediate progress.

Key Takeaway LlamaIndex Workflows routes on types: steps post typed events and the return annotations are the wiring. You get a declarative concurrency cap and a counting fan-in barrier nearly for free — provided every step always emits its event, because a step that raises leaves the barrier counting forever.

LangChain vs LangGraph — not the same decision

"LangChain or LangGraph" is a version question, not a choice.

LangChain popularized chains, prompt templates, and tool abstractions, and for a while it was the default answer to "how do I build an agent in Python." For agents, its own maintainers now route you to LangGraph, and the legacy AgentExecutor is described as deprecated in its docs. The agent loop moved; it did not fork.

What survives of LangChain in a modern stack is the integration layer — model wrappers, tool adapters, output parsers. That layer is genuinely useful and very much alive: the LangGraph implementation in this comparison imports langchain_deepseek and langchain_openai for its model clients and never touches a chain. You are using LangChain either way; you are just using the part of it that ages well.

So if someone asks you to compare them in an interview, the correct move is to refuse the framing politely and then explain it: they are the same project's past and present answer to agent orchestration. If you're choosing an orchestrator today, LangChain proper isn't on the menu — LangGraph is its answer to this problem, and the comparison that matters is LangGraph against everything else on this page.

The wider field

Three rebuilds was the budget. These didn't get one, which is a statement about the experiment's scope and not a verdict on the tools. Here is the honest version of what each is for.

CrewAI models the problem as a crew: agents with roles, goals, and backstories, plus tasks assigned to them. The ergonomics are genuinely delightful for demos — you describe a team in something close to plain English and it runs. The trade is control: when you need a precise fan-in (wait for exactly these three, merge them this way, and here is what happens if one fails), you are working with the abstraction rather than through it. Excellent for a convincing prototype in an hour; check it carefully against your determinism requirements before it carries production traffic.

AutoGen (Microsoft) is built around multi-agent conversation: agents talk to each other in group chats, with termination conditions and speaker-selection policies deciding who goes next. When the problem genuinely is a negotiation or a debate — reviewer arguing with author until convergence — it is the most natural fit on this page. For a straight three-way fan-out it is a heavier conceptual model than the job needs, and heavier concepts cost you when debugging.

Pydantic AI applies Pydantic's type-safety instincts to agents: typed dependencies, typed results, validation as a first-class step rather than a bolt-on. If your team already thinks in Pydantic models, it will feel immediately correct, and its structured-output story is among the best. Its graph/durable-execution story is younger than LangGraph's — that's the gap to check against your requirements.

smolagents (HuggingFace) bets on code as action: instead of emitting JSON tool calls, the agent writes Python and the runtime executes it. For research and data-wrangling tasks that is a remarkable fit — one snippet can do what five tool calls would — and the whole thing is small enough to read in an afternoon. It is not aimed at production orchestration, and executing model-written code brings a sandboxing requirement you must answer before you ship.

Google ADK is a Gemini-centric agent kit with strong ties to Vertex AI and the rest of Google Cloud. If your models, your deployment, and your evaluation tooling already live there, the integration story is the point and it is a strong one. If they don't, you are choosing a framework and a cloud at the same time.

Rolling your own with asyncio is always a legitimate baseline, and it deserves saying loudly: the OpenAI Agents SDK implementation in this comparison is about 80% plain asyncio anyway. gather, a Semaphore, a retry helper, and a dataclass will carry a surprising amount of production traffic. Adopt a framework when it gives you something you'd otherwise have to build — durable state, typed routing, a scheduler — not because "agent system" sounds like it needs one.

None of these are disqualified. They just weren't worth a fourth rebuild of the same app.

The decision tree

The rule the tree encodes: start from the requirement, not the framework. Notice that the first question is the only one that can force an answer. Durability is a capability the other two don't have, so if the requirement says "approve before publishing" or "survive a deploy mid-run," the decision is made and the remaining questions are moot. Everything below that node is a preference — typed routing versus minimum ceremony — and preferences are cheap to change later, because the app behind the contract is identical either way.

That is also the practical argument for putting your orchestration behind a signature like run_research(topic, emit) on day one. It cost nothing, and it is what made this comparison possible at all: three implementations, one caller, swappable by name.

And whatever you pick, measure the parallelism. Record when each agent started and finished, sum the spans, divide by wall-clock time:

speedup = timeline.serial_ms / timeline.wall_ms   # 2.0×–2.6× across all three

Overlapping bars in a gantt are the receipt. A framework's marketing page is not — and "we run the agents in parallel" is a claim that costs about fifteen lines to turn into a fact.

Method notes

How much to trust the numbers above:

• All three implementations expose the same run_research(topic, emit) signature and emit the same events (planned / started / done / failed), so a single SSE endpoint serves any of them by name. That uniformity is what keeps the comparison honest — the caller cannot tell them apart.
• A shared pytest suite runs identical assertions against all three via pytest.parametrize, stubbing each framework's model layer and asserting peak concurrency reached exactly 3. "These agents actually run in parallel" is a test, not a vibe.
• Speedups are wall-clock against summed agent spans from real runs against a live model, so they include provider latency variance. Treat 2.0×–2.6× as one finding with noise, not as a ranking of three frameworks.
• LOC figures count only framework-specific orchestration — state or event definitions, agent or node declarations, graph wiring, and the run_research wrapper — after subtracting the ~105 lines of timing and retry helpers deliberately duplicated across all three packages.
• The footguns are observed, not hypothetical: each one cost real debugging time during the builds, which is exactly why they're written as stories rather than as a list of warnings nobody reads.