Why your Temporal/LangGraph agent can’t really “resume”: the 3-layer architecture teams are converging on

Your agent can’t “resume” reliably because you don’t actually have a state model; you have a loop with side effects. Workflow engines are excellent at retries and timers, but when you try to make a non-deterministic agent loop durable, you either lose debuggability or blow up workflow history.

The failure mode: replay is not resume for LLM loops

Most workflow engines (Temporal is the clearest example) give you durability by replaying deterministic workflow code from an event history. That’s the whole trick: the worker dies, the history is replayed, and the workflow rebuilds the same in-memory state and emits the same commands. If your workflow code is not deterministic, you get a nondeterminism error rather than a clean “resume”. (community.temporal.io)

Agent loops are intrinsically non-deterministic:

• The LLM output can differ on each call, even with the “same” prompt.
• Tool calls can hit changing external state (web, tickets, pricing, feature flags).
• You may have concurrency and human-in-the-loop signals arriving at awkward times.

So teams try the obvious thing: “put the LLM call in an activity”. Temporal even spells out why: calling an LLM inside workflow code will re-execute on replay and produce a different result, which breaks determinism. (go.temporal.io)

That does avoid nondeterminism errors, but it immediately introduces a different problem: you still don’t have a clean state model for the agent. You have a pile of activities and workflow history that’s doing double duty as:

• the durable record of what happened,
• the debugging trace,
• and, accidentally, the agent’s “memory” of what it already did.

Those are not the same thing.

Why this gets painful in practice

Practitioners keep landing on the same two real-world constraints:

1) History is not a free audit log

Temporal workflow histories have practical limits, and history growth is a known long-run failure mode. If you model every tool call, every token stream, every observation, and every intermediate thought as history-bearing events, the workflow becomes harder to operate and can eventually hit history limits unless you aggressively use patterns like Continue-As-New. (keithtenzer.com)

The important part isn’t the exact threshold; it’s the shape of the failure:

• You “do the right thing” for observability (fine-grained activities, rich payloads).
• The workflow history grows with each loop iteration.
• You end up engineering around the workflow engine’s storage model rather than around the agent’s correctness model.

2) “Resume” is a data problem, not a control-flow problem

Retries are well-defined: you rerun an activity with the same inputs until it succeeds.

Resume is messier: you need to know what has already been committed to the outside world (emails sent, tickets filed, DB rows written), what outputs were observed (HTTP responses, tool results), and what decisions were made (which tool was chosen, which plan branch was taken). A crashed worker doesn’t just need a call stack; it needs a ledger.

You can see this exact tension in the field: one common workaround is to avoid “replay-the-code” recovery and instead “read-the-recorded-facts-and-resume”, but then you’re explicitly building a separate execution record. (reddit.com)

The 3-layer pattern: loop, durability substrate, workflow orchestrator

The architecture I now recommend (and increasingly see others converge on) is a strict separation into three layers:

1. Agent loop runtime (non-deterministic)
2. Durability substrate (append-only facts + checkpoints)
3. Workflow orchestrator (deterministic business process)

It’s not “more abstraction”. It’s carving the system at the seams where failure actually happens.

Layer 1: The agent loop (where non-determinism is allowed)

This is the code that:

• builds prompts,
• asks the model,
• selects tools,
• interprets results,
• and decides whether to iterate again.

Frameworks like LangGraph are explicitly designed around this reality: graphs with loops, typed state, conditional routing, and checkpointing so variability in model outputs isn’t conflated with orchestration bugs. (langchain.com)

Key design choice: treat the loop as a state machine, not as a long function.

Concretely, define:

• a serialisable state object (what the agent “knows” for this run),
• a set of steps/nodes (LLM call, tool call, scoring/gating, human approval),
• and an explicit transition rule.

If you can’t serialise the state, you can’t resume it.

Layer 2: The durability substrate (what “resume” actually reads)

This is the missing piece in most “just use Temporal” attempts. You need a persistence layer whose job is to record facts about execution in a way you can reliably resume from.

A practical durability substrate typically provides:

• Append-only event log of “facts” (model output X, tool invocation Y, tool result Z).
• Checkpoints/snapshots of the agent state every step (or every super-step), so resume doesn’t mean replaying from the beginning.
• Idempotency keys and “exactly-once-ish” guards for side effects (“did we already create ticket #123?”).
• Concurrency control so you don’t get zombie writes when a run is resumed twice.

LangGraph’s checkpointer concept is a clean example of this direction: it checkpoints graph state after node execution, keyed by a thread/run identifier, to enable interruption, inspection and recovery. (langchain.com)

The important architectural move is: the ledger/checkpoints are not the workflow engine history. They are your system-of-record for resuming an agent run.

When people say “resume is harder than retry”, this is usually what they mean.

Layer 3: The workflow orchestrator (where determinism buys you operations)

Once the agent has its own durability model, workflow engines become what they’re best at:

• timers and timeouts,
• retries with backoff,
• human approval gates,
• parallel branches,
• business SLAs,
• and a visible execution trace for deterministic business logic.

Temporal’s constraints still matter here (determinism, versioning, Continue-As-New patterns), but you’re no longer forcing the agent loop itself to live inside those constraints. (github.com)

The workflow layer should orchestrate coarse-grained operations:

• “Run the agent until it reaches decision point A or times out.”
• “Wait for legal approval signal.”
• “If approved, apply changes; if rejected, open a review task.”

It should not attempt to represent every internal thought and tool selection as workflow history.

How the three layers interact (a concrete run)

Here’s a typical long-running agent scenario: “Draft a customer-facing change proposal, gather evidence, and open a PR, with human approval gates.”

1. Workflow starts a run and allocates a run_id.
2. Workflow calls “advance agent one step” (activity or service call).
3. Agent runtime executes one step: - loads latest checkpoint for run_id, - produces an LLM output, - writes an event (LLM output) and a new checkpoint, - possibly writes a tool invocation request.
4. Tool execution happens (could be inside the runtime, or a separate worker): - reads requested tool call, - executes with idempotency key, - appends result event, - updates checkpoint.
5. Agent advances until it emits a “need human approval” event.
6. Workflow blocks on a signal.
7. On approval, workflow triggers the next coarse-grained phase.

If anything crashes, “resume” means:

• agent runtime reads checkpoint + events,
• continues from the last committed step,
• without replaying the entire workflow history,
• and without depending on the model to regenerate the same output.

Trade-offs and sharp edges (where this is the wrong answer)

This pattern is not free. You are explicitly building a runtime.

Costs you are choosing

• You are implementing and operating a durability substrate (storage, migrations, retention, access controls).
• You need a schema for events/checkpoints and a compatibility story.
• You must design idempotency around every side effect.

Risks you are avoiding

• Workflow history becomes your “agent memory” by accident.
• Nondeterminism traps where the workflow can’t replay after a code change or environment change.
• Debugging that devolves into “why did replay pick a different tool this time?”

“Do not do this” cases

• If the agent task completes in seconds and you can rerun it end-to-end, do not build durability. Treat it like a normal request/response job.
• If you can model the process as a deterministic business workflow with bounded activities, do that. Temporal shines there.
• If you can’t define a serialisable state model for the agent, durability will be a mess regardless of tooling.

A pragmatic checklist before you pick Temporal-as-agent-runtime

If you’re evaluating Temporal (or any workflow engine) as the runtime for long-running agents, ask these in order:

1. Do we have an explicit agent state object we can snapshot and restore?
2. What is our unit of durability: every tool call, every loop iteration, or larger “super-steps”?
3. Where do we store the durable record: workflow history, or a dedicated ledger/checkpoint store?
4. How do we guarantee idempotency for each side effect?
5. What is our strategy for history growth (if using a workflow engine for any fine-grained tracing)?
6. How will we debug a bad run: by replaying deterministic code, or by inspecting a factual execution record?

If you can’t answer (3) and (4), you don’t have “resume”. You have “hope the worker doesn’t crash”.

Durable agents are not “workflows with an LLM inside”. They’re state machines with an execution ledger, and workflows sit around them.

Where codeversols fits (briefly, and honestly)

If you’re at the 20–500 person stage, the decision is rarely “Temporal or not”; it’s “how much runtime do we build ourselves, and where do we want our operational complexity to live”. At codeversols we typically help teams formalise the state model, define the durability substrate (often Postgres-backed to start), and integrate it with an orchestrator where it genuinely adds value (Temporal, step functions, or a lighter queue), without forcing the agent loop into a determinism box.

Close

Retries are a control-flow feature. Resume is a state-and-ledger feature. Once you separate the non-deterministic loop from the durability substrate, workflow orchestration becomes straightforward again: timers, approvals, retries and visibility at the business-process level, while the agent remains resumable without pretending it can be replayed deterministically.