[Blog](https://cohesivity.ai/blog)

Architecture

Published Jul 15, 2026·Updated Aug 30, 2026

![](https://cohesivity.ai/authors/arag.webp)![](https://cohesivity.ai/authors/anshu.webp)[Arag](https://www.linkedin.com/in/aragagrawal/) and [Anshu](https://www.linkedin.com/in/aanshuaggrawal120/)

# AI Agent Architecture: Memory, Tools, State, and Durable Execution

[Read as Markdown](https://cohesivity.ai/blog/ai-agent-architecture-memory-tools-state-durable-execution.md)

Agent

cohesivity

Tools

Execution

State

Durability

An AI agent architecture needs more than a model and a tool loop. A production design separates identity, tools, task state, long-term memory, durable execution, backend services, and telemetry because each layer owns a different failure.

The cleanest test is a crash between two external writes. The system should know who acted, what completed, what may have completed, and where to resume without asking the model to reconstruct reality from conversation text.

## Start with the execution path

```mermaid
flowchart TB
    Trigger[User or event] --> Loop[Model and agent loop]
    Loop --> Policy[Identity and policy]
    Policy --> Tools[Tool registry]
    Tools --> Durable[Durable execution]
    Durable --> Backend[Backend services]
    Loop <--> State[(Task state)]
    Loop <--> Memory[(Long-term memory)]
    Loop --> Telemetry[Traces, logs, metrics, evals]
    Tools --> Telemetry
    Durable --> Telemetry
    Backend --> Telemetry

```

The model proposes the next action. Policy decides whether that identity may take it. The tool layer translates the decision into a bounded contract. Durable execution records progress around external effects. Backend services hold authoritative business data. Telemetry lets a human reconstruct the run.

That sequence belongs inside the broader [production agent stack](https://cohesivity.ai/blog/the-emerging-ai-agent-stack), but drawing it for one real workflow is more useful than drawing every service in the company.

## Give each layer one kind of truth

### The model loop chooses, but does not authorize

The loop holds the prompt, recent messages, tool results, and stopping rules. It can choose a tool, but the downstream system must enforce permission. A prompt that says “only read invoices” is guidance, not an access-control boundary.

### Tools expose bounded actions

A tool contract names an operation, constrains its arguments, and returns the fields needed for the next decision. It should not expose a broad administrative API merely because the model can describe what it wants. The [MCP tool design guide](https://cohesivity.ai/blog/how-to-design-mcp-tools-agents-can-use-reliably) covers selection, schemas, stable errors, and safe writes.

### Identity and policy decide what may happen

Carry the tenant, user, agent, and delegated scope to the system performing the action. Policy checks should happen before a consequential call, and the audit record should preserve the decision. Never ask the model to provide a tenant ID that the server can derive from its credential.

### Task state records where this run is

Task state includes the current step, completed operations, pending approval, retry count, and next scheduled attempt. It should survive the process that created it. Conversation history is an input to a run, but it is a weak substitute for explicit state because it does not reliably encode transaction status.

### Memory carries selected facts across runs

Memory may include user preferences, prior summaries, or retrieved documents. It is fallible context, so keep provenance, retention, and deletion rules. Do not store an irreversible side effect only as a memory sentence such as “invoice sent.” That fact belongs in authoritative backend state.

### Durable execution survives interruption

Durable execution coordinates retries, checkpoints, waits, and resumes. Temporal defines it as execution that continues despite crashes or outages by replaying recorded event history. That model is useful even if you implement a smaller operation ledger instead of adopting Temporal. See [Temporal’s durable execution description](https://docs.temporal.io/encyclopedia/durable-execution) and the detailed [agent durability guide](https://cohesivity.ai/blog/durable-execution-for-ai-agents).

### Backend services own business facts

Orders, payments, files, permissions, and customer records live in systems designed to enforce their invariants. Agents should reach them through scoped APIs or tools. A model response can explain an order, but it must not become the canonical order record.

### Telemetry joins the path back together

Use one trace across the model call, tool selection, policy check, tool execution, and downstream request. Include operation IDs and redacted arguments. Keep logs for events, metrics for aggregate health, and evaluations for expected behavior. None can replace the others.

## Separate the failures before choosing products

If an agent repeats an email after a timeout, investigate durable execution and idempotency. If it remembers the wrong preference next week, investigate memory and provenance. If it reads another tenant’s record, investigate identity and authorization. If it chooses `delete_project` instead of `archive_project`, investigate the visible tool set and its selection tests.

Buying one “agent platform” does not remove these boundaries. It changes which component owns them. Record that ownership explicitly, including the plain database or queue that sits behind the product name.

## Draw the ugly workflow first

Do not begin with a read-only chat. Pick a workflow that writes twice, waits for approval, calls an unreliable API, or can spend money. Mark the authoritative record before and after every side effect. The prompt below turns that path into an architecture map and points out the layers that currently exist only in the model’s context window.

or send it to[Claude Code](https://claude.ai/new?q=Inspect+this+repository+and+choose+the+AI+agent+workflow+with+the+most+external+side+effects.+Do+not+edit+code+or+invoke+external+tools.+Trace+one+run+through+the+model+loop%2C+tool+registry%2C+identity+and+policy+checks%2C+task+state%2C+memory%2C+durable+execution%2C+backend+services%2C+and+telemetry.+Name+the+component+that+owns+each+layer%2C+the+data+crossing+each+boundary%2C+and+what+happens+after+a+crash+between+two+side+effects.+Mark+every+missing+or+ambiguous+owner%2C+then+suggest+the+smallest+architecture+change+for+the+three+highest-risk+gaps. "Send to Claude")[Cursor](https://cursor.com/link/prompt?text=Inspect+this+repository+and+choose+the+AI+agent+workflow+with+the+most+external+side+effects.+Do+not+edit+code+or+invoke+external+tools.+Trace+one+run+through+the+model+loop%2C+tool+registry%2C+identity+and+policy+checks%2C+task+state%2C+memory%2C+durable+execution%2C+backend+services%2C+and+telemetry.+Name+the+component+that+owns+each+layer%2C+the+data+crossing+each+boundary%2C+and+what+happens+after+a+crash+between+two+side+effects.+Mark+every+missing+or+ambiguous+owner%2C+then+suggest+the+smallest+architecture+change+for+the+three+highest-risk+gaps. "Send to Cursor")[Codex](https://chatgpt.com/codex?prompt=Inspect+this+repository+and+choose+the+AI+agent+workflow+with+the+most+external+side+effects.+Do+not+edit+code+or+invoke+external+tools.+Trace+one+run+through+the+model+loop%2C+tool+registry%2C+identity+and+policy+checks%2C+task+state%2C+memory%2C+durable+execution%2C+backend+services%2C+and+telemetry.+Name+the+component+that+owns+each+layer%2C+the+data+crossing+each+boundary%2C+and+what+happens+after+a+crash+between+two+side+effects.+Mark+every+missing+or+ambiguous+owner%2C+then+suggest+the+smallest+architecture+change+for+the+three+highest-risk+gaps. "Send to Codex")[opencode](https://opencode.ai/?q=Inspect+this+repository+and+choose+the+AI+agent+workflow+with+the+most+external+side+effects.+Do+not+edit+code+or+invoke+external+tools.+Trace+one+run+through+the+model+loop%2C+tool+registry%2C+identity+and+policy+checks%2C+task+state%2C+memory%2C+durable+execution%2C+backend+services%2C+and+telemetry.+Name+the+component+that+owns+each+layer%2C+the+data+crossing+each+boundary%2C+and+what+happens+after+a+crash+between+two+side+effects.+Mark+every+missing+or+ambiguous+owner%2C+then+suggest+the+smallest+architecture+change+for+the+three+highest-risk+gaps. "Send to OpenCode")