Memory Systems for Agents

The Four Types of Agent Memory

In-context memory — the message history in the current conversation. Fast, zero latency, limited by context window size. Lost when the session ends.

External memory (episodic) — a database of past interactions, stored and retrieved as needed. Survives restarts. Requires retrieval logic.

Semantic memory — a vector database of concepts and knowledge, searched by similarity. Enables agents to "remember" information not in the current context.

Procedural memory — skills and workflows encoded in the system prompt or retrieved as needed. The agent's "how to do things" knowledge.

When Each Type Applies

Implementing Episodic Memory

async function loadRelevantMemory(userId: string, query: string) {
  const memories = await db.query.agentMemory.findMany({
    where: eq(agentMemory.userId, userId),
    orderBy: desc(agentMemory.createdAt),
    limit: 10,
  });
  return memories.map(m => m.content).join('\n');
}

Inject retrieved memories into the system prompt before each interaction.

The Memory Management Problem

More memory = more context = higher cost and latency. Memory management is the art of selecting what to remember and what to forget.

Strategies:

• Summarize old memories instead of storing verbatim
• Rank memories by relevance to current task
• Expire memories that haven't been accessed in N days
• Compress repetitive memories (merge "user likes Python" seen 20 times into one entry)