How AI Agent Memory Works: Making Agents That Learn

Without memory, every conversation starts from zero. Here's how to fix that.

The Memory Problem

Large language models have a fundamental limitation: the context window. Every conversation starts fresh. The model doesn't remember what you talked about yesterday, what decisions were made last week, or what your preferences are. It only knows what's in the current prompt.

Context windows are getting larger (100K+ tokens is common now), but they still have limits. And more importantly, stuffing everything into one prompt is expensive, slow, and noisy. You don't want your agent re-reading your entire project history every time it does a simple task.

This is the memory problem: how do you give an AI agent the knowledge it needs to be useful over time, without hitting context limits or drowning in irrelevant information?

File-Based Persistent Memory

The simplest and most effective pattern for agent memory is surprisingly low-tech: files on disk. No vector databases, no embedding pipelines, no complex retrieval systems. Just markdown files that the agent reads at the start of each session and updates as it learns.

project/
  memory/
    MEMORY.md        # Primary memory - always loaded
    preferences.md   # User preferences and style
    decisions.md     # Key decisions and their rationale
    projects.md      # Active projects and their status
    contacts.md      # People and relationships

Why This Works

• Simplicity. No infrastructure to set up or maintain. Files are files.
• Transparency. You can read the agent's memory anytime. It's just markdown. No black box.
• Version control. Memory files go in git. You get full history of what the agent knew and when.
• Selective loading. The agent reads MEMORY.md every session but only loads topic files when relevant. This keeps the context window lean.
• Self-updating. The agent can update its own memory files as it learns new information. Tell it to, and it will.

What to Store vs What to Derive

Not everything belongs in memory. The key distinction: store stable facts and patterns. Don't store things the agent can figure out from the codebase or derive from other data.

Memory Hygiene

Memory that grows without pruning becomes noise. Good memory hygiene means:

• Regular cleanup. Remove outdated information. That decision from three months ago that got reversed? Remove it or update it.
• Correct errors. When the agent stores something wrong, update the memory. Wrong memories are worse than no memories.
• Keep the index lean. MEMORY.md should be a quick-reference index, not an encyclopedia. Link to detail files.
• Topic separation. Don't put everything in one file. Separate by domain: preferences, projects, decisions, contacts.

Advanced Patterns

Session Summaries

At the end of each work session, the agent writes a brief summary of what happened, what was decided, and what's next. This file (like memory/last-session.md) gives the agent context on the very last conversation when it starts a new one.

Agent-Specific Memory

In a multi-agent system, each agent has its own memory directory. The research agent's memory is different from the finance agent's memory. They store domain-specific knowledge without polluting each other's context.

Shared Memory

Some information needs to be shared across agents: org-wide decisions, user preferences that apply everywhere, active project lists. A top-level memory/ directory serves as the shared knowledge base, while agent-specific directories handle domain knowledge.

project/
  memory/
    MEMORY.md              # Org-wide shared context
  agents/
    research-agent/
      memory/MEMORY.md     # Research-specific knowledge
    finance-agent/
      memory/MEMORY.md     # Finance-specific knowledge
    content-agent/
      memory/MEMORY.md     # Content-specific knowledge

The Bottom Line

Agent memory doesn't need to be complex. Files on disk, read at session start, updated as things change. The key is being deliberate about what goes in, keeping it organized, and cleaning it up regularly. An agent with good memory feels like a colleague who remembers your last conversation. An agent without memory feels like explaining your job to a new hire every morning.