AI Agent Memory: Short-Term vs Long-Term Explained

The Two Types of AI Agent Memory

When you work with AI coding agents, there are two fundamentally different types of memory at play — and understanding the difference is essential for building reliable AI-augmented workflows.

Short-term memory (in-context): Everything the agent can "see" right now — the conversation history, files read, tool outputs, and instructions in the current session. Temporary and powerful.

Long-term memory (persistent): Everything that survives when a session ends — files on disk, external databases, or tool-specific memory systems. Permanent but requires infrastructure to maintain.

The key insight: current AI coding tools have excellent short-term memory but weak, inconsistent long-term memory. This gap is the root cause of context rot — the degradation of AI behavior between sessions.

Short-Term Memory: The Context Window

Short-term memory in AI coding agents is implemented as the context window — the bounded space of text that the model processes to generate each response.

The context window includes, in order of what was added first:

• System prompt (tool configuration, base instructions)
• Project context (CLAUDE.md content, if present)
• Conversation history (all messages, in order)
• Tool call inputs and outputs (files read, commands run)
• The current instruction

For Claude Code with Claude Sonnet 4.6, this window is 200,000 tokens. For GPT-4o in Cursor, 128,000 tokens. For Gemini 1.5 Pro, up to 1,000,000 tokens.

What Short-Term Memory Can Hold

At 200,000 tokens, a Claude Code session can hold approximately:

• ~150,000 words of conversation
• ~10,000–15,000 lines of source code
• A complete codebase of 20–30 medium-sized files simultaneously

How Short-Term Memory Fails

Despite these large windows, short-term memory fails in two ways:

Within-session degradation: As the context window fills, the model applies attention across more tokens. Early content receives less relative attention — it's technically present but less influential. Your architectural decisions from the first hour of a 6-hour session may still be in the window but have less impact on the model's behavior.

Session boundaries: When a session ends, the context window is discarded entirely. The next session starts with an empty window. This is the most common and impactful short-term memory failure.

Long-Term Memory: The Persistent Layer

Long-term memory is any mechanism that stores information outside the context window and survives session restarts. It must be explicitly designed and maintained — it doesn't happen automatically.

The spectrum of long-term memory solutions, from simplest to most complex:

Level 1: Context Files (CLAUDE.md / AGENTS.md)

The simplest long-term memory: a Markdown file the agent reads at session start.

Mechanism: The agent reads the file's content and loads it into the context window at session start. Scope: Whatever fits in the file (aim for under 600 words) Persistence: Unlimited (it's a file) Portability: High — works across Claude Code, Cursor, Cline, Windsurf Maintenance: Manual (or automated with capture tools)

Context files are long-term memory in the most straightforward sense: you write information down, the agent reads it later.

Level 2: Tool-Specific Memory Features

Cursor Memories, Windsurf Memories, Cline Memory Bank — semi-structured persistent storage that specific tools manage:

Mechanism: Tool maintains an internal store of facts; retrieves relevant ones at session start Scope: Typically larger than a single context file Persistence: In tool's storage system Portability: Tool-specific — doesn't transfer Maintenance: Semi-automatic (tool may suggest what to remember)

These are more automatic than plain context files but are locked to a specific tool. For multi-tool workflows, they create fragmentation.

Level 3: Vector Database Memory (Mem0, custom RAG)

Semantic memory stored as vector embeddings, retrieved based on relevance:

Mechanism: Information stored as embeddings; semantic search retrieves relevant items Scope: Unlimited (scales to millions of memories) Persistence: External database Portability: Requires integration per tool Maintenance: Automatic indexing; requires infrastructure

This is the most powerful option but requires the most infrastructure. Appropriate for large-scale use cases, teams with many projects, or building AI products.

The Memory Gap: Where Current Tools Fall Short

All current AI coding tools have strong short-term memory (large context windows, good within-session retention) but weak long-term memory (no automatic cross-session persistence).

"Partial" means the tool can suggest what to save but requires human confirmation. Nothing is fully automatic out of the box.

The practical implication: Even the best-equipped tool (Windsurf with auto-Memories) will miss decisions made in sessions where you don't explicitly trigger memory saves. The gap between "information the agent established this session" and "information the agent will have next session" is substantial.

Building a Complete Memory System

A complete memory system combines short-term and long-term memory deliberately:

For short-term (in-session):

• Start every session with an explicit context reload: "Read CLAUDE.md. Here's what we're working on today: [task]."
• Use /clear between unrelated tasks to prevent context accumulation
• Use /compact in long sessions to compress history and preserve context window space

For long-term (cross-session):

• Maintain CLAUDE.md / AGENTS.md for structured architectural decisions
• Use tool-specific memory features (Cursor Memories, Windsurf Memories) for personal preferences
• Enable automatic capture (Context Keeper) to keep context files current without manual discipline

The hybrid result: An agent that starts every session with your architectural decisions pre-loaded, has access to tool-specific preferences, and doesn't require 15 minutes of re-explanation before productive work can begin.

When Long-Term Memory Is "Good Enough"

For most individual developers on a single project: a well-maintained CLAUDE.md is sufficient long-term memory. It covers 95% of the context re-explanation problem with near-zero infrastructure.

The cases where you need more than a context file:

• Working across 5+ projects simultaneously
• Team of 10+ developers all using AI agents
• Building a product where users need personalized AI memory
• Need to query historical decisions semantically ("what did we decide about caching 3 months ago?")

For everything else: write the file, keep it current, and let the agent read it.

Key Takeaways

• Short-term memory = the context window (temporary, session-scoped, powerful but finite)
• Long-term memory = files, databases, or tool memory features (persistent, requires design)
• All current AI coding tools have strong short-term but weak, inconsistent long-term memory
• Three levels of long-term memory: context files (simple), tool-specific features (semi-auto), vector databases (scalable)
• A well-maintained CLAUDE.md solves 95% of the long-term memory problem for most developers
• For complete coverage: CLAUDE.md + automatic capture + tool-specific memory features
• The memory gap (decisions made but not persisted) is the root cause of context rot

Related: What Is Context Rot in AI Coding Agents · Mem0 vs File-Based Context for AI Agents · Context Compression Techniques for Long Agent Sessions