An agent without persistent memory is afflicted with permanent amnesia. Every user turn restarts the universe from scratch.
Conversely, an agent that blindly dumps every raw message into its prompt suffers from Context Exhaustion: latency doubles, costs skyrocket, and the model drowns in historical noise.
Here is how modern production systems structure hierarchical agent memory.
1. The Three Tiers of Agent Memory
┌────────────────────────────────────────────────────────────────────────┐
│ HIERARCHICAL AGENT MEMORY │
├────────────────────────────────────────────────────────────────────────┤
│ Tier 1: Working Memory (In-Context Scratchpad) │
│ - Current turn, immediate tool outputs, active task plan │
├────────────────────────────────────────────────────────────────────────┤
│ Tier 2: Episodic Memory (Structured Historical Events) │
│ - User preferences, previous errors, completed milestones │
├────────────────────────────────────────────────────────────────────────┤
│ Tier 3: Semantic Memory (Vector / Graph Retrieval) │
│ - Long-term company knowledge base, documentation, past year's archive │
└────────────────────────────────────────────────────────────────────────┘
2. The Compaction and Extraction Loop
Never store raw conversation text in your long-term database. Run an asynchronous memory extraction worker after every completed user session:
from pydantic import BaseModel
from pydantic_ai import Agent
class UserMemoryFact(BaseModel):
category: str # e.g. "preference", "technical_stack", "constraint"
fact: str # e.g. "Customer uses Postgres 16 with TimescaleDB"
importance_score: int
memory_extractor = Agent(
"google-gla:gemini-2.5-flash",
result_type=list[UserMemoryFact],
system_prompt="Extract enduring facts and user preferences from this completed interaction."
)
These distilled facts are saved to PostgreSQL or Redis. When the user returns next week, the agent hydates with only the top 5 relevant facts—consuming 150 tokens instead of 30,000.
Get Weekly AI Architect Cost & Strategy Updates
Join 14,000+ developers receiving weekly, data-driven cost-reduction blueprints and production-ready agent guidelines.
