Self-evolving AI agents are no longer just tools that run a prompt. They are systems that get better the more you use them, and the reason is the memory and learning architecture sitting under the prompt. This guide breaks down the two main approaches, the three memory layers every serious agent uses, and how leading systems like Claude Code, OpenClaw and the Hermes agent actually implement them in production today.
The phrase "self-evolving" is overloaded, so it helps to set the boundary first. We are not talking about agents that retrain their weights on the fly. We are talking about agents that change what they know between sessions, without anyone editing a prompt by hand. That sounds small, but it is the difference between a chatbot you re-explain every Monday and a coworker who remembers last week's decisions.
The 30-second version
Two Approaches to Self-Evolving Agents
Most self-evolving designs fall into one of two camps. The names vary across papers, but the split is consistent.
Auto Agent: rewrite your own system file
The Auto Agent approach focuses on the agent rewriting its own instructions. It evaluates itself against a single system file and edits that file at the end of each run. It looks like a lightweight cousin of fine-tuning, except the underlying model and infrastructure stay frozen. Only the prompt changes. This is simple and easy to debug, but the system file gets bloated fast and conflicts compound.
Self-learning memory: write what you did, read it next time
The other approach is closer to how humans work. The agent writes the steps it took and the feedback it received into a memory store, and reads from that store on the next request. This is the in-context learning pattern, and it is the approach that actually ships in production. It scales further than rewriting a single file, because memory can be split, indexed and pruned.
The Three Layers of a Self-Learning Memory
A self-learning agent is built on three distinct memory layers. Each layer answers a different question and has a different access pattern.
| Layer | What it stores | When it loads |
|---|---|---|
| Facts and preferences | User profile, project rules, recurring constraints | Hot path on every request, with warm files pulled on demand |
| Skills | Reusable how-tos for a specific task or domain | When the agent recognizes the task type |
| History | Raw chat logs from past sessions | On explicit search or when context is missing |
Facts and preferences
This is the layer most projects start with. A small hot memory file always rides in the system prompt: the user's name, the stack they work in, the rules they care about. A larger warm memory sits next to it and gets pulled in only when the request hints at it. Splitting hot from warm is the single biggest win, because it keeps the system prompt small and the warm file growable.
Skills
Skills are the agent's domain know-how. A skill might be "how to write a Stripe webhook handler in this codebase" or "how to phrase a follow-up email for a sales lead". They are reusable, scoped, and only loaded when the agent recognizes the task. Without a skill layer, every new task starts from zero, which is why agents that only use a flat prompt feel like they never learn.
History
The last layer is raw chat history: every session, every step, every tool call. The agent does not load it by default; it searches it when memory is missing or when the user asks something like "what did we decide last week?". The hard part here is fast, relevance-aware search across a long log, not storing the log itself.
How Leading Systems Implement This
The pattern looks the same on the whiteboard, but the production implementations differ in interesting ways. Three systems are worth watching.
Claude Code: AutoDream and the move to three layers
Claude Code started with a single system file, the CLAUDE.md approach familiar to anyone who has set one up. Once that file got too large, the architecture moved to the three-layer split described above. The most interesting feature is AutoDream: after a session ends, a background process inspects the conversation, prunes stale facts, and reorganizes memory. The known weak spots are searching chat history while you are coding, and adding a brand new skill, both of which are still manual.
OpenClaw: search-first memory
OpenClaw pushes search to the center of the design. Memory files and a generic search tool are first-class, and the agent can query both the curated memory and the raw chat history with the same call. That makes recall noticeably better. The trade-off is that there is no autonomous background process that creates new skills or memory entries on its own; something or someone has to trigger it. If you are new to this stack, our OpenClaw 101 guide for new users walks through the building blocks.
Hermes Agent: async sub-agents that learn while you work
The Hermes agent goes further by removing the human trigger. It counts the steps the main agent takes; if ten steps pass without a new skill being created, a sub-agent kicks in and analyses the trial-and-error work, recording any repeating pattern as a new skill. The main loop is never blocked. The same pattern runs for general memory: every ten turns a memory review agent extracts preferences and constraints from the conversation and writes them back. New skills go through a safety scan before they are merged. The mobile-first version of this idea is covered in Hermes Workspace Mobile and agent orchestration on a phone.
The interesting move is not making the agent smarter at request time. It is making the system smarter between requests, with background processes the user never has to notice.
Do You Actually Need a Fully Autonomous System?
Full autonomy is fashionable, but it is not always the right trade-off. Background memory writers, sub-agents and review loops all burn extra tokens, and most of those tokens never affect the next user-facing answer.
- If your workflow is repetitive and well-understood, a deterministic pipeline often wins on cost and predictability.
- If your workflow has a long tail of edge cases, self-learning memory pays off because the agent gets to specialise without you writing prompts.
- If your workflow is interactive, the user is already a free signal source; a small memory update at the end of the session may be enough.
The orchestration question sits next to this one. Once you have a memory-aware agent, the next question is who calls it and when. We covered that side of the design in the Sakana Conductor multi-agent orchestration guide.
Building Your Own Self-Learning Agent
If you decide self-learning is worth it, the build order is boringly stable across stacks:
- Start with hot and warm memory files. No skills, no history search, just two files the agent reads from.
- Add a skill folder and let the agent load a skill only when the task name matches. Resist the urge to auto-create skills until the rest works.
- Log raw chat history to disk from day one, even if you cannot search it yet. You cannot recover history you never wrote down.
- Add a small background job that summarises each session into one or two facts and appends them to warm memory. This is the smallest useful version of AutoDream.
- Only after all of the above, consider a sub-agent that proposes new skills. Always run them through a review step before they go live.
What to measure
The Bottom Line
Self-evolving AI agents are not a science project anymore. The memory and learning architecture is converging on three layers (facts, skills, history), and the cutting edge in 2026 is automating the updates with background processes. Pick the amount of autonomy your workflow actually needs, build the layers in order, and measure reuse rather than raw activity. That is the path from a clever prompt to an agent that genuinely gets better over time.